Multi-Tool Integration

ostruct provides integration with multiple OpenAI tools and services, allowing you to combine different capabilities in a single workflow. This guide covers how to use Code Interpreter, File Search, Web Search, and MCP servers effectively.

Overview

ostruct supports four main tool categories:

  • Code Interpreter: Execute code, analyze data, generate visualizations

  • File Search: Semantic search across uploaded documents

  • Web Search: Access current information from the internet

  • MCP Servers: Connect to Model Context Protocol servers for custom tools

All tools can be used individually or combined in the same request for comprehensive analysis workflows.

Automatic File Routing

ostruct can automatically route files to appropriate tools based on file type detection:

Auto-routing Target: Use --file auto:alias file.txt to let ostruct decide:

  • Text files β†’ Template access (prompt)

  • Binary files β†’ User-data for vision models (ud)

File Type Detection:

  • Enhanced detection: Uses machine learning (Magika) for accurate file type identification

  • Extension fallback: Recognizes 30+ common file extensions (.txt, .md, .json, .py, .csv, etc.)

  • Installation: pip install ostruct-cli[enhanced-detection] for enhanced detection

Example:

# Auto-route based on file type
ostruct run analysis.j2 schema.json \
  --file auto:doc1 report.pdf \      # β†’ user-data (binary)
  --file auto:doc2 summary.txt \     # β†’ prompt (text)
  --file auto:doc3 data.json         # β†’ prompt (text)

Shared File Usage

One of ostruct’s most powerful features is the ability to provide the same file to multiple tools simultaneously. This enables comprehensive analysis workflows where a single file can be:

  • Programmatically analyzed with Code Interpreter

  • Semantically searched with File Search

  • Referenced in templates for context

Multi-Tool File Syntax:

# Same file available to both Code Interpreter and File Search
ostruct run analysis.j2 schema.json --file ci,fs:data report.csv

# Multiple files with different tool combinations
ostruct run analysis.j2 schema.json \
  --file ci,fs:financial quarterly_report.pdf \
  --file ci:sales sales_data.csv \
  --file fs:policies company_handbook.pdf

Common Use Cases:

  • Research Documents: Upload papers that need both statistical analysis and semantic search

  • Financial Reports: Analyze numbers programmatically while searching for contextual information

  • Technical Documentation: Process code examples while maintaining searchable knowledge base

  • Mixed Data Sets: Combine structured data analysis with unstructured document search

This shared approach eliminates file duplication and ensures consistency across different analysis methods.

Code Interpreter

Code Interpreter allows the AI model to execute Python code, analyze data, create visualizations, and work with files programmatically.

How It Works

  1. Files uploaded with --file ci: are made available in the execution environment

  2. The model can write and execute Python code to analyze your data

  3. Generated files (charts, reports, processed data) are automatically downloaded

  4. Results and insights are included in the structured output

Note

πŸ’‘ Multi-Tool Tip: Use --file ci,fs:alias file.csv to make the same file available for both programmatic analysis and semantic search. This is especially useful for research data, financial reports, and technical documentation that benefit from multiple analysis approaches.

CLI Options

File Upload:

# Upload single file for code execution
ostruct run analysis.j2 schema.json --file ci:data sales_data.csv

# Upload multiple files
ostruct run analysis.j2 schema.json \
  --file ci:sales sales_data.csv \
  --file ci:customers customer_data.csv

# Upload directory of files
ostruct run analysis.j2 schema.json --dir ci:project_data ./data/

Configuration Options:

  • --ci-cleanup: Clean up uploaded files after execution (default: True)

  • --ci-download-dir DIR: Directory to save generated files (default: ./downloads/)

  • --ci-duplicate-outputs {overwrite|rename|skip}: Handle duplicate file names

File Download Behavior

Automatic Workaround (v1.6+)

ostruct automatically handles the OpenAI API file download bug by enabling the two-pass sentinel strategy when:

  • Code Interpreter is enabled

  • File downloads are explicitly enabled (--ci-download flag)

This ensures reliable file downloads when needed while optimizing performance when downloads are not required.

CLI Usage

Enable file downloads explicitly when needed:

# Enable downloads for generated files (charts, reports, data)
ostruct run analysis.j2 schema.json --file ci:data data.csv --ci-download

# Computation only - no downloads (faster execution)
ostruct run analysis.j2 schema.json --file ci:data data.csv

Configuration (Deprecated)

Deprecated since version 1.6.0: The auto_download configuration option is deprecated. Use --ci-download CLI flag instead.

# ostruct.yaml (legacy configuration)
tools:
  code_interpreter:
    download_strategy: "two_pass_sentinel"  # Force enable
    auto_download: false  # New default (changed from true)
    output_directory: "./downloads"

Performance Notes

  • Two-pass execution adds 2-3 seconds to requests

  • Single-pass mode is faster but files won’t download with structured output

  • The workaround will be removed when OpenAI fixes the underlying API bug

Best Practices

  1. Data Preparation: Upload clean, well-structured data files

  2. Clear Instructions: Specify exactly what analysis or processing you need

  3. File Organization: Use descriptive aliases for uploaded files

  4. Output Management: Organize download directories for different projects

Example Template

---
system_prompt: You are a data analyst expert in Python and visualization.
---

Analyze the uploaded sales data and create visualizations showing:
1. Monthly sales trends
2. Top-performing products
3. Regional performance comparison

Please:
- Load and clean the uploaded CSV data
- Create meaningful visualizations using matplotlib/seaborn
- Generate a summary report with key insights
- Save charts as PNG files with descriptive names
- Provide download links for your sales analysis report

MCP Servers

Model Context Protocol (MCP) servers allow you to connect to custom tools and services, extending ostruct’s capabilities with specialized functionality.

How It Works

  1. Connect to MCP servers using --mcp-server with server URLs

  2. Available tools from connected servers are made accessible to the model

  3. Tool usage can be controlled with approval settings and allowed tool lists

  4. Custom headers can be provided for authentication

CLI Options

Server Connection:

# Connect to MCP server
ostruct run analysis.j2 schema.json --mcp-server https://api.example.com/mcp

# Connect with custom label
ostruct run analysis.j2 schema.json --mcp-server mytools@https://api.example.com/mcp

# Connect to multiple servers
ostruct run analysis.j2 schema.json \
  --mcp-server tools@https://api.example.com/mcp \
  --mcp-server data@https://data.example.com/mcp

Authentication and Headers:

# Provide authentication headers
ostruct run analysis.j2 schema.json \
  --mcp-server https://api.example.com/mcp \
  --mcp-headers '{"Authorization": "Bearer token123", "X-API-Key": "key456"}'

Tool Control:

  • --mcp-require-approval [always|never]: Control tool usage approval

  • --mcp-allowed-tools TEXT: Specify allowed tools per server

Note

πŸ’‘ Multi-Tool Tip: MCP servers work excellently with shared files. Use --file ci,fs:data report.pdf to make business documents available for both programmatic analysis and semantic search, while MCP tools access your CRM or analytics systems for additional context.

Best Practices

  1. Secure Connections: Use HTTPS for MCP server connections

  2. Authentication: Properly configure headers for authenticated servers

  3. Tool Approval: Set appropriate approval levels for tool usage

  4. Error Handling: Monitor for connection and tool execution errors

Example Template

---
system_prompt: You are a business analyst with access to custom tools.
---

Using the available MCP tools, please:

1. Retrieve the latest sales data from our CRM system
2. Generate a quarterly performance report
3. Create forecasts for the next quarter

Use the appropriate tools to gather data and perform analysis, then provide a comprehensive business summary.

Multi-Tool Workflows

Combining Tools for Comprehensive Analysis

ostruct’s multi-tool capabilities shine when you need comprehensive analysis that combines different approaches. The key is using shared files that serve multiple purposes simultaneously.

Research Analysis Workflow:

# Research paper analysis with shared files
ostruct run research_analysis.j2 schema.json \
  --file ci,fs:paper research_paper.pdf \
  --file ci,fs:data experiment_data.csv \
  --enable-tool web-search \
  --ws-country "United States"

This command makes the research paper and data available to both Code Interpreter (for statistical analysis and visualization) and File Search (for semantic queries about methodology and findings), while also enabling web search for current context.

Financial Analysis Workflow:

# Comprehensive financial analysis
ostruct run financial_analysis.j2 schema.json \
  --file ci,fs:quarterly quarterly_report.pdf \
  --file ci,fs:historical historical_data.csv \
  --file fs:policies investment_policies.pdf \
  --enable-tool web-search \
  --ws-country "United States"

The quarterly report and historical data serve dual purposes: Code Interpreter performs numerical analysis and trend calculations, while File Search enables contextual queries about business strategy and policy compliance.

Technical Documentation Analysis:

# Code and documentation analysis
ostruct run tech_analysis.j2 schema.json \
  --file ci,fs:specs technical_specification.pdf \
  --file ci:codebase source_code.zip \
  --file fs:docs user_manual.pdf \
  --mcp-server dev@https://api.company.com/mcp

Technical specs benefit from both programmatic analysis (extracting requirements, parsing formats) and semantic search (finding related concepts and dependencies).

Example Multi-Tool Template

---
system_prompt: You are a senior analyst with access to multiple data sources and analysis tools.
---

# Comprehensive Analysis Report

## Quantitative Analysis
Using Code Interpreter, analyze the uploaded data files:
- Calculate key metrics and statistical summaries
- Identify trends and patterns in the numerical data
- Create visualizations showing important relationships
- Generate charts showing analysis results

## Contextual Research
Search the uploaded documents for:
- Background information and methodology
- Historical context and previous findings
- Policy implications and constraints
- Related research and citations

## Current Market Context
Research current developments related to our analysis topic:
- Recent industry trends and changes
- Competitive landscape updates
- Regulatory or policy changes
- Expert opinions and market sentiment

## Integrated Findings
Combine insights from all sources:
1. **Data-Driven Insights**: Key findings from quantitative analysis
2. **Contextual Understanding**: Background and historical perspective
3. **Current Relevance**: How findings relate to current market conditions
4. **Strategic Implications**: Actionable recommendations based on comprehensive analysis

## Supporting Evidence
Reference specific data points, document sections, and current sources that support each conclusion.

Shared File Benefits:

  1. Consistency: Same source data across all analysis methods

  2. Efficiency: Single upload serves multiple purposes

  3. Completeness: Comprehensive analysis without data gaps

  4. Cross-Validation: Findings can be verified across different approaches

Tool Combination Strategies:

Effective Tool Combinations

Use Case

Tool Combination

Shared File Strategy

Research Analysis

CI + FS + Web Search

ci,fs:paper for papers, ci,fs:data for datasets

Financial Reporting

CI + FS + MCP

ci,fs:reports for statements, fs:policies for compliance

Technical Documentation

CI + FS + MCP

ci,fs:specs for requirements, ci:code for implementation

Market Research

FS + Web Search + MCP

fs:reports for historical data, web search for current trends

Compliance Analysis

CI + FS + Web Search

ci,fs:data for metrics, fs:regulations for compliance docs

Tool Selection Guidelines

Choose tools based on your analysis needs:

Code Interpreter for: - Data processing and analysis - Statistical calculations - Visualization creation - File format conversions

File Search for: - Large document collections - Research and reference materials - Historical data analysis - Knowledge base queries

Web Search for: - Current events and trends - Market research - Competitive analysis - Real-time information

MCP Servers for: - Custom business tools - Proprietary data sources - Specialized APIs - Internal systems integration

Configuration Management

Environment Variables

Configure tool behavior using environment variables:

# MCP server configuration
export OSTRUCT_MCP_URL_crm="https://api.company.com/mcp"
export OSTRUCT_MCP_URL_analytics="https://analytics.company.com/mcp"

# Default tool settings
export OSTRUCT_DEFAULT_WS_COUNTRY="United States"
export OSTRUCT_DEFAULT_CI_DOWNLOAD_DIR="./analysis_outputs"

Configuration File

Use ostruct.yaml for persistent configuration:

template:
  max_file_size: null  # unlimited (can also use size suffixes like "128KB", "1MB")

tools:
  web_search:
    enabled: true
    country: "United States"
    context_size: "medium"

  code_interpreter:
    cleanup: true
    download_dir: "./downloads"
    duplicate_outputs: "rename"

  file_search:
    cleanup: true
    timeout: 60.0
    retries: 3

mcp:
  servers:
    - label: "crm"
      url: "https://api.company.com/mcp"
      headers:
        Authorization: "Bearer ${CRM_TOKEN}"
    - label: "analytics"
      url: "https://analytics.company.com/mcp"

  require_approval: "never"

Troubleshooting

Common Issues

Code Interpreter: - Large file upload timeouts: Reduce file sizes or use streaming - Memory errors: Process data in smaller chunks - Package availability: Check Python environment limitations

File Search: - Empty results: Known OpenAI API issue - see known issue documentation - Indexing failures: Ensure documents are text-readable - Search timeouts: Increase timeout values for large document sets - Poor search results: Use more specific queries and relevant documents

Web Search: - No results: Check internet connectivity and search terms - Geographic restrictions: Verify location settings - Rate limiting: Reduce search frequency in templates

MCP Servers: - Connection failures: Verify server URLs and network connectivity - Authentication errors: Check headers and credentials - Tool unavailability: Verify server status and tool permissions

Performance Optimization

  1. Selective Tool Usage: Only enable tools needed for your specific task

  2. File Size Management: Optimize file sizes for faster uploads and processing

  3. Concurrent Requests: Use appropriate timeout values for your network

  4. Caching: Reuse vector stores and downloaded files when appropriate

See Also