> ## Documentation Index > Fetch the complete documentation index at: https://connect.watson-orchestrate.ibm.com/llms.txt > Use this file to discover all available pages before exploring further. # Validating your native agent The evaluation framework is a tool that composes the [watsonx Orchestrate Agent Development Kit (ADK)](https://developer.watson-orchestrate.ibm.com), which allows you to test, evaluate and analyze native agents that you have created. To test your agents, you have to set up a file with the expected interaction responses, and run the agent evaluation to check if your agent matches the expectations. **External agents**: External agents require TSV validation files but do not use the ADK validation framework. See the [TSV File Requirements for External Agents](#tsv-file-requirements-for-external-agents) section below and [External Agent Onboarding](../agent/onboard-external) for details. ## Before you begin * You must have a **native agent** built with the [Agent Development Kit ](https://developer.watson-orchestrate.ibm.com/). * If you need to run the framework with model-proxy and a `WO_INSTANCE` that points to a non-Dallas region, you can supply the model override flag: ```bash theme={null} export MODEL_OVERRIDE="meta-llama/llama-3-2-90b-vision-instruct" ``` ```bash theme={null} export MODEL_OVERRIDE="meta-llama/llama-3-3-70b-instruct" ``` * Install the watsonx Orchestrate Agent Development Kit. For more information, see [Installing the ADK ](https://developer.watson-orchestrate.ibm.com/getting_started/installing). * Install the watsonx Orchestrate Developer Edition. For more information, see [Installing the Developer Edition ](https://developer.watson-orchestrate.ibm.com/getting_started/wxOde_setup). * For more information about the evaluation framework, see the [Evaluation framework overview ](https://developer.watson-orchestrate.ibm.com/evaluate/overview). ## Validating your native agent **Native agents** refer to agents that were created with the **watsonx Orchestrate Agent Development Kit** or inside the watsonx Orchestrate platform. The `validate-native` command validates the native agent and registered tools, collaborator agents, and knowledge bases against a set of inputs. ### Running the Validation Prepare a TSV file with three columns: * The first column contains user stories. * The second column is the expected summary or output. * The third column is the name of the native agent that you want to validate. For example: ```tsv example.tsv theme={null} My username is nwaters. I want to find out my timeoff schedule from: 2025-01-01 to: 2025-03-03. Your timeoff schedule for 20250101 to 20250303 is: 20250105 hr_agent ``` The provided user stories and expected output are used to generate the json formatted test case used to evaluate the agent. The generated test cases are saved at the path: `/native_agent_evaluations/generated_test_data` **Running the command** ```bash theme={null} orchestrate evaluations validate-native -t -o ``` Path to .tsv file of inputs. The first column of the TSV is the user story, the second column is the expected final output from the agent, and the third column is the name of the agent. Path to save the validation results. Path to the `.env` file that overrides the default environment. ## Preparing for submission To prepare your native agent for submission, you must include all the result files from the validation and evaluation stages. Compress the results into a zip file: ```none theme={null} company-name-onboarding-validation.zip/ └── native_agent_evaluations/ ├── generated_test_data/ ├── native_agent_evaluation_test_0.json ├── ... ├── knowledge_base_summary_metrics/ ├── messages/ └── summary_metrics.csv └── ... # other relevant files for the evaluation └── evaluations/ ├── ... # other relevant files for the evaluation ├── sample_agent.yaml ├── knowledge_base_summary_metrics.json └── summary_metrics.csv ``` ## Next steps After you prepare the evaluation files, you must **package your agent**. The validation results should be placed under the `evaluations/` folder in your package structure: ``` . ├── agents/ │ └── my_agent.yaml ├── connections/ │ └── my_connections.yaml ├── offerings/ │ └── my_offering.yaml ├── tools/ │ └── sample_tool/ │ ├── tool.py │ └── requirements.txt └── evaluations/ └── company-name-onboarding-validation.zip ``` For complete packaging and submission instructions, see [Native Agent Onboarding](../agent/onboard-native). ## TSV File Requirements for External Agents External agents also require TSV validation files for testing, though they don't use the ADK validation framework. ### TSV File Format Create a TSV file with three columns: * **Column 1**: User prompt or query * **Column 2**: Expected response or outcome * **Column 3**: Agent identifier (your external agent name) ### Example ```tsv test.tsv theme={null} What is the weather in Paris? The current weather in Paris is 18°C and sunny. weather_agent Find me a hotel in Tokyo Here are available hotels in Tokyo: [list] travel_agent Book a flight to New York I found 5 flights to New York. Which date would you prefer? travel_agent ``` ### Submission Process 1. **Create your TSV file** with comprehensive test cases covering your agent's capabilities 2. **Test thoroughly** with your external agent endpoint to ensure accuracy 3. **Submit to IBM** by emailing your TSV file to: **[IBMAgentConnect@ibm.com](mailto:IBMAgentConnect@ibm.com)** * Subject line: "TSV Validation - \[Your Agent Name]" * Include the TSV file as an attachment 4. **Include in submission**: Upload the TSV file when submitting through the IBM Concierge app ### Best Practices * Include diverse test cases covering different scenarios * Ensure expected responses match your agent's actual behavior * Test edge cases and error handling * Keep responses concise but representative of actual output * Use consistent agent identifiers across all test cases For complete external agent onboarding instructions, see [External Agent Onboarding](../agent/onboard-external).