Скачать или смотреть How to Fully Test AI Agents with IBM watsonx Orchestrate ADK — Hands-On Evaluation Framework

Curious how to test AI agents end-to-end using IBM’s watsonx Orchestrate ADK?
In this 35 min hands-on walk-through, I’ll show you how to:

Define Stories (i.e. user scenarios) for your agents
Automatically generate synthetic test cases from those stories
Set up a sample backend + a local Python tool for lookups
Import & configure your Agent, connect Tools & REST APIs
Run the ADK’s built-in Evaluation Framework and inspect results

Useful Links & Resources

*Read the full blog post: Testing AI Agents with the watsonx Orchestrate ADK – Evaluation Framework A Hands-on Example*: https://suedbroecker.net/2025/09/21/t...
IBM’s ADK Overview & Evaluation Framework docs: https://developer.watson-orchestrate....
Langfuse observability integration with watsonx Orchestrate ADK : https://github.com/langfuse/langfuse
Galaxium Travels Evaluation Example https://github.com/thomassuedbroecker...

Table of Contents:

00:00 - Intro & who am I
00:25 - Date information
00:55 - What is the Evaluation Framework
02:42 - Overview of the example application
03:26 - Folder structure of the example
03:53 - Starting point of the example
04:23 - Link to setup example
04:38 - ADK version used in this video
05:01 - Overview of all steps
05:53 - Main component overview
06:40 - Creating a tool (part 1)
10:39 - Testing the tool locally
11:57 - Saving requirements.txt
12:44 - Importing the tool!
15:00 - Adding a connection
17:50 - Importing an agent
19:50 - Generating synthetic test cases!
21:01 - Building the stories file
25:47 - Running the evaluation!
28:43 - Inspecting test results
30:06 - Langfuse observability
32:18 - Summary of key findings
33:03 - Outro

Quick Checklist: Testing AI Agents with watsonx Orchestrate ADK

✅ Define clear user stories that include context, goal, and expected interaction
✅ Build or import tools & REST APIs your agent needs
✅ Generate synthetic test cases automatically from your stories
✅ Set up backend + local environment to test tools locally
✅ Use Evaluation Framework to simulate “trajectories” → compare actual behavior to ground-truth data
✅ Check for correct ordering of tool calls + correct input parameters
✅ Evaluate summary quality: final answers should be relevant, concise, accurate
developer.watson-orchestrate.ibm.com
✅ Use “ground truth datasets” with expected conversations, tool-calls, goal dependencies etc.
✅ Inspect evaluation results, logs, detect where errors occur → iterate to improve

#AIAgents #watsonxOrchestrate #IBM #ADK #AgentTesting #SyntheticTestCases #EvaluationFramework #Langfuse #ToolIntegration #MachineLearningOps #AgentDevelopment #DeveloperTutorial #PythonTools #RESTAPIs #Observability #AgentOrchestration