Back to catalog / Agent evaluation
Test agent orchestration with a deterministic mock model
Verify application-owned tool dispatch, handoffs, and output handling without calling a live model service.
Goal
Check workflow control logic with repeatable inputs and scripted model outputs.
Steps
- Separate behavior owned by the application from behavior requiring a live model or remote service. [openai-testing]
- Script model output with
agents.testing.ScriptedModel([assistant_message(...)]); includefunction_call(...)for tool trajectories. [openai-testing] - Run workflow tests with the SDK’s test runner and mock model; disable trace export in test configuration. [openai-testing]
- Assert call order, final output, and rejection/failure branches without relying on random model wording. [openai-testing]
- Reserve provider- or network-specific behavior for isolated integration tests. [openai-testing]
How to verify
Repeated runs of the same script should produce the same tool sequence and result, with no live model request.
Troubleshooting
If a test unexpectedly calls a provider, verify the mock model is installed and tracing export is disabled.
Permissions
Prerequisites
- A Python runtime and the corresponding SDK version/configuration
Permissions
- Use a local or isolated test environment; use least privilege and approval for external data or side effects
Limitations
This tests application-owned orchestration, not live model quality, provider behavior, or network failures. No API key is needed; disable trace export if credentials exist.
Pinned upstream source
- Repository
- openai/openai-agents-python
- File path
- docs/testing.md
- Pinned commit
f53c62451c7f34c15f0c87fa745e05404025420f- Source checked
- 2026-10-14 · Reviewed against the pinned upstream source
Repository metrics
These values describe the upstream repository, not popularity or likes for an individual skill or playbook.
- Stars
- Not collected
- Forks
- Not collected
- Observed at
- Not collected
No metrics snapshot is available yet.
License: MIT
Main topic
Take a coding Agent change from context to verification