Agent Wealth HubFree public resources
中文 EN العربية

Back to catalog / Agent evaluation

Test agent orchestration with a deterministic mock model

Verify application-owned tool dispatch, handoffs, and output handling without calling a live model service.

Goal

Check workflow control logic with repeatable inputs and scripted model outputs.

Steps

  1. Separate behavior owned by the application from behavior requiring a live model or remote service. [openai-testing]
  2. Script model output with agents.testing.ScriptedModel([assistant_message(...)]); include function_call(...) for tool trajectories. [openai-testing]
  3. Run workflow tests with the SDK’s test runner and mock model; disable trace export in test configuration. [openai-testing]
  4. Assert call order, final output, and rejection/failure branches without relying on random model wording. [openai-testing]
  5. Reserve provider- or network-specific behavior for isolated integration tests. [openai-testing]

How to verify

Repeated runs of the same script should produce the same tool sequence and result, with no live model request.

Troubleshooting

If a test unexpectedly calls a provider, verify the mock model is installed and tracing export is disabled.

Permissions

Prerequisites

Permissions

Limitations

This tests application-owned orchestration, not live model quality, provider behavior, or network failures. No API key is needed; disable trace export if credentials exist.

Pinned upstream source

https://github.com/openai/openai-agents-python/blob/f53c62451c7f34c15f0c87fa745e05404025420f/docs/testing.md

Repository
openai/openai-agents-python
File path
docs/testing.md
Pinned commit
f53c62451c7f34c15f0c87fa745e05404025420f
Source checked
2026-10-14 · Reviewed against the pinned upstream source

Repository metrics

These values describe the upstream repository, not popularity or likes for an individual skill or playbook.

Stars
Not collected
Forks
Not collected
Observed at
Not collected

No metrics snapshot is available yet.

License: MIT

Main topic

Take a coding Agent change from context to verification

Related resources