Back to catalog / Agent workflows

Fix a failing test fixture and verify independently

Iterate on failing tests in a locked-down fixture, then rerun assertions outside the agent session.

Goal

Have an agent fix known defects and confirm the final files pass the original tests without over-editing.

Upload source and test files separately and create an environment with networking: {"type":"limited"}; mounted uploads are read-only. [iterate]
Use agent_toolset_20260401 for the test/read-traceback/edit/rerun loop; fix the failing assertion instead of weakening tests. [iterate]
After the run, execute the full suite outside the session and inspect the final diff; the example independently reruns pytest. [iterate]

External tests pass, the diff is limited to the defect, and the read-only mounted originals remain unchanged.

If the same failure repeats, confirm the agent copied read-only mounts to a writable directory; `session idle` is not a passing test.

Python 3.11+, an Anthropic API key, the SDK, and a disposable source/pytest fixture.

Use a secret-free test repository; disable arbitrary outbound networking and confine tools to the sandbox.

Agent-run tests are not independent verification; trusted CI must rerun them before merge or release.

These values describe the upstream repository, not popularity or likes for an individual skill or playbook.

No metrics snapshot is available yet.

License: MIT