Back to catalog / Agent evaluation

Improve a code draft with explicit pass criteria

Have a generator implement a task, an evaluator score only against criteria, and iterate within a bounded loop.

Goal

Iterate a code draft until independent evaluation returns PASS or the process stops explicitly.

Define correctness, complexity, and style criteria; require PASS/NEEDS_IMPROVEMENT/FAIL and feedback from the evaluator only. [optimizer]
Give the generator the task and prior feedback; let the evaluator inspect the candidate without rewriting it. [optimizer]
Set a maximum number of rounds, record each candidate/evaluation, then run local tests and review the final diff after PASS. [optimizer]

Seed a faulty implementation; the evaluator gives actionable feedback and the loop converges or reports its limit.

If it always passes, calibrate criteria with counterexamples; if it repeats the same change, stop and escalate.

Write only to a temporary branch/patch; do not let the loop commit, merge, or deploy.

An LLM evaluator is not formal verification; the caller must bound rounds, and PASS does not replace tests or review.

These values describe the upstream repository, not popularity or likes for an individual skill or playbook.

No metrics snapshot is available yet.

License: MIT