Back to catalog / Agent workflows

Compare context and tool-cost optimization on benchmark cases

Compare lean context, selective retrieval, subagents, and batching under the same quality threshold.

Goal

Goal: Compare lean context, selective retrieval, subagents, and batching under the same quality threshold.

Use the cookbook’s synthetic claims and labeled expected outcomes as a fixed set; run a full-context baseline first. [cost-opt]
Measure deferred tool_search, on-demand manual reads, isolated subagents, and Message Batches; change one cost lever at a time. [cost-opt]
Record accuracy, calls, input tokens, and API cost; adopt a cheaper design only if accuracy meets baseline. [cost-opt]

Each design is compared on the same cases; cost falls while accuracy stays above the chosen threshold.

Review disagreements as possible regressions; do not claim savings or unchanged quality from a single sample.

Python 3.11+, Anthropic API key, cookbook synthetic fixtures, billing visibility, and fixed tasks.

Use synthetic data only; do not send real claims/customer data; run within an approved API budget.

The optimizations are specific to synthetic insurance tasks; prices, caching, models, and tools change, so rerun on your own workload.

These values describe the upstream repository, not popularity or likes for an individual skill or playbook.

No metrics snapshot is available yet.

License: MIT