Back to catalog / Agent workflows
Extract nutrition facts from a label image
Convert a supplied nutrition-label image into typed fields through a read-only tool call, then review uncertain values.
Goal
Convert a supplied nutrition-label image into typed fields through a read-only tool call, then review uncertain values.
Steps
- Load a permitted PNG/JPEG and send it as an image block with a prompt naming calories, fat, cholesterol, carbs, and protein. [source02]
- Define
print_nutrition_infowith numeric tool properties and submit it throughclient.messages.create(..., tools=tools). [source02] - Validate units and compare the JSON fields with the image; mark unreadable or ambiguous values for human review. [source02]
How to verify
Compare all five returned nutrients and units with the supplied image; confirm any obscured value is marked uncertain instead of presented as exact.
Troubleshooting
If the tool omits a nutrient or confuses serving units, crop the label more clearly and request human review for unreadable text; do not infer a precise value.
Permissions
Prerequisites
- Python, Anthropic API key, an image accepted by the API, and permission to process its contents.
Permissions
- Use approved test or anonymized data only; scope credentials to each workflow, and require explicit approval for external writes.
Limitations
Model output may be incomplete or wrong; API and beta behavior can change, so verify against the pinned source and current service behavior.
Pinned upstream source
- Repository
- anthropics/anthropic-cookbook
- File path
- tool_use/vision_with_tools.ipynb
- Pinned commit
9c1e493ddd2b82df399d2a61b9c806c539e18be6- Source checked
- 2026-10-14 · Reviewed against the pinned upstream source
Repository metrics
These values describe the upstream repository, not popularity or likes for an individual skill or playbook.
- Stars
- Not collected
- Forks
- Not collected
- Observed at
- Not collected
No metrics snapshot is available yet.
License: MIT
Main topic
Turn ecommerce campaign materials into a reviewable weekly brief