Agent Wealth HubFree public resources
中文 EN العربية

Back to catalog / Agent workflows

Extract nutrition facts from a label image

Convert a supplied nutrition-label image into typed fields through a read-only tool call, then review uncertain values.

Goal

Convert a supplied nutrition-label image into typed fields through a read-only tool call, then review uncertain values.

Steps

  1. Load a permitted PNG/JPEG and send it as an image block with a prompt naming calories, fat, cholesterol, carbs, and protein. [source02]
  2. Define print_nutrition_info with numeric tool properties and submit it through client.messages.create(..., tools=tools). [source02]
  3. Validate units and compare the JSON fields with the image; mark unreadable or ambiguous values for human review. [source02]

How to verify

Compare all five returned nutrients and units with the supplied image; confirm any obscured value is marked uncertain instead of presented as exact.

Troubleshooting

If the tool omits a nutrient or confuses serving units, crop the label more clearly and request human review for unreadable text; do not infer a precise value.

Permissions

Prerequisites

Permissions

Limitations

Model output may be incomplete or wrong; API and beta behavior can change, so verify against the pinned source and current service behavior.

Pinned upstream source

https://github.com/anthropics/anthropic-cookbook/blob/9c1e493ddd2b82df399d2a61b9c806c539e18be6/tool_use/vision_with_tools.ipynb

Repository
anthropics/anthropic-cookbook
File path
tool_use/vision_with_tools.ipynb
Pinned commit
9c1e493ddd2b82df399d2a61b9c806c539e18be6
Source checked
2026-10-14 · Reviewed against the pinned upstream source

Repository metrics

These values describe the upstream repository, not popularity or likes for an individual skill or playbook.

Stars
Not collected
Forks
Not collected
Observed at
Not collected

No metrics snapshot is available yet.

License: MIT

Main topic

Turn ecommerce campaign materials into a reviewable weekly brief

Related resources