Invoice field extraction
invoice-field-extraction · v1
Pull vendor, date, currency, and total from a short invoice into JSON. The fixture is one short invoice with an exact gold object.
Bench / Episodes / Invoice, receipt & intent extraction · 4 models
Navigator Bench episode
Invoice, receipt & intent extraction · 4 models
Invoice field extraction, Receipt total extraction, Support intent classification, Repair broken JSON · openai/gpt-4o-mini vs google/gemini-2.5-flash vs google/gemini-3.1-flash-lite vs deepseek/deepseek-chat-v3.1
As of Oct 1, 2026, 8:23 PM ET (runs stored from Oct 1, 2026, 8:23 PM ET). 16 stored runs. Methodology
A rank on this page is best_fit_for_inputs for one task, volume, and budget. It is not a universal winner.
Tasks
Receipt total extraction
receipt-total-extraction · v1
Read a short receipt and return the merchant and total as JSON. Format compliance and exact values are both checked.
Support intent classification
support-intent-classify · v1
Classify a support ticket into one closed label. The check is an exact string match, so this measures label compliance, not writing quality.
Repair broken JSON
json-repair · v1
Turn a small invalid JSON document into valid JSON with the same keys and values. The gold object is an exact match.
Models: openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Methodology: v1
Published:
Candidate runs total: 0.00087508
Judge cost: not recorded
No grand total is shown.
Judge cost not recorded for these runs
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (16 runs).
Comparison 65c072a8-816f-416c-9be6-7ef7256b9e70
Funding visitor-key / video-demo
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| google/gemini-3.1-flash-lite | Invoice field extraction | Deterministic check | 770 ms | 0.00010725 | Run 4845e03f | |
| google/gemini-2.5-flash | Invoice field extraction | Deterministic check | 778 ms | 0.0001797 | Run ac3734fc | |
| openai/gpt-4o-mini | Invoice field extraction | Deterministic check | 713 ms | 0.0000411 | Run 8b98b1b5 | |
| deepseek/deepseek-chat-v3.1 | Invoice field extraction | Deterministic check | 517 ms | 0.00006811 | Run 714996a4 | |
| google/gemini-3.1-flash-lite | Receipt total extraction | Deterministic check | 536 ms | 0.00005125 | Run f4186251 | |
| openai/gpt-4o-mini | Receipt total extraction | Deterministic check | 494 ms | 0.00002115 | Run e398198c | |
| google/gemini-2.5-flash | Receipt total extraction | Deterministic check | 728 ms | 0.0000908 | Run d5592418 | |
| deepseek/deepseek-chat-v3.1 | Receipt total extraction | Deterministic check | 331 ms | 0.00003541 | Run fc44553d | |
| google/gemini-2.5-flash | Support intent classification | Deterministic check | 483 ms | 0.0000169 | Run 9c50e1d9 | |
| google/gemini-3.1-flash-lite | Support intent classification | Deterministic check | 968 ms | 0.0000135 | Run 8e51fc6a | |
| openai/gpt-4o-mini | Support intent classification | Deterministic check | 1171 ms | 0.00000855 | Run 4f68c95f | |
| deepseek/deepseek-chat-v3.1 | Support intent classification | Deterministic check | 1291 ms | 0.0000186 | Run 9f9083ae | |
| google/gemini-3.1-flash-lite | Repair broken JSON | Deterministic check | 537 ms | 0.00005675 | Run 29cd06d9 | |
| google/gemini-2.5-flash | Repair broken JSON | Deterministic check | 525 ms | 0.0001009 | Run 596f347b | |
| openai/gpt-4o-mini | Repair broken JSON | Deterministic check | 747 ms | 0.00002595 | Run 4ea66be8 | |
| deepseek/deepseek-chat-v3.1 | Repair broken JSON | Deterministic check | 226 ms | 0.00003916 | Run 7d831ad1 |
Notable flags
No stored run in this comparison is flagged incomplete, mixed, or as an automated metadata check.
B-roll
Race, radar, run replay, fit walkthrough, PNG stills, and CSV stay marked todo. The links below are the artifacts that exist for this comparison.
JSON packet · Scatter · Checklist · Episode packet · Race (not generated)
Measured cost vs quality
Open in the interactive Bench · Comparison page · Episode packet · Checklist