Invoice field extraction
invoice-field-extraction · v1
Pull vendor, date, currency, and total from a short invoice into JSON. The fixture is one short invoice with an exact gold object.
Bench / Comparisons / Extraction & classification · 4 models
Navigator Bench comparison
Extraction & classification · 4 models
Invoice field extraction, Receipt total extraction, Support intent classification, Repair broken JSON · openai/gpt-4o-mini vs google/gemini-2.5-flash vs google/gemini-3.1-flash-lite vs deepseek/deepseek-chat-v3.1
As of Oct 1, 2026, 9:55 AM ET (runs stored from Oct 1, 2026, 9:55 AM ET). 16 stored runs. Methodology
A rank on this page is best_fit_for_inputs for one task, volume, and budget. It is not a universal winner.
Tasks
Receipt total extraction
receipt-total-extraction · v1
Read a short receipt and return the merchant and total as JSON. Format compliance and exact values are both checked.
Support intent classification
support-intent-classify · v1
Classify a support ticket into one closed label. The check is an exact string match, so this measures label compliance, not writing quality.
Repair broken JSON
json-repair · v1
Turn a small invalid JSON document into valid JSON with the same keys and values. The gold object is an exact match.
Models: openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Methodology: v1
Published:
Candidate runs total: 0.00088043
Judge cost: not recorded
No grand total is shown.
Judge cost not recorded for these runs
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (16 runs).
Comparison 21e2892b-5bb1-4a68-b8a2-fc6196958ed6
Funding visitor-key / video-demo
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| google/gemini-2.5-flash | Invoice field extraction | Deterministic check | 743 ms | 0.0001797 | Run 11442ae9 | |
| openai/gpt-4o-mini | Invoice field extraction | Deterministic check | 821 ms | 0.0000411 | Run 84f6325f | |
| google/gemini-3.1-flash-lite | Invoice field extraction | Deterministic check | 911 ms | 0.00010725 | Run 15dad997 | |
| deepseek/deepseek-chat-v3.1 | Invoice field extraction | Deterministic check | 1391 ms | 0.0000756 | Run 769f4d32 | |
| google/gemini-2.5-flash | Receipt total extraction | Deterministic check | 665 ms | 0.0000908 | Run 286fb9a5 | |
| google/gemini-3.1-flash-lite | Receipt total extraction | Deterministic check | 646 ms | 0.00005125 | Run 91d1826a | |
| openai/gpt-4o-mini | Receipt total extraction | Deterministic check | 1152 ms | 0.00002115 | Run 175a9a0f | |
| deepseek/deepseek-chat-v3.1 | Receipt total extraction | Deterministic check | 278 ms | 0.00003517 | Run c096a777 | |
| google/gemini-2.5-flash | Support intent classification | Deterministic check | 532 ms | 0.0000169 | Run 0c1ca431 | |
| google/gemini-3.1-flash-lite | Support intent classification | Deterministic check | 706 ms | 0.0000135 | Run b9951250 | |
| deepseek/deepseek-chat-v3.1 | Support intent classification | Deterministic check | 749 ms | 0.0000186 | Run 33f090b8 | |
| openai/gpt-4o-mini | Support intent classification | Deterministic check | 1090 ms | 0.00000855 | Run 31d79fdd | |
| google/gemini-2.5-flash | Repair broken JSON | Deterministic check | 546 ms | 0.0001009 | Run 69748770 | |
| google/gemini-3.1-flash-lite | Repair broken JSON | Deterministic check | 1008 ms | 0.00005675 | Run e75c3746 | |
| openai/gpt-4o-mini | Repair broken JSON | Deterministic check | 1553 ms | 0.00002595 | Run 8f0ba767 | |
| deepseek/deepseek-chat-v3.1 | Repair broken JSON | Deterministic check | 189 ms | 0.00003726 | Run bfc0827f |
Notable flags
No stored run in this comparison is flagged incomplete, mixed, or as an automated metadata check.
B-roll
Race, radar, run replay, fit walkthrough, PNG stills, and CSV stay marked todo. The links below are the artifacts that exist for this comparison.
JSON packet · Scatter · Checklist · Episode packet · Race (not generated)
Measured cost vs quality
Fit for your inputs
A rank is best_fit_for_inputs for one task, volume, and budget. It is computed from the stored runs on this page. It is not a universal winner, and it is not stored as its own score. Methodology
No fit rank is shown until you choose a task, volume, and budget.
Other published comparisons with this task and model set
Open in the interactive Bench · Episode page · Episode packet · Checklist