Navigator Bench
Invoice, receipt & intent extraction · 4 models
Rankings are for one task type, volume, and budget. They are not a universal winner.
Every cost on this page is a measured OpenRouter amount stored on a run. A blank cost was not reported. Estimates are not shown. Published comparisons · This comparison · Episode
Judge cost not recorded for these runs
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (16 runs).
Depth at a glance
Active library: 16 tasks across 6 categories (coding, extraction, reasoning, summarization, support, writing). This comparison stores 4 tasks and 4 models. Last run .
Library size is the task list. Scores and costs below come only from stored runs.
Stored runs in this comparison do not yet show a score or completeness difference. Harder library tasks stay unpublished until a BYO-key run is stored. This page does not invent a spread.
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| google/gemini-3.1-flash-lite | Invoice field extractionv1 | Deterministic check | 770 ms | 0.00010725 | Run 4845e03f | |
| google/gemini-2.5-flash | Invoice field extractionv1 | Deterministic check | 778 ms | 0.0001797 | Run ac3734fc | |
| openai/gpt-4o-mini | Invoice field extractionv1 | Deterministic check | 713 ms | 0.0000411 | Run 8b98b1b5 | |
| deepseek/deepseek-chat-v3.1 | Invoice field extractionv1 | Deterministic check | 517 ms | 0.00006811 | Run 714996a4 | |
| google/gemini-3.1-flash-lite | Receipt total extractionv1 | Deterministic check | 536 ms | 0.00005125 | Run f4186251 | |
| openai/gpt-4o-mini | Receipt total extractionv1 | Deterministic check | 494 ms | 0.00002115 | Run e398198c | |
| google/gemini-2.5-flash | Receipt total extractionv1 | Deterministic check | 728 ms | 0.0000908 | Run d5592418 | |
| deepseek/deepseek-chat-v3.1 | Receipt total extractionv1 | Deterministic check | 331 ms | 0.00003541 | Run fc44553d | |
| google/gemini-2.5-flash | Support intent classificationv1 | Deterministic check | 483 ms | 0.0000169 | Run 9c50e1d9 | |
| google/gemini-3.1-flash-lite | Support intent classificationv1 | Deterministic check | 968 ms | 0.0000135 | Run 8e51fc6a | |
| openai/gpt-4o-mini | Support intent classificationv1 | Deterministic check | 1171 ms | 0.00000855 | Run 4f68c95f | |
| deepseek/deepseek-chat-v3.1 | Support intent classificationv1 | Deterministic check | 1291 ms | 0.0000186 | Run 9f9083ae | |
| google/gemini-3.1-flash-lite | Repair broken JSONv1 | Deterministic check | 537 ms | 0.00005675 | Run 29cd06d9 | |
| google/gemini-2.5-flash | Repair broken JSONv1 | Deterministic check | 525 ms | 0.0001009 | Run 596f347b | |
| openai/gpt-4o-mini | Repair broken JSONv1 | Deterministic check | 747 ms | 0.00002595 | Run 4ea66be8 | |
| deepseek/deepseek-chat-v3.1 | Repair broken JSONv1 | Deterministic check | 226 ms | 0.00003916 | Run 7d831ad1 |
Notable flags
No stored run in this comparison is flagged incomplete, mixed, or as an automated metadata check.
Measured cost vs quality
Fit for your inputs
Label: best_fit_for_inputs. The top rank is for this task, volume, and budget. It is not a universal winner. Methodology
Choose a task, volume, and budget to rank the stored runs for that task.