Navigator Bench
Extraction & classification · 4 models
Rankings are for one task type, volume, and budget. They are not a universal winner.
Every cost on this page is a measured OpenRouter amount stored on a run. A blank cost was not reported. Estimates are not shown. Published comparisons · This comparison · Episode
Judge cost not recorded for these runs
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (16 runs).
Depth at a glance
Active library: 16 tasks across 6 categories (coding, extraction, reasoning, summarization, support, writing). This comparison stores 4 tasks and 4 models. Last run .
Library size is the task list. Scores and costs below come only from stored runs.
Stored runs in this comparison do not yet show a score or completeness difference. Harder library tasks stay unpublished until a BYO-key run is stored. This page does not invent a spread.
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| google/gemini-2.5-flash | Invoice field extractionv1 | Deterministic check | 743 ms | 0.0001797 | Run 11442ae9 | |
| openai/gpt-4o-mini | Invoice field extractionv1 | Deterministic check | 821 ms | 0.0000411 | Run 84f6325f | |
| google/gemini-3.1-flash-lite | Invoice field extractionv1 | Deterministic check | 911 ms | 0.00010725 | Run 15dad997 | |
| deepseek/deepseek-chat-v3.1 | Invoice field extractionv1 | Deterministic check | 1391 ms | 0.0000756 | Run 769f4d32 | |
| google/gemini-2.5-flash | Receipt total extractionv1 | Deterministic check | 665 ms | 0.0000908 | Run 286fb9a5 | |
| google/gemini-3.1-flash-lite | Receipt total extractionv1 | Deterministic check | 646 ms | 0.00005125 | Run 91d1826a | |
| openai/gpt-4o-mini | Receipt total extractionv1 | Deterministic check | 1152 ms | 0.00002115 | Run 175a9a0f | |
| deepseek/deepseek-chat-v3.1 | Receipt total extractionv1 | Deterministic check | 278 ms | 0.00003517 | Run c096a777 | |
| google/gemini-2.5-flash | Support intent classificationv1 | Deterministic check | 532 ms | 0.0000169 | Run 0c1ca431 | |
| google/gemini-3.1-flash-lite | Support intent classificationv1 | Deterministic check | 706 ms | 0.0000135 | Run b9951250 | |
| deepseek/deepseek-chat-v3.1 | Support intent classificationv1 | Deterministic check | 749 ms | 0.0000186 | Run 33f090b8 | |
| openai/gpt-4o-mini | Support intent classificationv1 | Deterministic check | 1090 ms | 0.00000855 | Run 31d79fdd | |
| google/gemini-2.5-flash | Repair broken JSONv1 | Deterministic check | 546 ms | 0.0001009 | Run 69748770 | |
| google/gemini-3.1-flash-lite | Repair broken JSONv1 | Deterministic check | 1008 ms | 0.00005675 | Run e75c3746 | |
| openai/gpt-4o-mini | Repair broken JSONv1 | Deterministic check | 1553 ms | 0.00002595 | Run 8f0ba767 | |
| deepseek/deepseek-chat-v3.1 | Repair broken JSONv1 | Deterministic check | 189 ms | 0.00003726 | Run bfc0827f |
Notable flags
No stored run in this comparison is flagged incomplete, mixed, or as an automated metadata check.
Measured cost vs quality
Fit for your inputs
Label: best_fit_for_inputs. The top rank is for this task, volume, and budget. It is not a universal winner. Methodology
Choose a task, volume, and budget to rank the stored runs for that task.