Navigator Bench
Summaries & writing · 4 models
Rankings are for one task type, volume, and budget. They are not a universal winner.
Every cost on this page is a measured OpenRouter amount stored on a run. A blank cost was not reported. Estimates are not shown. Published comparisons · This comparison · Episode
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (16 runs).
Depth at a glance
Active library: 16 tasks across 6 categories (coding, extraction, reasoning, summarization, support, writing). This comparison stores 4 tasks and 4 models. Last run .
Library size is the task list. Scores and costs below come only from stored runs.
Stored runs in this comparison do not yet show a score or completeness difference. Harder library tasks stay unpublished until a BYO-key run is stored. This page does not invent a spread.
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| google/gemini-2.5-flash | Meeting action itemsv1 | Deterministic check | 580 ms | 0.0002295 | Run 3a2851ec | |
| openai/gpt-4o-mini | Meeting action itemsv1 | Deterministic check | 482 ms | 0.0000519 | Run 198dc3a1 | |
| google/gemini-3.1-flash-lite | Meeting action itemsv1 | Deterministic check | 1019 ms | 0.00013475 | Run 1a65178d | |
| deepseek/deepseek-chat-v3.1 | Meeting action itemsv1 | Deterministic check | 531 ms | 0.00008417 | Run 58ade0b2 | |
| google/gemini-2.5-flash | Five-bullet briefv1 | Deterministic check | 721 ms | 0.0001148 | Run bd6ed3cc | |
| google/gemini-3.1-flash-lite | Five-bullet briefv1 | Deterministic check | 988 ms | 0.0000855 | Run 01fbc12d | |
| deepseek/deepseek-chat-v3.1 | Five-bullet briefv1 | Deterministic check | 1239 ms | 0.00005863 | Run 52f01466 | |
| openai/gpt-4o-mini | Five-bullet briefv1 | Deterministic check | 1060 ms | 0.00004095 | Run fa9c0387 | |
| google/gemini-2.5-flash | Earnings call summaryv1 | LLM judge | 574 ms | 0.0002474 | Run 74a20a81 | |
| google/gemini-3.1-flash-lite | Earnings call summaryv1 | LLM judge | 742 ms | 0.0001665 | Run e6c7f5ff | |
| deepseek/deepseek-chat-v3.1 | Earnings call summaryv1 | LLM judge | 730 ms | 0.00020045 | Run 63f45176 | |
| openai/gpt-4o-mini | Earnings call summaryv1 | LLM judge | 1193 ms | 0.00006525 | Run 51dff993 | |
| google/gemini-2.5-flash | Customer changelog draftv1 | LLM judge | 472 ms | 0.0001547 | Run 7656aabb | |
| google/gemini-3.1-flash-lite | Customer changelog draftv1 | LLM judge | 648 ms | 0.000104 | Run aea93a18 | |
| openai/gpt-4o-mini | Customer changelog draftv1 | LLM judge | 674 ms | 0.00002685 | Run 6a7c2320 | |
| deepseek/deepseek-chat-v3.1 | Customer changelog draftv1 | LLM judge | 1718 ms | 0.00005325 | Run 0e3ff8e9 |
Notable flags
No stored run in this comparison is flagged incomplete, mixed, or as an automated metadata check.
Measured cost vs quality
Fit for your inputs
Label: best_fit_for_inputs. The top rank is for this task, volume, and budget. It is not a universal winner. Methodology
Choose a task, volume, and budget to rank the stored runs for that task.