Navigator Bench
Meeting notes & customer summaries · 4 models
Rankings are for one task type, volume, and budget. They are not a universal winner.
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (16 runs).
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| google/gemini-2.5-flash | Meeting action itemsv1 | Deterministic check | 474 ms | 0.0002295 | Run 09df9762 | |
| google/gemini-3.1-flash-lite | Meeting action itemsv1 | Deterministic check | 692 ms | 0.00013475 | Run efbf5b21 | |
| openai/gpt-4o-mini | Meeting action itemsv1 | Deterministic check | 568 ms | 0.0000519 | Run 602d9a84 | |
| deepseek/deepseek-chat-v3.1 | Meeting action itemsv1 | Deterministic check | 178 ms | 0.00008441 | Run 49ea913a | |
| google/gemini-3.1-flash-lite | Five-bullet briefv1 | Deterministic check | 532 ms | 0.0000855 | Run f078e9be | |
| google/gemini-2.5-flash | Five-bullet briefv1 | Deterministic check | 918 ms | 0.0001148 | Run 2481d57d | |
| openai/gpt-4o-mini | Five-bullet briefv1 | Deterministic check | 505 ms | 0.00004095 | Run a456fef7 | |
| deepseek/deepseek-chat-v3.1 | Five-bullet briefv1 | Deterministic check | 1976 ms | 0.0000601 | Run e4c47496 | |
| google/gemini-2.5-flash | Earnings call summaryv1 | LLM judge | 468 ms | 0.0002474 | Run 961a8dc7 | |
| openai/gpt-4o-mini | Earnings call summaryv1 | LLM judge | 427 ms | 0.00006765 | Run adebaa89 | |
| google/gemini-3.1-flash-lite | Earnings call summaryv1 | LLM judge | 514 ms | 0.0001665 | Run af33314b | |
| deepseek/deepseek-chat-v3.1 | Earnings call summaryv1 | LLM judge | 1069 ms | 0.00011562 | Run 1ec54c69 | |
| google/gemini-2.5-flash | Customer changelog draftv1 | LLM judge | 656 ms | 0.0001547 | Run 9f12b9a0 | |
| deepseek/deepseek-chat-v3.1 | Customer changelog draftv1 | LLM judge | 186 ms | 0.00009955 | Run 2a09ac45 | |
| google/gemini-3.1-flash-lite | Customer changelog draftv1 | LLM judge | 749 ms | 0.000104 | Run d26f4e7a | |
| openai/gpt-4o-mini | Customer changelog draftv1 | LLM judge | 1267 ms | 0.00002685 | Run c05c57fc |
Notable flags
No stored run in this comparison is flagged incomplete, mixed, or as an automated metadata check.
Measured cost vs quality
Fit for your inputs
Label: best_fit_for_inputs. The top rank is for this task, volume, and budget. It is not a universal winner. Methodology
Choose a task, volume, and budget to rank the stored runs for that task.