Navigator Bench
Bug explanation & support reply · 4 models
Rankings are for one task type, volume, and budget. They are not a universal winner.
Comparison c84cb8c1-af67-4ac2-8460-3723111f6859
Published
Methodology v1 · How scoring works
Funding visitor-key / video-demo
Models openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Candidate runs total: 0.00149216
Judge cost: 0.0004952
Total (candidate + judge): 0.00198736
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (8 runs).
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| openai/gpt-4o-mini | Explain a bug in plain languagev1 | LLM judge | 687 ms | 0.0000963 | Run fe33c3ea | |
| google/gemini-3.1-flash-lite | Explain a bug in plain languagev1 | LLM judge | 802 ms | 0.00025025 | Run c994454e | |
| google/gemini-2.5-flash | Explain a bug in plain languagev1 | LLM judge | 4/5 · incompleteStored overall 5/5jargon 5 · causality 5 · correctness 5 · actionability 3Needs review · incomplete outputHuman spot checkMethodology | 427 ms | 0.0005177 | Run 94e844f4 |
| deepseek/deepseek-chat-v3.1 | Explain a bug in plain languagev1 | LLM judge | 304 ms | 0.00012611 | Run f468cb57 | |
| google/gemini-2.5-flash | Tone-matched support replyv1 | LLM judge | 4/5 · mixedStored overall 5/5accuracy 5 · tone_match 5 · completeness 3 · policy_safety 5Mixed resultHuman spot checkMethodology | 455 ms | 0.0001416 | Run 95392567 |
| deepseek/deepseek-chat-v3.1 | Tone-matched support replyv1 | LLM judge | 243 ms | 0.000165 | Run f6d12eae | |
| google/gemini-3.1-flash-lite | Tone-matched support replyv1 | LLM judge | 701 ms | 0.00014075 | Run 49938747 | |
| openai/gpt-4o-mini | Tone-matched support replyv1 | LLM judge | 1147 ms | 0.00005445 | Run 740c93db |
Notable flags
- google/gemini-2.5-flash on Explain a bug in plain language. 4/5 · incomplete.Needs review · The output ends mid-token inside an unclosed inline code span.
- google/gemini-2.5-flash on Tone-matched support reply. 4/5 · mixed.Displayed overall capped at 4 because completeness scored 3 or lower. The stored judge overall remains 5.
Measured cost vs quality
Fit for your inputs
Label: best_fit_for_inputs. The top rank is for this task, volume, and budget. It is not a universal winner. Methodology
Choose a task, volume, and budget to rank the stored runs for that task.