Navigator Bench · Index v2.0 · August 2026 (demo)
Demo data · not real resultsTop-rank quality at 1/4 the cost per successful task
5 models share rank 1 (none is significantly better than another on the same tasks); Model F · low effort (Index 93.1) costs $0.031 per successful task, Model D · high effort (Index 93.6) $0.14.
How to read this chart
The two axes
Up is better quality (the Navigator Index, 0–100). Left is cheaper per successful task. Cost is on a log scale: each bold gridline is ten times the one to its left.
The green line
The green line joins the models nobody beats on both quality and cost.
Rank ranges
A rank like “1–5” means we can’t statistically separate these models, so they share the range.
The four areas
The dashed lines mark the quality bar (Index 70) and the middle cost ($0.020). They make four areas:
- Value sweet spot (top left): good enough, for less.
- Frontier (top right): good enough, but costs more.
- Outclassed (bottom left): cheaper, but under the bar.
- Paying for the badge (bottom right): costs more, still under the bar.
Colours
Dot colour shows the provider:
- Provider 1
- Provider 2
- Provider 3
- Provider 4
- Provider 5
Data through 28 Aug 20261,440 runsIndex v2.0ChangelogMethodChart card 16:91:1
02Leaderboard
Pick a category. Watch the order change.
Top-5 cardOrdered by rank. Ranks are ranges: 2–4 means we can't statistically separate it from the models in that range (paired test on the same tasks; the chance of any false win on the board is at most 5%). The bar under each rank shows where its range sits.
Navigator Bench · Index v2.0 · data through 28 Aug 2026Whiskers: 95% task-clustered intervals · How ties work · Cost per success
03Kill zone
Better and cheaper, or it doesn't count
Model K · low effort dominates 3 of 11 models on Navigator Bench — higher Index, lower cost per successful task.
Showing the largest kill zone this cycle. Pick any model here, from a point's card, or with the ⊘ button in the leaderboard; the scatter and the table dim with it.
A kill needs both: significantly better on the Index (a paired test on the same tasks, with the chance of any false win on the board held to 5%) and cheaper per successful task. A gap inside the noise never counts. How dominance is decided
- 1–5Model A · low93.8 · $0.060
- 1–5Model D · high93.6 · $0.14
- 1–5Model D · low93.2 · $0.053
- 1–5Model F · low93.1 · $0.031
- 1–5Model I · low93.0 · $0.034
- 6–8Model K · low78.2 · $0.0057
- 6–8Model G · low78.1 · $0.0085
- 6–8Model B · low72.9 · $0.020
- 9–11Model L · low51.3 · $0.0053
- 9–12Model C · low45.7 · $0.021dominated
- 9–12Model E · low44.3 · $0.011dominated
- 10–12Model H41.5 · $0.010dominated
04Commoditization curve
What Index 70 costs, cycle by cycle
Curve cardCheapest at Index ≥ 70 · August 2026 (demo)
$0.0057per successful task · Model K · low effort
Tracking since 28 Aug 2026. The curve grows one point per published cycle; earlier history is never back-filled.
August 2026 (demo): Model K · low effort was the cheapest model at Index 70 or better, at $0.0057 per successful task. Data through 28 Aug 2026.
05Compare
Two or three models, every category
- Model A · low effort
- Model K · low effort
- Shaded rings: 95% intervals
- Cost / success
- $0.060
- Median latency
- 9.0 s
- Pass rate
- 100% (91%–100%)
- Passes every time
- 100%
- Cost / success
- $0.0057
- Median latency
- 7.7 s
- Pass rate
- 78% (67%–88%)
- Passes every time
- 63%
Per-task receipts (prompt, output, checks and cost for every run) open from each model's receipts link.
06Fit-finder
The cheapest model that clears your bar
Cheapest at Navigator Index ≥ 70
$57.05a month · $0.0057 per successful task
Cheapest model at Navigator Index 70.0 or better: Model K · low effort, $0.0057 per successful task (about $57.05 for 10,000 successful tasks a month).
- Model K · lowNavigator Index score 78.2monthly cost $57.05/mo, clears the bar
- Model G · lowNavigator Index score 78.1monthly cost $84.79/mo, clears the bar
- Model B · lowNavigator Index score 72.9~monthly cost $196/mo, clears the bar
- Model F · lowNavigator Index score 93.1monthly cost $309/mo, clears the bar
- Model I · lowNavigator Index score 93.0monthly cost $338/mo, clears the bar
- Model D · lowNavigator Index score 93.2monthly cost $534/mo, clears the bar
- Model A · lowNavigator Index score 93.8monthly cost $603/mo, clears the bar
- Model D · highNavigator Index score 93.6monthly cost $1,433/mo, clears the bar
- Model L · lowNavigator Index score 51.3monthly cost $53.05/mo, below the bar
- Model HNavigator Index score 41.5monthly cost $101/mo, below the bar
- Model E · lowNavigator Index score 44.3monthly cost $107/mo, below the bar
- Model C · lowNavigator Index score 45.7monthly cost $214/mo, below the bar
Bars are the Navigator Index score on a plain 0–100 scale; the tick is your bar, and the number beside each bar is the score itself. Monthly cost = cost per successful task × successful tasks, printed as a number. Failed runs are already priced in: cost per success divides everything spent by the tasks that passed. Definition
07Methodology & freshness
Every number, defined
Demo data · not real results This board is built from development fixtures, not from a published cycle. Model names are placeholders and no number on it is a measurement.
- Navigator Index
- 0–100. The weighted mean of 8 category scores; a category score is the mean task score (partial credit, 0–1) × 100 over that category’s active tasks and every repeat. Weights: Coding 15% · Data analysis 15% · Extraction 10% · Reasoning 15% · Research & agentic 15% · Summarization 10% · Support 10% · Writing 10%. Research & agentic: For now these are single-call multi-step tasks: one model call returns plan → steps → deliverable, checked step by step. Tool-using, browsing and long agentic tasks are deferred.
- Intervals
- 95%, task-clustered: a bootstrap over tasks (stratified by category for the Index), never naive per-run intervals. Every score on this page shows its interval as a whisker or a range.
- Ranks are ranges
- Every pair of models is compared on the same tasks (a paired test of the Index difference, with family-wise error control, so the chance of any false “win” on the board is at most 5%). A rank is the range the data allows: best = 1 + models significantly better, worst = N − models significantly worse. “2–4” means we can't statistically separate it from the models in that range; a single number means it is pinned down.
- Dominance (kill zone)
- A model dominates another only when it is significantly better on the Index (the same paired test) and has a lower cost per successful task. A point-estimate gap inside the noise is never a kill. Reference rows are never counted.
- Cost per successful task
- Everything measured on candidate calls (OpenRouter's reported amounts) divided by the tasks passed, where a pass is a score at or above the task's threshold. Judge cost is excluded. Blank when nothing passed.
- Latency
- Median wall-clock latency per run, as measured on the pinned provider.
- Movement
- Δ compares with the previous published cycle at the same Index version only. Without one, the entry shows NEW or “first cycle”; movement is never estimated.
- Commoditization curve
- One point per published cycle: the cheapest cost per successful task among models at Index 70 or better. With fewer than three cycles it reads “tracking since”.
- Frontier and quadrants
- The frontier joins models that no other model beats on both Index and cost per success. Quadrants split at the quality bar (Index 70) and at the median plotted cost per success; they are reading aids, not verdicts.