Meeting action items
meeting-action-items · v1
Extract action items as a JSON array of owner and task. The deterministic check compares the set of owners only. Wording of the task strings is not scored here; that would be a rubric.
Bench / Episodes / Summaries & writing · 4 models
Navigator Bench episode
Summaries & writing · 4 models
Meeting action items, Five-bullet brief, Earnings call summary, Customer changelog draft · openai/gpt-4o-mini vs google/gemini-2.5-flash vs google/gemini-3.1-flash-lite vs deepseek/deepseek-chat-v3.1
As of Oct 1, 2026, 9:56 AM ET (runs stored from Oct 1, 2026, 9:56 AM ET). 16 stored runs. Methodology
A rank on this page is best_fit_for_inputs for one task, volume, and budget. It is not a universal winner.
Tasks
Five-bullet brief
five-bullet-brief · v1
Rewrite a note as exactly five bullets under a word cap. This tier-1 task measures format compliance only. It does not score whether the bullets are accurate.
Earnings call summary
earnings-call-summary · v1
Summarize an earnings-call excerpt into five bullets. The fixture is a short excerpt, not a full transcript, so this version does not measure a 2,000-word run. Scoring is a blinded rubric, not a hardcoded quality score.
Customer changelog draft
changelog-customer-draft · v1
Rewrite internal release notes into a customer-facing changelog. A blinded rubric scores accuracy, clarity, audience fit, and whether anything important was dropped.
Models: openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Methodology: v1
Published:
Candidate runs total: 0.00181860
Judge cost: 0.0004669
Total (candidate + judge): 0.00228550
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (16 runs).
Comparison ba117668-cc71-417b-a676-1e982d874a8e
Funding visitor-key / video-demo
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| google/gemini-2.5-flash | Meeting action items | Deterministic check | 580 ms | 0.0002295 | Run 3a2851ec | |
| openai/gpt-4o-mini | Meeting action items | Deterministic check | 482 ms | 0.0000519 | Run 198dc3a1 | |
| google/gemini-3.1-flash-lite | Meeting action items | Deterministic check | 1019 ms | 0.00013475 | Run 1a65178d | |
| deepseek/deepseek-chat-v3.1 | Meeting action items | Deterministic check | 531 ms | 0.00008417 | Run 58ade0b2 | |
| google/gemini-2.5-flash | Five-bullet brief | Deterministic check | 721 ms | 0.0001148 | Run bd6ed3cc | |
| google/gemini-3.1-flash-lite | Five-bullet brief | Deterministic check | 988 ms | 0.0000855 | Run 01fbc12d | |
| deepseek/deepseek-chat-v3.1 | Five-bullet brief | Deterministic check | 1239 ms | 0.00005863 | Run 52f01466 | |
| openai/gpt-4o-mini | Five-bullet brief | Deterministic check | 1060 ms | 0.00004095 | Run fa9c0387 | |
| google/gemini-2.5-flash | Earnings call summary | LLM judge | 574 ms | 0.0002474 | Run 74a20a81 | |
| google/gemini-3.1-flash-lite | Earnings call summary | LLM judge | 742 ms | 0.0001665 | Run e6c7f5ff | |
| deepseek/deepseek-chat-v3.1 | Earnings call summary | LLM judge | 730 ms | 0.00020045 | Run 63f45176 | |
| openai/gpt-4o-mini | Earnings call summary | LLM judge | 1193 ms | 0.00006525 | Run 51dff993 | |
| google/gemini-2.5-flash | Customer changelog draft | LLM judge | 472 ms | 0.0001547 | Run 7656aabb | |
| google/gemini-3.1-flash-lite | Customer changelog draft | LLM judge | 648 ms | 0.000104 | Run aea93a18 | |
| openai/gpt-4o-mini | Customer changelog draft | LLM judge | 674 ms | 0.00002685 | Run 6a7c2320 | |
| deepseek/deepseek-chat-v3.1 | Customer changelog draft | LLM judge | 1718 ms | 0.00005325 | Run 0e3ff8e9 |
Notable flags
No stored run in this comparison is flagged incomplete, mixed, or as an automated metadata check.
B-roll
Race, radar, run replay, fit walkthrough, PNG stills, and CSV stay marked todo. The links below are the artifacts that exist for this comparison.
JSON packet · Scatter · Checklist · Episode packet · Race (not generated)
Measured cost vs quality
Open in the interactive Bench · Comparison page · Episode packet · Checklist