Meeting action items
meeting-action-items · v1
Extract action items as a JSON array of owner and task. The deterministic check compares the set of owners only. Wording of the task strings is not scored here; that would be a rubric.
Bench / Comparisons / Meeting notes & customer summaries · 4 models
Navigator Bench comparison
Meeting notes & customer summaries · 4 models
Meeting action items, Five-bullet brief, Earnings call summary, Customer changelog draft · openai/gpt-4o-mini vs google/gemini-2.5-flash vs google/gemini-3.1-flash-lite vs deepseek/deepseek-chat-v3.1
As of Oct 1, 2026, 8:23 PM ET (runs stored from Oct 1, 2026, 8:23 PM ET). 16 stored runs. Methodology
A rank on this page is best_fit_for_inputs for one task, volume, and budget. It is not a universal winner.
Tasks
Five-bullet brief
five-bullet-brief · v1
Rewrite a note as exactly five bullets under a word cap. This tier-1 task measures format compliance only. It does not score whether the bullets are accurate.
Earnings call summary
earnings-call-summary · v1
Summarize an earnings-call excerpt into five bullets. The fixture is a short excerpt, not a full transcript, so this version does not measure a 2,000-word run. Scoring is a blinded rubric, not a hardcoded quality score.
Customer changelog draft
changelog-customer-draft · v1
Rewrite internal release notes into a customer-facing changelog. A blinded rubric scores accuracy, clarity, audience fit, and whether anything important was dropped.
Models: openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Methodology: v1
Published:
Candidate runs total: 0.00178418
Judge cost: 0.0004657
Total (candidate + judge): 0.00224988
Amounts are the exact stored decimal strings. No rounding is applied. candidate_cost_usd is the sum of measured run costs. judge_cost_usd is the sum of stored tier-2 judge ledger rows, or null when none were recorded. total_cost_usd equals candidate plus judge only when every candidate cost is present and a judge cost was recorded.
Candidate costs are the measured amounts on the receipts below (16 runs).
Comparison 7fb16500-712d-4e11-bcab-20a9d05e61e5
Funding visitor-key / video-demo
Stored runs
| Model | Task | Tier | Score | Latency | Measured cost | Receipt |
|---|---|---|---|---|---|---|
| google/gemini-2.5-flash | Meeting action items | Deterministic check | 474 ms | 0.0002295 | Run 09df9762 | |
| google/gemini-3.1-flash-lite | Meeting action items | Deterministic check | 692 ms | 0.00013475 | Run efbf5b21 | |
| openai/gpt-4o-mini | Meeting action items | Deterministic check | 568 ms | 0.0000519 | Run 602d9a84 | |
| deepseek/deepseek-chat-v3.1 | Meeting action items | Deterministic check | 178 ms | 0.00008441 | Run 49ea913a | |
| google/gemini-3.1-flash-lite | Five-bullet brief | Deterministic check | 532 ms | 0.0000855 | Run f078e9be | |
| google/gemini-2.5-flash | Five-bullet brief | Deterministic check | 918 ms | 0.0001148 | Run 2481d57d | |
| openai/gpt-4o-mini | Five-bullet brief | Deterministic check | 505 ms | 0.00004095 | Run a456fef7 | |
| deepseek/deepseek-chat-v3.1 | Five-bullet brief | Deterministic check | 1976 ms | 0.0000601 | Run e4c47496 | |
| google/gemini-2.5-flash | Earnings call summary | LLM judge | 468 ms | 0.0002474 | Run 961a8dc7 | |
| openai/gpt-4o-mini | Earnings call summary | LLM judge | 427 ms | 0.00006765 | Run adebaa89 | |
| google/gemini-3.1-flash-lite | Earnings call summary | LLM judge | 514 ms | 0.0001665 | Run af33314b | |
| deepseek/deepseek-chat-v3.1 | Earnings call summary | LLM judge | 1069 ms | 0.00011562 | Run 1ec54c69 | |
| google/gemini-2.5-flash | Customer changelog draft | LLM judge | 656 ms | 0.0001547 | Run 9f12b9a0 | |
| deepseek/deepseek-chat-v3.1 | Customer changelog draft | LLM judge | 186 ms | 0.00009955 | Run 2a09ac45 | |
| google/gemini-3.1-flash-lite | Customer changelog draft | LLM judge | 749 ms | 0.000104 | Run d26f4e7a | |
| openai/gpt-4o-mini | Customer changelog draft | LLM judge | 1267 ms | 0.00002685 | Run c05c57fc |
Notable flags
No stored run in this comparison is flagged incomplete, mixed, or as an automated metadata check.
B-roll
Race, radar, run replay, fit walkthrough, PNG stills, and CSV stay marked todo. The links below are the artifacts that exist for this comparison.
JSON packet · Scatter · Checklist · Episode packet · Race (not generated)
Measured cost vs quality
Fit for your inputs
A rank is best_fit_for_inputs for one task, volume, and budget. It is computed from the stored runs on this page. It is not a universal winner, and it is not stored as its own score. Methodology
No fit rank is shown until you choose a task, volume, and budget.
Other published comparisons with this task and model set
Open in the interactive Bench · Episode page · Episode packet · Checklist