Bench / comparisons
Published model comparisons
Each title is a published comparison. Costs and scores are on the comparison page and come from stored runs.
- Schema extraction, loop bug & long-thread summary · 4 models
Schema-bound extraction, Explain a subtle loop bug, Bat and ball, Long support-thread summary · openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Published · as of · Episode
- Bug explanation & support reply · 4 models
Explain a bug in plain language, Tone-matched support reply, Ambiguous ticket reply, Conflicting status update · openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Published · as of · Episode
- Meeting notes & customer summaries · 4 models
Meeting action items, Five-bullet brief, Earnings call summary, Customer changelog draft · openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Published · as of · Episode
- Invoice, receipt & intent extraction · 4 models
Invoice field extraction, Receipt total extraction, Support intent classification, Repair broken JSON · openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Published · as of · Episode
- Bug explanation & support reply · 4 models
Explain a bug in plain language, Tone-matched support reply · openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Published · as of · Episode
- Summaries & writing · 4 models
Meeting action items, Five-bullet brief, Earnings call summary, Customer changelog draft · openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Published · as of · Episode
- Extraction & classification · 4 models
Invoice field extraction, Receipt total extraction, Support intent classification, Repair broken JSON · openai/gpt-4o-mini, google/gemini-2.5-flash, google/gemini-3.1-flash-lite, deepseek/deepseek-chat-v3.1
Published · as of · Episode