FELM
Meta-benchmark for evaluating factuality in large language models.
Pricing
Free tier
Flat rate
Adoption
→StableLicense
Open Source
Data freshness
UnverifiedOverview
What is FELM?
FELM is a meta-benchmark designed to evaluate how well different tools assess the factuality of outputs from large language models, aiding developers and researchers in understanding model reliability.
Key differentiator
“FELM stands out as the only meta-benchmark specifically designed to evaluate the factuality of outputs from large language models, providing a unique tool for researchers and developers focused on model reliability.”
Capability profile
Capability Radar
Honest assessment
Strengths & Weaknesses
↑ Strengths
↓ Weaknesses
API requires Python-specific patterns, TypeScript SDK is community-maintained
v0.1 to v0.2 migration required rewriting chain definitions
Lacks native support for popular monitoring systems like Prometheus or Grafana
Requires manual configuration of multiple dependencies and environment variables
Fit analysis
Who is it for?
✓ Best for
Researchers and developers who need to evaluate the factual accuracy of LLMs.
Teams working on improving model reliability through benchmarking.
✕ Not a fit for
Users looking for real-time factuality evaluation in production environments.
Projects that require a graphical user interface (GUI) for interaction.
Cost structure
Pricing
Free Tier
Available
Open source — free to use
Starts at
$0
Model
Flat rate
Enterprise
None
Performance benchmarks
How Fast Is It?
Ecosystem
Relationships
Integrations
Next step
Get Started with FELM
Step-by-step setup guide with code examples and common gotchas.