FELM
Meta-benchmark for evaluating factuality in large language models.
- Status
- Not yet determined
- Last checked
- Unverified
- License
- Open source
- Pricing
- Free tier · from $0
What it is
FELM is a meta-benchmark designed to evaluate how well different tools assess the factuality of outputs from large language models, aiding developers and researchers in understanding model reliability.
FELM stands out as the only meta-benchmark specifically designed to evaluate the factuality of outputs from large language models, providing a unique tool for researchers and developers focused on model reliability.
When it fits — and when it doesn’t
A good fit for
Researchers and developers who need to evaluate the factual accuracy of LLMs.
Teams working on improving model reliability through benchmarking.
Look elsewhere if
Users looking for real-time factuality evaluation in production environments.
Projects that require a graphical user interface (GUI) for interaction.
Known weaknesses
Each one links to the public issue, changelog or docs page it came from, where we have one.
- 01
Limited integrations with other observability tools
Lacks native support for popular monitoring systems like Prometheus or Grafana
What it does well
Evaluates the factuality of outputs from large language models.
Provides a standardized benchmark for comparing different evaluators.
Aids in understanding model reliability and accuracy.
Pricing
- Free tier
- YesOpen source — free to use
- Paid from
- $0
- Model
- Flat rate
- Enterprise plan
- No
Pricing not independently confirmed — check the vendor’s page.
Performance benchmarks
How Fast Is It?
Set up FELM
Setup steps, common gotchas and what to check after 30 days.