Leeky
Black box language model leakage/contamination testing tool
Pricing
See website
Flat rate
Adoption
→StableLicense
Open Source
Data freshness
—Overview
What is Leeky?
Leeky is an open-source tool designed to test for data leakage and contamination in black-box language models, ensuring the integrity of AI systems.
Key differentiator
“Leeky stands out as an open-source tool specifically designed for testing data leakage and contamination in black-box language models, offering detailed insights into potential risks.”
Capability profile
Strength Radar
Honest assessment
Strengths & Weaknesses
↑ Strengths
Fit analysis
Who is it for?
✓ Best for
Teams developing black-box language models who need to ensure no data leakage occurs
Organizations that require rigorous testing for AI safety and compliance
Researchers studying the security aspects of machine learning systems
✕ Not a fit for
Projects where real-time performance is critical (Leeky may be resource-intensive)
Teams without access to local compute resources (requires a local setup)
Cost structure
Pricing
Free Tier
None
Starts at
See website
Model
Flat rate
Enterprise
None
Performance benchmarks
How Fast Is It?
Next step
Get Started with Leeky
Step-by-step setup guide with code examples and common gotchas.