Evaluation & Testing

Abstruse

Abstruse is a powerful CI/CD tool designed to streamline the integration and deployment processes in software developmen...

Adrenaline

Adrenaline is a debugger tool that not only identifies but also fixes errors in code, providing explanations using GPT-3...

Agentic Radar

Agentic Radar is an open-source CLI tool that scans your workflow's source code, detects vulnerabilities, and generates ...

Agentic Security

Agentic Security is an LLM-based vulnerability scanner designed for red teaming activities. It helps identify potential ...

AI Gateway Provider

The AI Gateway Provider is a tool that integrates the AI-SDK with Cloudflare's AI Gateway, providing developers with adv...

AI SDK Guardrails

AI SDK Guardrails provides middleware to enforce guardrails on inputs and outputs of applications built with the Vercel ...

AIF360

A comprehensive set of fairness metrics to evaluate the bias in datasets and machine learning models, crucial for ensuri...

Airtrain

Airtrain is a no-code batch compute platform designed to streamline the process of evaluating and fine-tuning large lang...

Aisheets

Aisheets allows users to build, enrich, and transform datasets using AI models seamlessly. It is designed for those who ...

ajv

ajv is a high-performance JSON Schema validation tool that supports the latest schema versions. It's designed to validat...

Algorithm Impact Assessment Toolkit

The Algorithm Impact Assessment Toolkit helps organizations evaluate the potential impacts of their AI systems, ensuring...

Algorithmia

Algorithmia provides secure governance for machine learning operations and supports the entire ML lifecycle, from model ...

Alibi

Alibi is an open-source Python library that enables developers to inspect and interpret machine learning models, enhanci...

Alibi Detect

Alibi Detect is an open-source Python library that provides tools for detecting outliers, adversarial attacks, and data ...

AlpacaEval

AlpacaEval is an Automatic Evaluator designed to assess the performance of instruction-following language models using t...

Aporia

Aporia provides observability with customized monitoring and explainability features specifically designed for machine l...

Arize AI

Arize AI provides comprehensive model validation, drift detection, explainability, and visualization capabilities across...

Arthur Shield

Arthur Shield is a paid product designed to detect various forms of harmful content including toxicity, hallucinations, ...

athina-evals

Athina-Evals is a Python SDK that enables developers to run evaluations on Large Language Model generated responses, pro...

Atlas Guardrails

Atlas Guardrails CLI helps developers implement safety and compliance guardrails in their machine learning operations an...

Atmo

Atmo is an open-source tool that allows developers to create server-side mocks of APIs, enabling them to test their appl...

AVA

AVA is a fast and futuristic test runner that supports the latest language features. It's designed to be easy to use, wi...

Aws Bedrock Alpha

The @aws-cdk/aws-bedrock-alpha is a construct library that enables developers to define and deploy infrastructure as cod...

axe-core

Axe-core is an open-source accessibility engine that helps developers identify and fix issues in their web UIs, ensuring...

Azure AI Content Safety

Azure AI Content Safety uses machine learning to detect potentially offensive or unsafe text, images, and videos. It hel...

Azure's PyRIT

PyRIT is an open-source tool that enhances observability and implements guardrails for AI safety within MLOps pipelines....

Benchmark.js

Benchmark.js is a JavaScript library that provides high-resolution timers and returns statistically significant results....

Berkeley Function-Calling Leaderboard

The Berkeley Function-Calling Leaderboard assesses the capability of large language models to call external functions or...

Blinky Debugging Agent

The Blinky Debugging Agent is an open-source tool designed to help developers automatically identify and resolve bugs wi...

Captum

Captum provides a suite of tools to help understand how machine learning models make predictions, specifically designed ...