Evaluation & Testing

Agenta

Agenta is an open-source platform designed for teams to collaboratively build and manage AI agents and automations. It s...

ChatGPT-Shortcut

ChatGPT-Shortcut is an ultimate hub for managing, customizing, and sharing prompts to enhance productivity. It supports ...

Ciphey

Ciphey is an open-source tool that automatically decrypts various types of encryption, decodes encodings, and cracks has...

Deepeval

Deepeval is an open-source framework designed to evaluate the performance of large language models. It provides a struct...

garak

Garak is an open-source tool designed to scan and identify vulnerabilities in large language models. It helps developers...

LangWatch

LangWatch is an open-source platform designed to provide observability into Large Language Models, evaluate prompts effe...

Presidio

Presidio is an open-source framework that detects, redacts, masks, and anonymizes personal information across text, imag...

Promptfoo

Promptfoo is an open-source tool designed to evaluate and secure large language models by testing prompts, models, agent...

sqlmap

sqlmap is an open-source penetration testing tool that automates the process of detecting and exploiting SQL injection f...

Beelzebub ChatGPT Honeypot

Beelzebub is a tool that creates a honeypot environment mimicking the behavior of ChatGPT, aimed at identifying and capt...

Evidently

Evidently provides interactive reports to analyze machine learning models during validation or production monitoring, he...

Hypothesis Tests

HypothesisTests.jl is a Julia package providing a wide range of hypothesis tests used in statistical inference, making i...

Inciteful

Inciteful uses citation data to help researchers and academics explore and understand the connections within academic li...

NALP

NALP is a Natural Adversarial Language Processing framework that leverages TensorFlow for advanced NLP tasks. It provide...

Plugin Secure Proxy

Orama plugin for generating embeddings securely on the front-end, enabling developers to integrate AI capabilities direc...

SET

The Social-Engineer Toolkit (SET) is a Python-driven tool that automates many of the steps in social-engineering attacks...

Snyk

Snyk is a developer-first security platform for scanning code (SAST), open-source dependencies (SCA), containers, and Ia...

SonarQube

SonarQube provides static analysis for code quality and security (bugs, vulnerabilities, code smells, coverage) across 3...

Trulens

Trulens provides tools to evaluate and track the performance of large language models and AI agents, enabling developers...

vulture

Vulture is a tool that helps developers identify unused code in their Python projects, aiding in the maintenance of clea...

AIF360

AIF360 is an open-source toolkit developed by IBM Research that provides a comprehensive set of fairness metrics for dat...

COMPAS Analysis Using Aequitas

A tool to analyze and mitigate bias in machine learning models, specifically using the COMPAS recidivism prediction data...

Deepchecks

Deepchecks provides comprehensive validation and testing of machine learning models during development, deployment, and ...

JetBrains Qodana

JetBrains Qodana is an automated code analysis tool that helps developers refactor their code, identify security vulnera...

OpenAI Evals

An open-source library for evaluating task performance of language models and prompts. It helps developers understand ho...

Prompttools

Prompttools is an open-source library that provides developers with the necessary tools to test and evaluate various AI ...

Ragas

Ragas is a framework designed to help developers evaluate and improve their Retrieval Augmented Generation (RAG) pipelin...

Inspector

Inspector is a visual testing tool designed specifically for Model Context Protocol (MCP) servers. It helps developers e...

Pentagi

Pentagi is an advanced system that leverages AI to perform sophisticated penetration testing. It automates the process o...

Prompt Optimizer

Prompt Optimizer is a tool designed to help users create high-quality prompts by optimizing the input text. It's particu...