Observability & Monitoring

SHAP

SHAP provides a unified measure of feature importance by applying game theory. It helps in understanding how each featur...

Shapash

Shapash is a Python library that provides several types of visualization to display explicit labels everyone can underst...

Weights and Biases

Weights and Biases is a tool for visualizing and tracking machine learning experiments. It helps data scientists and dev...

BertViz

BertViz is a tool for visualizing attention mechanisms within transformer models like BERT. It helps developers and rese...

Sacred

Sacred is a Python tool designed to help configure, organize, log, and reproduce experiments. It provides a standard fra...

TensorWatch

TensorWatch is a powerful tool that provides real-time visualizations of data in running processes such as machine learn...

TraceRoot AI

TraceRoot AI provides intelligent agents that automatically identify and resolve bugs in your production environment. Th...

@langfuse/langchain

Langfuse provides detailed insights into the performance of your LangChain applications. It helps in debugging, auditing...

Azure's PyRIT

PyRIT is an open-source tool that enhances observability and implements guardrails for AI safety within MLOps pipelines....

Dowel

Dowel is a logging tool designed specifically for machine learning researchers. It simplifies the process of logging dat...

Instrumentation Anthropic

This package provides instrumentation capabilities for Anthropic models, enabling developers to track and analyze the pe...

Langchain Visualizer

Langchain Visualizer is a tool designed to help developers visualize and debug their LangChain workflows, making it easi...

MLNotify

MLNotify simplifies the monitoring of machine learning model training by sending notifications upon completion, eliminat...

Pastalog

Pastalog provides simple and real-time visualization for monitoring the performance of neural networks during training. ...

scikit-plot

Scikit-plot is a visualization library that provides easy-to-use functions to generate common plots used in machine lear...

Yellowbrick

Yellowbrick provides visual diagnostics to help data scientists choose the right machine learning models. It integrates ...

Shapley

A data-driven framework to quantify the value of classifiers within a machine learning ensemble. Shapley helps understan...

ORB-SLAM

ORB-SLAM is a real-time simultaneous localization and mapping (SLAM) library that processes images from a single camera ...

Arize

Arize provides a comprehensive solution for monitoring and observing machine learning models in production. It helps tea...

Comet

Comet is an ML platform that helps users track experiments, hyper-parameters, artifacts, and more. It integrates with ov...

Helicone

Helicone is an open-source platform that enables developers to monitor, evaluate, and experiment with large language mod...

Langfuse

Langfuse is an open-source platform designed to help developers trace, evaluate, manage prompts, and test large language...

Openllmetry

Openllmetry is an open-source observability tool designed specifically for Large Language Model applications, leveraging...

YOLOv5

YOLOv5 is a state-of-the-art real-time object detection model built on PyTorch, offering high performance and flexibilit...

Instrumentation Openai

This tool provides observability features for applications using the `openai` client library, enabling detailed monitori...

DVClive

DVClive is a Python library that simplifies the process of tracking and logging experiment metrics by storing them in ea...

Eurybia

Eurybia is a tool that monitors data and model drift over time, ensuring the integrity of machine learning models throug...

Neptune AI

Neptune AI is a powerful yet lightweight tool designed to manage machine learning experiments efficiently. It integrates...

Laminar

Laminar is an open-source observability platform designed specifically for monitoring and evaluating AI agents. It provi...

aiWARE

aiWARE helps MLOps teams evaluate, deploy, integrate, scale & monitor ML models efficiently. It streamlines the entire l...

Arize-Phoenix

Arize-Phoenix is an open-source tool designed to monitor and fine-tune machine learning models, including LLMs, CV model...

Fiddler AI

The all-in-one AI Observability and Security platform that helps enterprises operationalize ML, GenAI, and LLM applicati...

Guild.ai

Guild.ai is a tool to log, analyze, compare, and optimize machine learning experiments. It supports cross-platform opera...

Katonic

Katonic is an MLOps platform that automates the machine learning lifecycle, from model training to deployment and monito...

LangSmith

LangSmith is a comprehensive tool from the LangChain framework designed to facilitate evaluation, collaboration in Human...

Openinference Semantic Conventions

This package provides semantic conventions for tracing Large Language Model (LLM) applications, enabling better observab...

Superwise

Superwise provides fully automated observability for machine learning models in a self-service SaaS platform, enabling t...

FastTrackML

FastTrackML is an open-source experiment tracking server designed to handle large-scale machine learning experiments wit...

Mandala

Mandala is an experiment tracking framework that integrates persistence logic and best practices directly into Python, m...

MLEM

MLEM allows you to version control your machine learning models and deploy them using GitOps practices, ensuring reprodu...

NannyML

NannyML is a Python library that helps in capturing the impact of data drift on model performance after deployment. It a...

Radicalbit

Radicalbit is an open-source tool designed to monitor and ensure the performance of AI models once they are deployed int...

Whylogs

Whylogs is an open-source library that enables developers to log and monitor their machine learning models' performance....

Grafana

Grafana is a powerful open-source platform for monitoring and observability. It provides rich visualizations, customizab...

Netdata

Netdata provides real-time monitoring and alerting with AI-driven insights to help developers and operations teams quick...

OpenMetadata

OpenMetadata is a platform that provides a single place for discovering, collaborating on, and ensuring the accuracy of ...

PostHog

PostHog is an all-in-one, open-source product analytics platform: event analytics, session replay, feature flags, A/B te...

Redash

Redash allows you to connect to various data sources, create visualizations, build dashboards, and share insights with o...

Sentry

Sentry is the developer-favorite error tracking and performance monitoring tool, capturing exceptions with rich context,...

SigNoz

SigNoz is an open-source APM and observability platform built on OpenTelemetry, unifying metrics, traces, and logs; self...

Superset

Superset is a powerful data visualization tool that allows users to explore large datasets through an intuitive, user-fr...

axe-core

Axe-core is an open-source accessibility engine that helps developers identify and fix issues in their web UIs, ensuring...

Axiom

Axiom is a logging and event-analytics platform designed for very low cost at scale, ingesting all your logs/events with...

chaostoolkit

chaostoolkit is a Chaos Engineering toolkit and orchestration tool designed to help developers test the resilience of th...

Datadog

Datadog is a comprehensive observability platform unifying infrastructure metrics, APM/tracing, log management, RUM, and...

DataHub

DataHub is LinkedIn's open-source platform designed to help organizations discover and understand their data assets by p...

Detect Agent

This tool helps developers determine whether their code is executing within an AI-driven automation context, aiding in t...

Django Debug Toolbar

The Django Debug Toolbar is a configurable set of panels that help you view and monitor variables, SQL queries, template...

Flask Debug Toolbar

The Flask Debug Toolbar is a powerful tool that provides detailed information about each request and response, aiding de...

fsociety

Fsociety is an open-source penetration testing framework designed to help security professionals and developers identify...

hook-std

hook-std is a utility that allows developers to intercept and manipulate the standard output and error streams in Node.j...

Insight

Insight helps developers understand how their tools are being used by anonymously reporting usage metrics to Google Anal...

log-symbols

Log-symbols is a simple Node.js library that provides colored symbols for different log levels, making console outputs m...

log-update

Log-update is a utility that allows overwriting the previous console output in real-time. This makes it ideal for creati...

New Relic

New Relic provides APM, infrastructure, logs, and full-stack observability with a consumption-based pricing model and a ...

observable-to-promise

This tool simplifies the process of converting Observables into Promises, making it easier to work with asynchronous ope...

psutil

psutil is a cross-platform library for retrieving information on running processes and system utilization (CPU, memory, ...

Sentry Python

Sentry Python is the official SDK for integrating Python applications with Sentry. It provides robust error tracking, pe...

Tape

Tape is a simple testing framework that generates Test Anything Protocol (TAP) output, making it easy to integrate with ...

Watchdog

Watchdog is a powerful tool for monitoring changes in the filesystem. It provides both an API and command-line utilities...

Winston

Winston is a fast, versatile, and widely-used logging library that supports multiple transports. It allows developers to...

Aim

Aim is an open-source tool designed to track and visualize machine learning experiments. It supports a variety of framew...

Consola

Consola is a powerful console logger that enhances the traditional console.log() function, providing more readable outpu...