Browse tools in the Cloud Inference space.
40 tools
The @aws-cdk/aws-bedrock-alpha is a construct library that enables developers to define and deploy infrastructure as cod...
LMDeploy is a high-performance inference and serving framework designed to deliver fast and efficient deployment of larg...
LocalAI is a drop-in replacement REST API that’s compatible with OpenAI API specifications, enabling local inference wit...
MNN-LLM is a device-inference framework that enables efficient deployment of large language models (LLMs) directly onto ...
TensorRT-LLM is a high-performance inference framework from NVIDIA designed to optimize and deploy large language models...
Baseten provides model deployment and serving infrastructure (Truss, model APIs) with autoscaling, fast cold starts, and...
Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to writ...
DeepInfra offers cheap, simple inference for popular open-source LLMs and other models via a straightforward per-token A...
Kserve is a standardized distributed generative and predictive AI inference platform that supports multi-framework deplo...
A scalable inference server designed to deploy and manage machine learning models that have been optimized using the Ope...
This library provides a high-performance Node.js client for Baseten.co endpoints, including embeddings, reranking, and c...
Together AI runs hundreds of open-source models via a fast inference API with competitive pricing, dedicated endpoints, ...
Vespa is a powerful tool for storing, searching, organizing, and making machine-learned inferences over large datasets. ...
Bifrost is the fastest LLM gateway, offering just 11μs overhead at 5,000 RPS. It's designed to be significantly faster t...
Chitu is a high-performance inference framework designed to serve large language models efficiently and flexibly, making...
The @openrouter/ai-sdk-provider is an official provider for the Vercel AI SDK, offering seamless access to a wide array ...
Banana allows developers to host their machine learning models for inference on serverless GPUs, making it easy to integ...
Enchanted is an iOS and macOS application designed to facilitate chat interactions with privately hosted language models...
Fireworks AI is a fast inference platform for open-source and custom LLMs. It specializes in ultra-low latency inference...
Groq is an AI inference platform built on its proprietary Language Processing Unit (LPU) — custom silicon purpose-built ...
Quix is a serverless platform that enables developers to process and analyze data streams in real-time using machine lea...
The Scaleway SDK for Inference provides a set of tools to deploy, manage, and scale machine learning models. It is desig...
The Serverless AI Framework SDK provides a comprehensive set of tools to build, deploy, and manage serverless applicatio...
The Stability.ai REST API allows developers to integrate AI capabilities into their applications. Access requires DreamS...
Wllama provides a WebAssembly binding for llama.cpp, enabling developers to perform large language model inference direc...
Algorithmia provides a platform for deploying and consuming algorithms as APIs. It offers a marketplace where developers...
BudgetML allows developers to deploy machine learning inference services quickly and cost-effectively in less than 10 li...
Cerebras is a developer tool written in TypeScript, focusing on providing solutions for specific development challenges....
MII is a tool that enables low-latency and high-throughput inference, similar to vLLM, powered by DeepSpeed. It optimize...
Groq Cloud runs LLM models fast and cheap. This is a convenience client library for Ruby, allowing developers to easily ...
Showing 30 of 40
