LiteLLM

LiteLLM is an open-source AI gateway and Python SDK from BerriAI that standardizes access to 100+ LLM providers behind a single OpenAI-compatible API. Developers use it to route, load-balance, and fail over between models while tracking spend and enforcing budgets, guardrails, and rate limits. The core is free to self-host, and an Enterprise tier adds SSO, RBAC, audit logs, and SLA-backed support for organizations that need centralized governance over model access.

LiteLLM addresses a common headache in multi-model AI development: every provider has a slightly different API, authentication scheme, and response shape. The open-source core normalizes all of them into the familiar OpenAI request/response format, so application code can swap or fan out across providers without rewrites. It supports chat, completion, embeddings, image generation, and more across 100+ providers. Beyond translation, LiteLLM adds gateway features that teams need in production: virtual API keys and budgets, spend and usage tracking per key or team, retries and fallbacks, load balancing across deployments, rate limiting, and guardrails/logging hooks. The proxy can be deployed in your own infrastructure, and the project has a very active GitHub presence with a large contributor community. In 2024 the core was rewritten with performance in mind (a Rust core with the Python SDK). The project is free to self-host under an open-source license. BerriAI, the company behind it, monetizes through an Enterprise tier that adds SSO/SAML, RBAC, audit logs, SLA-backed support, and a dedicated support channel. This makes LiteLLM appealing both to individual developers who want a free abstraction layer and to enterprises that need a governed, centralized gateway in front of many model providers.