Open Source Developer Tools API ProxyMulti-ProviderPythonOpenAI CompatibleLoad Balancing

LiteLLM

Python SDK and proxy server that standardises calls to OpenAI, Anthropic, Gemini, Bedrock, and 100+ models behind a single OpenAI-compatible interface.

BerriAI/litellm 19.0k stars 2.4k forks MIT

Install

$pip install litellm

About Getting Started Features References

About

LiteLLM's SDK wraps every provider's SDK in a single completion() call with the same input/output format as OpenAI. Code written for GPT-4o works unchanged with Claude, Gemini, or a local Ollama model by changing the model string.

The LiteLLM Proxy (litellm --model ...) starts a local OpenAI-compatible server. You point any OpenAI SDK at localhost and route traffic to any backend, with built-in load balancing, usage logging, and per-key budget limits.

Getting Started

pip install litellm

import litellm

# Same code works for any model
response = litellm.completion(
    model="anthropic/claude-sonnet-4-5",
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

# Or start the proxy
litellm --model anthropic/claude-sonnet-4-5