A gateway sits between your applications and the model providers and presents one API. LiteLLM is the project most people mean; "gateway" has also become a pattern with several implementations. Notes on when it earns its place.
What LiteLLM does. As a library, it translates one call shape (the OpenAI format) to over a hundred providers. As a proxy (litellm --config config.yaml), it adds the things a team needs: virtual API keys per user or project with budgets, spend logging per key and per model, fallbacks when a provider errors or rate-limits, retries, and routing between deployments. The core is MIT; some enterprise features are paid. The proxy needs Postgres once you use keys and budgets.
When you need it. More than one provider in use, or the plan to switch. More than one team or application sharing a bill and needing to know who spent what. A provider outage that must not be an incident. Any of these is enough.
When you do not. One application, one provider, one key. The SDK is fine, and the gateway is an extra hop, an extra process and an extra config file to keep in sync with the model names. Add it when the second provider arrives, not before.
The hidden cost. Model names. A gateway gives you aliases (gpt-4o can point anywhere), which is the feature, and also means the name in your code no longer tells you what you are calling. Name the aliases by purpose (chat-fast, extract-json) rather than by model, or the indirection buys nothing.
/learn/llm-gateway covers the pattern; /compare/gateway-vs-litellm compares LiteLLM with another implementation of it.