Starting a LiteLLM gateway with Docker Compose

Docker Compose provides a practical bounded pilot for a LiteLLM gateway. Choose the operating mode first: a simple API check or a stateful setup for managing keys, models and usage. These modes do not establish the same capabilities, even when both return a model answer.

Inspect the official example

Start with the documented Compose setup and read the service definitions before launching. Check images, environment variables, ports and volumes. Fix the image version used for the pilot rather than accepting an implicit update every time the environment is recreated.

Use a separate pilot directory and keep secret-bearing files out of a public repository. Record where configuration and database data live. One successful inference request is not evidence that management records survive a restart or that budget enforcement works. Preserve the intended access boundary in the deployment notes.

Identify persistent state

Component	Question to answer
Gateway	Which port listens and which model names are allowed?
Postgres	Where are key, configuration and usage records stored?
Environment	Who owns credentials and rotation?
Volumes	What survives container recreation?

Virtual-key management has documented database requirements. Do not transfer assumptions from a database-backed installation to a minimal proxy without that state. An administrative key that authenticates does not by itself prove all management features are available. State persistence and request execution need separate acceptance checks.

Validate before starting

After preparing the official file and environment, an illustrative inspection command is:

docker compose -f docker-compose.quickstart.yml config --services

This checks configuration, not access to an external provider. Review intended bind addresses and ports. A pilot needed only from the local machine should not accidentally expose administration to unrelated network clients. Expanded configuration can contain secrets, so avoid publishing it without review.

Start the prepared stack through the documented procedure and record service states. Confirm database readiness before treating a virtual-key issuance failure as a model error. Keep safe diagnostic output associated with the tested image version.

Test one model and a restart

Begin with an authorized endpoint and separate application access instead of passing the administrative key to the client. Make a short request and verify the expected usage record. Restart the test setup and confirm required records persist. These are two results: the route works and its necessary state survives.

Define shutdown and retention before the pilot ends. Stopping a gateway does not remove resources billed in another cloud account. Preparation and operations can be discussed around observed results; a gateway does not create inference demand automatically.

Official references: docs.litellm.ai — docker quick start, docs.litellm.ai — virtual keys.

Related guides: LiteLLM virtual keys: separate access for each application, Preparing LiteLLM UI for model and key management.