How to Self-Host LiteLLM Proxy with Docker Compose on Ubuntu 24.04

TL;DR:

• Run LiteLLM proxy v1.104.0 from the pinned ghcr.io/berriai/litellm:v1.104.0 image with PostgreSQL 16 in Docker Compose, published only on 127.0.0.1:4000.
• Nginx Proxy Manager serves it at https://litellm.example.com. Docker-published ports bypass UFW, so the 127.0.0.1 binding keeps port 4000 private.
• LITELLM_MASTER_KEY is the admin credential, and v1.104.0 refuses to start without one. LITELLM_SALT_KEY encrypts stored provider keys and must never change.
• config.yaml routes gpt-6-astra and claude-opus-5-5 with fallbacks between them, and each app gets a virtual key limited to its models.
• n8n, Open WebUI, OpenClaw and Codex call the OpenAI-style /v1 endpoint; Claude Code calls /v1/messages.
• LiteLLM's production checklist sizes each proxy worker at 1 vCPU and 4 GiB of memory as a floor, and this stack runs the default single worker.

Applies to: Ubuntu 24.04 LTS · LiteLLM v1.104.0 · PostgreSQL 16 · Nginx Proxy Manager 2.16.0 · Docker Compose 5.6.0 · checked October 2026

LiteLLM proxy, also called the LiteLLM AI Gateway, is an open-source gateway from BerriAI, MIT-licensed outside its enterprise directory, that puts 100+ model providers behind one OpenAI-compatible API with virtual keys, spend tracking, budgets and fallbacks. This tutorial self-hosts LiteLLM proxy v1.104.0 on Ubuntu 24.04 with Docker Compose and PostgreSQL, adds HTTPS through Nginx Proxy Manager, and connects n8n, Open WebUI, OpenClaw, Codex and Claude Code with one virtual key each. Apps hold only their virtual keys, and provider API keys stay on the server.

How the LiteLLM proxy stack works

Only Nginx Proxy Manager accepts connections from the internet:

LiteLLM checks each request's virtual key, calls the provider with the provider key from .env, and records spend per key in PostgreSQL. Virtual keys, budgets and the Admin UI need the database.

LiteLLM's March 2026 security notice reports that the PyPI packages litellm 1.82.7 and 1.82.8 contained a malicious payload and were live for about 40 minutes on March 24, 2026, before PyPI quarantined them, and that customers running the official Docker image were not impacted. This tutorial uses that image with a pinned tag.

Prerequisites

• A VPS running Ubuntu 24.04 LTS. LiteLLM's production checklist gives each proxy worker 1 vCPU and 4 GiB of memory as a floor, and PostgreSQL and Nginx Proxy Manager need memory on top. The Compose file runs LiteLLM's default single worker; each extra worker set with NUM_WORKERS adds 1 vCPU and 4 GiB to that floor. On Arct Cloud, General Purpose plans such as vm.tiny have 4 vCPU, 8 GB RAM and 80 GB NVMe storage, and sustained full-CPU load is not permitted, so send requests to local models on demand.
• A non-root user with sudo privileges and SSH key login. See How to Generate an SSH Key on Windows, macOS, and Linux.
• Docker Engine and the Docker Compose plugin from How to Install Docker and Docker Compose on Ubuntu 24.04 and 26.04.
• Nginx Proxy Manager and its shared proxy Docker network from How to Install Nginx Proxy Manager with Docker on Ubuntu 24.04.
• A DNS A record for a subdomain such as litellm.example.com that points to your server's IP address.
• An OpenAI API key and an Anthropic API key, or keys from two other providers.

Commands run as the sudo user unless a section names another user. Replace values written in capitals, such as YOUR_SERVER_IP, with your own.

Step 1: Create the project directory and secrets

Compose reads variables from a .env file in the project directory:

mkdir ~/litellm
cd ~/litellm

This block writes .env with the pinned version, two generated LiteLLM keys and a database password. umask 077 makes the file readable only by your user:

(
umask 077
cat > .env <<EOF
LITELLM_VERSION=v1.104.0
LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)
LITELLM_SALT_KEY=sk-$(openssl rand -hex 32)
POSTGRES_PASSWORD=$(openssl rand -hex 24)
OPENAI_API_KEY=YOUR_OPENAI_API_KEY
ANTHROPIC_API_KEY=YOUR_ANTHROPIC_API_KEY
EOF
)

Open the file with nano .env and replace the two provider placeholders, so the keys stay out of your shell history. Save with Ctrl+O and exit with Ctrl+X.

• LITELLM_MASTER_KEY authorizes management API calls and is the Admin UI password. Since v1.104.0, the proxy refuses to start with an unset or empty master key or the old example value sk-1234.
• LITELLM_SALT_KEY encrypts provider credentials stored in PostgreSQL. It cannot be rotated: a new value makes every stored credential unreadable.

Print both keys and save them in your password manager, apart from the database backups:

grep -E '^LITELLM_(MASTER|SALT)_KEY=' .env

Step 2: Write the LiteLLM config.yaml

LiteLLM needs the address range of the proxy network to tell Nginx Proxy Manager apart from clients:

docker network inspect proxy --format '{{range .IPAM.Config}}{{.Subnet}} {{end}}'

The output looks similar to this:

172.18.0.0/16

Create the file with nano config.yaml, with that range in place of YOUR_PROXY_SUBNET:

model_list:
  - model_name: gpt-6-astra
    litellm_params:
      model: openai/gpt-6-astra
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude-opus-5-5
    litellm_params:
      model: anthropic/claude-opus-5-5
      api_key: os.environ/ANTHROPIC_API_KEY

router_settings:
  fallbacks:
    - gpt-6-astra: ["claude-opus-5-5"]
    - claude-opus-5-5: ["gpt-6-astra"]

litellm_settings:
  request_timeout: 600

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  enforce_fallback_model_access: true
  trusted_proxy_ranges: ["YOUR_PROXY_SUBNET"]
  use_x_forwarded_for: true
  mcp_trusted_proxy_ranges: ["YOUR_PROXY_SUBNET"]

• model_name is the name clients send, and litellm_params.model is the provider prefix plus the provider's model ID. os.environ/ reads keys from the container environment. Both IDs appear on OpenAI's and Anthropic's model pages in October 2026.
• fallbacks sends a failed gpt-6-astra request to claude-opus-5-5, and the reverse.
• enforce_fallback_model_access skips fallback targets the calling key may not use, so the Claude-only key from Step 5 never fails over to a GPT model.
• request_timeout: 600 ends a hanging request after 10 minutes instead of the default 6,000 seconds.
• trusted_proxy_ranges marks Nginx Proxy Manager as a proxy, so the Admin UI sign-in limit of 10 wrong passwords in 60 seconds counts each client address. With the optional IPv6 port mappings from the Nginx Proxy Manager tutorial, IPv6 visitors arrive from the Docker gateway address and share one count.
• use_x_forwarded_for with mcp_trusted_proxy_ranges lets LiteLLM read X-Forwarded-Proto: https from Nginx Proxy Manager, so Admin UI cookies on https://litellm.example.com are marked Secure.

Add a local Ollama model (optional)

LiteLLM reaches Ollama through the ollama_chat/ prefix. Ollama from the Ollama on Ubuntu tutorial listens on the host's 127.0.0.1, which the container cannot reach. Run Ollama as a container on the proxy network instead. It publishes no host port, so 11434 stays off the internet:

docker run -d --name ollama --network proxy --restart unless-stopped \
  -v ollama:/root/.ollama ollama/ollama:0.35.1

Pull the model inside the container:

docker exec ollama ollama pull qwen3.5:4b

Then add this under model_list:

  - model_name: qwen-local
    litellm_params:
      model: ollama_chat/qwen3.5:4b
      api_base: http://ollama:11434

A CPU server stays busy while a local model generates, so keep batch work on the API models.

Step 3: Write the Docker Compose file

Open nano compose.yaml, paste this file, then save and exit:

services:
  litellm:
    image: ghcr.io/berriai/litellm:${LITELLM_VERSION:?set it in .env}
    restart: unless-stopped
    command: ["--config", "/app/config.yaml", "--port", "4000"]
    ports:
      - "127.0.0.1:4000:4000"
    environment:
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:?set it in .env}
      LITELLM_SALT_KEY: ${LITELLM_SALT_KEY:?set it in .env}
      DATABASE_URL: postgresql://litellm:${POSTGRES_PASSWORD}@db:5432/litellm
      STORE_MODEL_IN_DB: "True"
      OPENAI_API_KEY: ${OPENAI_API_KEY}
      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
      NO_DOCS: "True"
      NO_REDOC: "True"
      NO_OPENAPI: "True"
    volumes:
      - ./config.yaml:/app/config.yaml:ro
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test:
        - CMD-SHELL
        - python3 -c "import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness')"
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    networks:
      - default
      - proxy

  db:
    image: postgres:16
    restart: unless-stopped
    environment:
      POSTGRES_USER: litellm
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: litellm
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U litellm -d litellm"]
      interval: 5s
      timeout: 5s
      retries: 10

volumes:
  postgres_data:

networks:
  proxy:
    external: true

• 127.0.0.1:4000:4000 publishes LiteLLM on loopback only. The Docker tutorial's daemon default does the same for a plain 4000:4000. The explicit address keeps the port private on a server without that default, where a published port bypasses UFW.
• The proxy network lets Nginx Proxy Manager reach litellm:4000. PostgreSQL publishes no port.
• STORE_MODEL_IN_DB lets you add models and provider keys in the Admin UI, stored encrypted with the salt key. Settings changed in the UI override config.yaml.
• NO_DOCS, NO_REDOC and NO_OPENAPI turn off the API docs, which LiteLLM serves without authentication by default.

The health check comes from LiteLLM's own Compose file, which also uses PostgreSQL 16.

Step 4: Start LiteLLM and check readiness

Pull the images, about 560 MB compressed, and start both containers:

docker compose up -d

LiteLLM waits for PostgreSQL, applies its database migrations and starts. When docker compose ps shows (healthy) in the litellm container's status, check readiness and the version:

curl -s http://127.0.0.1:4000/health/readiness
docker compose exec litellm litellm --version

{"status":"healthy","db":"connected"}
LiteLLM: Current Version = 1.104.0

To open the Admin UI, start an SSH tunnel from your own computer, as in the SSH tunnel tutorial:

ssh -N -L 4000:127.0.0.1:4000 YOUR_USER@YOUR_SERVER_IP

Open http://localhost:4000/ui and sign in as admin with the master key. LiteLLM's security guide treats that login as a bootstrap step. Under Internal Users, click + Invite User, set the role to proxy_admin, open the invitation link to set a password, and sign in with that account. Then add disable_env_credential_login: true under general_settings in config.yaml and run docker compose restart litellm on the server in ~/litellm. Turning it on before that account has a password locks you out of the Admin UI.

Step 5: Create a virtual key for each app

A virtual key is a LiteLLM API key for one app, with its own model list, budget and rate limits. Create the n8n key with the master key:

cd ~/litellm
MASTER_KEY=$(grep '^LITELLM_MASTER_KEY=' .env | cut -d= -f2)
curl -s http://127.0.0.1:4000/key/generate \
  -H "Authorization: Bearer $MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key_alias": "n8n", "models": ["gpt-6-astra", "claude-opus-5-5"]}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["key"])'

The command prints a key that starts with sk-. LiteLLM stores only its hash, so save it now. Repeat with these values:

/key/generate also accepts max_budget with budget_duration, such as "30d", for a spend limit that resets, and rpm_limit for requests per minute. To revoke a key, post {"key_aliases": ["openclaw"]} to /key/delete.

Step 6: Test a virtual key

Send a chat request with the n8n key:

curl -s http://127.0.0.1:4000/v1/chat/completions \
  -H "Authorization: Bearer YOUR_N8N_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "claude-opus-5-5", "messages": [{"role": "user", "content": "Say hello in five words."}]}'

The reply is OpenAI-format chat.completion JSON, with the answer in choices[0].message.content. Repeat the request with the Codex key: LiteLLM rejects it with The requested model 'claude-opus-5-5' is not available for this API key.

LiteLLM's docs advise testing fallbacks with a real provider error on a test setup. The x-litellm-attempted-fallbacks response header shows 0 for a direct response and 1 or more when a fallback model served it.

Step 7: Add HTTPS with Nginx Proxy Manager

In the Nginx Proxy Manager admin panel, open Hosts > Proxy Hosts and click Add Proxy Host. On the Details tab, set Domain Names to litellm.example.com, Scheme to http, Forward Hostname / IP to litellm and Forward Port to 4000. On the SSL tab, choose Request a new Certificate and turn on Force SSL and HTTP/2 Support.

Nginx Proxy Manager waits 90 seconds for a reply by default, and an agent request can take longer. On the gear-icon tab, add this to Custom Nginx Configuration, then click Save:

proxy_read_timeout 600s;

If your provider runs a network firewall in front of the server, open ports 80 and 443 there too. Test from any computer:

curl -s https://litellm.example.com/health/readiness

{"status":"healthy","db":"connected"}

Step 8: Connect your apps and coding agents to LiteLLM

Each client needs the gateway address and its own key: http://127.0.0.1:4000 on the same server, http://litellm:4000 from containers on the proxy network, and https://litellm.example.com from anywhere else.

n8n

n8n from the n8n Docker Compose tutorial joins the proxy network in the Nginx Proxy Manager tutorial. Create an OpenAI credential with the n8n key as API Key and http://litellm:4000/v1 as Base URL. In an OpenAI Chat Model node, the Model list then shows the key's models.

Open WebUI

Open WebUI from the Ollama tutorial uses host networking, so it reaches the loopback port. Open Settings > Admin > Connections, click Add Connection under Manage OpenAI API Connections, and enter http://127.0.0.1:4000/v1 as URL and the open-webui key as API Key. Click Verify Connection, then Save.

OpenClaw

OpenClaw from the OpenClaw tutorial runs as the openclaw user, and its LiteLLM provider defaults to http://localhost:4000. Run onboarding as that user and paste the openclaw key at the Enter LiteLLM API key prompt:

sudo -iu openclaw
openclaw onboard --auth-choice litellm-api-key

Onboarding sets the default model to litellm/claude-opus-4-6, which config.yaml does not define. Switch it and restart the gateway:

openclaw models set litellm/claude-opus-5-5
systemctl --user restart openclaw-gateway.service

Codex CLI

Codex from the Codex CLI tutorial runs as the coder user and reads custom providers from ~/.codex/config.toml. Connect with ssh coder@YOUR_SERVER_IP, then add these lines at the top of that file, before any other table:

model = "gpt-6-astra"
model_provider = "litellm"

[model_providers.litellm]
name = "LiteLLM"
base_url = "https://litellm.example.com/v1"
env_key = "LITELLM_API_KEY"
wire_api = "responses"

Add the key to ~/.bashrc with nano ~/.bashrc, then run source ~/.bashrc:

export LITELLM_API_KEY="YOUR_CODEX_KEY"

Start codex and run /status to confirm the provider. If Codex prints Model metadata for gpt-6-astra not found, requests still go through with fallback metadata.

Claude Code

LiteLLM serves Anthropic's Messages format at /v1/messages, and Anthropic supports only Claude models through a gateway. Test the claude-code key:

curl -sS -X POST https://litellm.example.com/v1/messages \
  -H "Authorization: Bearer YOUR_CLAUDE_CODE_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "claude-opus-5-5", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'

A JSON reply with a content field means the key works. For Claude Code from the Claude Code on a VPS tutorial, merge this env block into the coder user's ~/.claude/settings.json, never into a project's committed .claude/settings.json:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://litellm.example.com",
    "ANTHROPIC_AUTH_TOKEN": "YOUR_CLAUDE_CODE_KEY",
    "ANTHROPIC_MODEL": "claude-opus-5-5",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "claude-opus-5-5"
  }
}

Claude Code sends background tasks such as session titles to its Haiku model, so ANTHROPIC_DEFAULT_HAIKU_MODEL points them at a model the key allows. If you set ANTHROPIC_API_KEY for the Claude Code tutorial, remove that line from your shell profile, such as ~/.bashrc, and run unset ANTHROPIC_API_KEY. If Claude Code shows a startup warning that names two credential sources, run /logout to keep only the gateway key.

Start claude and run /status: the Anthropic base URL line shows the gateway. Requests now use the Anthropic API key in LiteLLM instead of a claude.ai login, so subscription limits do not apply, and Remote Control is off while ANTHROPIC_BASE_URL points at the gateway.

Step 9: Verify port 4000 is closed to the internet

Check the listening address on the server as your sudo user:

sudo ss -tlnp | grep ':4000'

The Local Address column shows 127.0.0.1:4000. From your own computer, test the public address:

curl --max-time 5 http://YOUR_SERVER_IP:4000/health/liveliness

The request times out. A reply of "I'm alive!" means the port is public. A request to https://litellm.example.com/v1/models without a key returns an error that contains No api key passed in.

Back up LiteLLM

PostgreSQL holds the virtual keys, spend logs and any models added in the Admin UI. pg_dump writes a dump while LiteLLM keeps running:

mkdir -p ~/litellm-backups
cd ~/litellm
STAMP=$(date +%F-%H%M)
docker compose exec -T db pg_dump -U litellm -d litellm -Fc > ~/litellm-backups/litellm-db-$STAMP.dump
cp config.yaml compose.yaml ~/litellm-backups/
chmod -R go-rwx ~/litellm-backups

Keep .env out of this folder. A restore needs the same LITELLM_SALT_KEY, which you saved in your password manager in Step 1. Copy the folder off the server with rsync or scp. To restore:

docker compose stop litellm
docker compose exec -T db pg_restore -U litellm -d litellm --clean --if-exists < ~/litellm-backups/litellm-db-YOUR_TIMESTAMP.dump
docker compose start litellm

Never run docker compose down -v here: -v deletes the database volume. For server-level copies, see VPS backups.

Update LiteLLM

LiteLLM ships a stable minor release most weeks and uses patch releases for hotfixes; -rc and -dev tags are pre-releases. For each update:

1. Read every release note between your version and the target on LiteLLM's GitHub releases page. Look for breaking changes, such as the feat(proxy)!: entry in v1.104.0 that rejects a weak master key.
2. Run the backup. The new version migrates the database when it starts, so the dump is your way back.
3. LiteLLM signs its images with cosign. With cosign installed, verify the new tag first:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:YOUR_NEW_VERSION

4. Change the tag, pull and restart:

cd ~/litellm
sed -i 's/^LITELLM_VERSION=.*/LITELLM_VERSION=YOUR_NEW_VERSION/' .env
docker compose pull
docker compose up -d

5. Check /health/readiness and repeat the Step 6 request.
6. Leave LITELLM_SALT_KEY unchanged.

postgres:16 pulls PostgreSQL 16 minor releases; a new major version needs a dump and restore.

Troubleshoot common LiteLLM errors

FAQ

What is LiteLLM proxy?

LiteLLM proxy is a self-hosted gateway that gives apps one OpenAI-compatible API for 100+ model providers. It adds a virtual key per app, spend tracking, budgets, rate limits and fallbacks, and stores keys and spend in PostgreSQL.

Is LiteLLM proxy free?

Yes. LiteLLM's code is open source under the MIT license, apart from its enterprise directory, and this tutorial runs the gateway image without a license key. Enterprise features need a LITELLM_LICENSE key. Model providers bill API usage to your own provider accounts.

How do I check the LiteLLM version?

Run docker compose exec litellm litellm --version in the project folder. It prints LiteLLM: Current Version followed by the number, such as 1.104.0. The LITELLM_VERSION line in .env shows the image tag that Compose runs.

Which LiteLLM Docker image should I use?

Use ghcr.io/berriai/litellm with a pinned version tag such as v1.104.0. It includes the Prisma tools that PostgreSQL setups need. litellm-database is a legacy alias of the same image, and LiteLLM's docs advise against latest and other moving tags.

How do I use LiteLLM proxy with Claude Code?

Set ANTHROPIC_BASE_URL to your LiteLLM address, ANTHROPIC_AUTH_TOKEN to a virtual key and ANTHROPIC_MODEL to a Claude model_name from config.yaml. LiteLLM serves Anthropic's format at /v1/messages. Anthropic supports only Claude models through a gateway.

Can LiteLLM proxy route requests to Ollama?

Yes. Add a model with the ollama_chat/ prefix and an api_base the LiteLLM container can reach, such as http://ollama:11434 on a shared Docker network. A CPU server stays busy while a local model generates, so send local requests on demand.

What is the difference between LiteLLM and vLLM?

vLLM is a library and server for LLM inference: it loads model weights and generates tokens on your own hardware. LiteLLM runs no model. It forwards requests to providers, including a vLLM server, and adds keys, budgets and fallbacks.

Next steps

• Pick a local model for the optional Ollama route in Best Ollama Models for CPU Servers by RAM Size.
• How to Build an n8n AI Agent with an API Model or Ollama
• Set spend limits and rate limits per virtual key with LiteLLM's budgets and rate limits guide.
• Review LiteLLM's production best practices before adding workers.

Arct Cloud General Purpose plans run on AMD EPYC Milan 7003 Series processors up to 3.7 GHz with NVMe storage. Compare plans.