Skip to content

Litellm Docker Compose Stack

This Docker Compose stack consists of three containers: litellm, database, prometheus.

I'm going to create a tree folder only for this backup: 

backups:
    litellm:
        litellm
        database
        prometheus

Litellm

From this container we only want to backup the config.yml, to see if it's a volume or a bind see inside the docker-compose.yaml file: 

volumes:
      - type: bind
        source: /home/sysadmin/repositories/Apps/litellm-stack/config.yaml
        target: /app/config.yaml
        bind: {}

Postgres

This stack uses PG as a database, for database backups we dump the database in this case to be extra cautious I will also backup the volume: 

volumes:
      - type: volume
        source: postgres_data
        target: /var/lib/postgresql/data
        volume: {}

Prometheus

Prometheus has a config.yml file and a named volume where it stores TSDB data, we will backup both: 

volumes:
      - type: volume
        source: prometheus_data
        target: /prometheus
        volume: {}
      - type: bind
        source: /home/sysadmin/repositories/Apps/litellm-stack/prometheus.yml
        target: /etc/prometheus/prometheus.yml
        bind: {}

Backup

Create the backup directory layout in the host: 

mkdir -p /home/sysadmin/backups/litellm/{postgres,prometheus,litellm}

For the Litellm config file, since is a bind mount we compress it directly in the host. From the docker-compose-yml we know where it is located: 

sudo tar czf /home/sysadmin/backups/litellm/litellm/litellm_config_$(date +%F).tar.gz -C /home/sysadmin/repositories/Apps/litellm-stack/ config.yaml

For the Database backup, we dump the pg database, this would be the primary backup (In order to prevent corrupt data it might be a good idea stop the litellm app temporarily while we create the dump sudo docker stop litellm-litellm-1): 

cd /home/sysadmin/repositories/Apps/litellm-stack/
sudo docker compose exec -T db pg_dump -U llmproxy litellm > /home/sysadmin/backups/litellm/postgres/litellm_pg_dump_$(date +%F).sql

NOTE: the database user and name we get from the docker-compose.yml

It might be a good idea to check the contents of this dump, sometimes it fails because a wrong username or db name.

Now the volume: 

sudo docker run --rm \
-v litellm_postgres_data:/data \
-v /home/sysadmin/backups/litellm/postgres/:/backup \
alpine \
tar czf /backup/litellm_pg_volume_$(date +%F).tar.gz -C /data .

Now for Prometheus we stop the container: 

sudo docker compose stop prometheus 
# OR
sudo docker stop litellm-prometheus-1

Backup Volume: Create a temporary container, mount the volume that prometheus uses, create the .tar.gz file and copy to the host (through the bind mount): 

sudo docker run --rm -v litellm_prometheus_data:/data -v /home/sysadmin/backups/litellm/prometheus:/backup alpine tar czf /backup/litellm_prometheus_$(date +%F).tar.gz -C /data .

Backup config file: Since the config file lives also in the host (because of the bind mount), we simply compress it and locate it in the right place: 

sudo tar czf \
/home/sysadmin/backups/litellm/prometheus/litellm_prometheus_config_$(date +%F).tar.gz \
-C /home/sysadmin/repositories/Apps/litellm-stack/ prometheus.yml

Now you could start again everything with sudo docker compose up or start the specific containers that were stopped.

Restic

To make a backup of all those files we just created we can simply backup the litellm folder inside backups. To make things easier and not create extra folders in this backup we cd to the exact location:

cd /home/sysadmin/backups
sudo restic -r <path/to/restic/repo> backup ./litellm --tag full_litellm_bk

This tag will help us restore the specific backup that we want later on.

Restore

Stop everything: 

sudo docker compose down

Delete volumes: 

sudo docker volume rm litellm_postgres litellm_prometheus_data

To simulate a fresh installation of the stack maybe also delete the contents of the litellm config.yml file.

Restore files from Restic: 

sudo restic -r <path/to/restic/repo> restore --tag full_litellm_bk latest --target <where/ever/>

If we check sudo ls <where/ever/> we will see a litellm folder created.

Then copy the litellm config compressed file to outside the folder and extract it and put the extracted files inside the actual litellm folder app: 

sudo cp <where/ever/>litellm/litellm/litellm_config_2026-01-13.tar.gz /tmp
tar xzf /tmp/litellm_config_2026-01-13.tar.gz  -C /home/sysadmin/repositories/Apps/litellm/

Restore Postgres: Create the respecive volume, start only the database container and go inside the container and execute the dump from inside the container (other way would be copying the dump inside the container and executing again from inside it):

sudo docker volume create litellm_postgres_data
sudo docker compose up -d db
sudo docker compose exec -T db psql -U llmproxy litellm < <where/ever>/litellm/postgres/litellm_pg_dump_2026-01-13.sql

NOTE: We could've also restored the volume, but for databases really the main thing is the dump.

Restore Prometheus: Create the respective volume, run a light weight container to put the volume data in there:

sudo docker volume create litellm_prometheus_data
sudo docker run --rm -v litellm_prometheus_data:/data -v <where/ever/>litellm/prometheus:/backup alpine tar xzf /backup/litellm_prometheus_2026-01-13.tar.gz -C /data

Now also restore the config file for Prometheus, since this is a bind mount we can just extract the file and locate it in the litellm app folder: 

sudo tar xzf <where/ever/>litellm/prometheus/litellm_prometheus_config_2026-01-13.tar.gz -C /home/sysadmin/repositories/Apps/litellm/

Now start everything with sudo docker compose up -d. Check that everything works correctly and that for example the virtual keys still the same ones as before.

ls -l /etc/nginx/sites-enabled/default