Skip to content

Kokoro TTS

In the ~/repositories/Apps folder, created the corresponding Kokoro folder, then inside of it created the following docker-compose.yml file:

name: kokoro
services:
    kokoro-fastapi-gpu:
        expose:
            - "8880"
        image: ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.1
        restart: always
        deploy:
            resources:
                reservations:
                    devices:
                        - driver: nvidia
                          count: all
                          capabilities:
                              - gpu
        networks:
          - litellm_network

networks:
  litellm_network:
    external: true
    name: litellm-stack_default

The Kokoro container is attached to the same Docker network as LiteLLM, this allows LiteLLM to reach Kokoro via container hostname: http://kokoro-kokoro-fastapi-gpu-1:8880.

Then in LiteLLM's config file Kokoro is added as an OpenAI compatible model:

model_list:
# =====================
  # Kokoro-TTS
  # =====================
  - model_name: kokoro-tts
    litellm_params:
      model: openai/kokoro
      api_base: http://kokoro-kokoro-fastapi-gpu-1:8880/v1  # container name as hostname
      openai_compatible: true
      api_key: none

Now in OWUI > Admin Panel > Settings > Audio, configure OWUI to use the Qween TTS model exposed via LiteLLM for audio generation requests:

alt text

  1. Select OpenAI from the dropdown menu
  2. Insert LiteLLM's API base URL: https://tracking.mlmp.ti.bfh.ch/v1
  3. Insert API Key
  4. Insert TTS voice fable
  5. Insert model's name (same as in model_name in model_list)
  6. Save

Finally start Kokoro container and restart OWUI and LiteLLM.