Running Kokoro TTS on NVIDIA DGX Spark (ARM64/GB10)

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
,
1

I successfully got Kokoro TTS running on the DGX Spark with full GPU acceleration. Here’s a step-by-step guide for anyone else looking to deploy this. Also Japanese voices ready (which was the biggest painpoint).

What is Kokoro TTS?

Kokoro is a fast, multilingual text-to-speech model (82M parameters) that supports multiple languages and 67 voice packs. Critically for ARM64 users: it includes working Japanese TTS with proper phonemization - something that’s notoriously difficult to get working on ARM architecture due to MeCab/Unidic dependencies.

Why This Guide?

Most TTS solutions struggle with:

  • Japanese language support on ARM64 - MeCab and Unidic are tricky to compile/configure

  • GPU acceleration on GB10 - CUDA 12.1 compatibility warnings

  • Docker build failures - The original Dockerfile has ordering issues

This guide solves all three.

Prerequisites

  • NVIDIA DGX Spark with DGX OS

  • Docker with GPU support enabled

  • ~18GB disk space for the image

Step-by-Step Setup

1. Clone the Repository

bash

cd ~
git clone https://github.com/remsky/Kokoro-FastAPI.git
cd Kokoro-FastAPI

2. Update the Dockerfile for ARM64

The original GPU Dockerfile needs modification to work properly on ARM64. The key issue is that project files need to be copied BEFORE running uv sync.

Edit the Dockerfile:

bash

cd ~/Kokoro-FastAPI/docker/gpu
nano Dockerfile

Replace with this corrected version:

dockerfile

# Base image
FROM nvidia/cuda:12.2.0-base-ubuntu22.04

# 1. Install system dependencies
RUN apt-get update -y &&  \
    apt-get install -y \
    python3.10 python3.10-dev python3-venv \
    espeak-ng espeak-ng-data git libsndfile1 curl ffmpeg g++ build-essential cmake && \
    apt-get clean && rm -rf /var/lib/apt/lists/*

# 2. Setup folders and user
RUN mkdir -p /app && \
    useradd -m -u 1001 appuser && \
    chown -R appuser:appuser /app

# 3. Install uv globally
RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \
    mv /root/.local/bin/uv /usr/local/bin/

WORKDIR /app
USER appuser

# 4. Copy ALL project files first (needed for uv sync to build the package)
COPY --chown=appuser:appuser pyproject.toml ./pyproject.toml
COPY --chown=appuser:appuser api ./api
COPY --chown=appuser:appuser web ./web
COPY --chown=appuser:appuser docker/scripts/ ./

# 5. Install CUDA-enabled PyTorch and dependencies
RUN uv venv --python 3.10 && \
    uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 && \
    uv sync --extra gpu --no-cache && \
    uv pip install unidic-lite && \
    mkdir -p /app/.venv/lib/python3.10/site-packages/unidic && \
    ln -s /app/.venv/lib/python3.10/site-packages/unidic_lite/dicdir /app/.venv/lib/python3.10/site-packages/unidic/dicdir

RUN chmod +x ./entrypoint.sh

# 6. Set environment variables
ENV PATH="/app/.venv/bin:$PATH" \
    PYTHONUNBUFFERED=1 \
    PYTHONPATH=/app:/app/api \
    USE_GPU=true \
    DEVICE="cuda" \
    MECABRC=/app/.venv/lib/python3.10/site-packages/unidic/dicdir/mecabrc \
    PHONEMIZER_ESPEAK_PATH=/usr/bin \
    PHONEMIZER_ESPEAK_DATA=/usr/share/espeak-ng-data

# 7. Download model
RUN mkdir -p api/src/models/v1_0 && \
    python ./download_model.py --output api/src/models/v1_0

CMD ["./entrypoint.sh"]

Key change from original: Step 4 now copies all project files BEFORE running uv sync (which needs them to build the package).

3. Build the Docker Image

bash

cd ~/Kokoro-FastAPI 
docker build --platform linux/arm64 -t kokoro-tts-arm64 -f docker/gpu/Dockerfile .

Build time: ~5-10 minutes depending on network speed.

4. Run the Container

The service internally uses port 8880 (not 8000). Clean up any existing container and start fresh:

bash

docker stop kokoro-tts || true && docker rm kokoro-tts || true

docker run -d \
  --name kokoro-tts \
  --gpus all \
  -p 7860:8880 \
  -e USE_GPU=true \
  -e DEVICE=cuda \
  kokoro-tts-arm64

5. Verify It’s Running

Check the logs:

bash

docker logs -f kokoro-tts

You should see:

Model warmed up on cuda: kokoro_v1
CUDA: True
67 voice packs loaded
Beta Web Player: http://0.0.0.0:8880/web/

Accessing the Service

  • Web Interface: http://localhost:7860/web/

  • API Documentation: http://localhost:7860/docs

  • API Endpoint: http://localhost:7860

Replace localhost with your DGX Spark’s IP address for remote access.

Notes & Observations

CUDA Compatibility Warning

You’ll see this warning in the logs:

Found GPU0 NVIDIA GB10 which is of cuda capability 12.1.
Minimum and Maximum cuda capability supported by this version of PyTorch is (8.0) - (12.0)

This is harmless - PyTorch gracefully handles the GB10’s SM 12.1 architecture despite the warning. The model loads and runs perfectly.

Performance

  • Model warmup: ~4 seconds

  • Running on CUDA with full GPU acceleration

  • 67 voice packs available out of the box

  • Supports multiple languages including English, Japanese, German, and more

Managing the Container

bash

# View logs
docker logs -f kokoro-tts

# Stop and remove
docker stop kokoro-tts && docker rm kokoro-tts

# Restart
docker restart kokoro-tts

# Rebuild after Dockerfile changes
cd ~/Kokoro-FastAPI
docker build --platform linux/arm64 -t kokoro-tts-arm64 -f docker/gpu/Dockerfile .

Troubleshooting

Build fails with “api/src does not exist or is not a directory”:

  • The original Dockerfile copies files in the wrong order

  • Use the corrected Dockerfile above which copies project files BEFORE uv sync

Port 7860 already in use:

  • Change the external port mapping: -p YOUR_PORT:8880

GPU not detected:

  • Verify Docker GPU support: docker run --rm --gpus all nvidia/cuda:12.2.0-base-ubuntu22.04 nvidia-smi

Container exits immediately:

  • Check logs: docker logs kokoro-tts

  • Ensure all required files are present in the repo

Use Cases

This setup is great for:

  • Local voice assistant pipelines

  • Multilingual TTS applications

  • Japanese language learning tools

  • Real-time voice synthesis

  • Alternative to cloud TTS APIs

  • Integration with robotics projects (teleoperation feedback, etc.)

Complete Quick Reference

bash

# Clone and setup
cd ~
git clone https://github.com/remsky/Kokoro-FastAPI.git
cd Kokoro-FastAPI/docker/gpu
nano Dockerfile  # Apply the corrected Dockerfile above

# Build
cd ~/Kokoro-FastAPI
docker build --platform linux/arm64 -t kokoro-tts-arm64 -f docker/gpu/Dockerfile .

# Run
docker stop kokoro-tts || true && docker rm kokoro-tts || true
docker run -d \
  --name kokoro-tts \
  --gpus all \
  -p 7860:8880 \
  -e USE_GPU=true \
  -e DEVICE=cuda \
  kokoro-tts-arm64

# Verify
docker logs -f kokoro-tts

Credits

Hope this helps other DGX Spark users! The key fix is ensuring project files are copied before the build step tries to install the package.

4 Likes