Vllm docker-compose - on DGX Spark from first time user looking for suggestions and question about RAM utilization

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

First time user and jumping full in on learning all thing AI and llms.

Below is my first crack of getting a model running on the DGX Spark and then exposed via a nginx proxy to an openwebui instance I have running.

Everything is working and I see no glaring errors in the container logs. However, I’m looking for suggestions on what I can improve or should tune regarding this config.

Furthermore for a model of this size is it expected to take over 100GB of RAM? I was under impression it should use less based on the 20b parameter size, or do I need to put some additional config in place to ensure it does not consume everything so that I can also test with a coder model and a model for RAG in parallel?

services:
  vllm:
    image: nvcr.io/nvidia/vllm:25.09-py3
    container_name: vllm
    command:
      - vllm
      - serve
      - openai/gpt-oss-20b
    networks:
      llm:
      proxy:
    # ports:
    #   - "8000:8000"
    volumes:
      - /home/chris/models/vllm_cache_huggingface:/root/.cache/huggingface # Cache Hugging Face models
      - /home/chris/models/manual_download:/models # mount your home directory models folder
      - /home/chris/models/manual_download/openai_gpt-oss-encodings_fix/cl100k_base.tiktoken:/etc/encodings/cl100k_base.tiktoken # openai fix
      - /home/chris/models/manual_download/openai_gpt-oss-encodings_fix/o200k_base.tiktoken:/etc/encodings/o200k_base.tiktoken # openai fix
    environment:
      # openai gpt-oss-20b/120b fix 10/25/25
      - TIKTOKEN_ENCODINGS_BASE=/etc/encodings
      # Explicitly tell Hugging Face to use default directory as of 10/25/25.
      - HF_HOME=/root/.cache/huggingface
      # - HUGGING_FACE_HUB_TOKEN=${HF_TOKEN} # Optional: if you need to download private models
      - VLLM_API_KEY=${API_KEY} # Optional: for API key authentication
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: "all"
              capabilities: ["gpu"]
    ulimits:
      memlock: -1
      stack: 67108864
    ipc: host
    restart: unless-stopped
    tty: true

networks:
  llm:
    name: llm
  proxy:
    external: true
    name: proxy
2

--gpu-memory-utilization

Default is 0.9 (90%)

see Engine Arguments - vLLM

without limiting it uses 90% for the loading the weights (gpt-oss-20b has 16 GB) and the rest for the cache.

You can see the exakt usage in the log after loading. And you can run more than one instance for different models. You only need another port for each instance.

3

Thanks I assume I should set it to something like .3-.4 (30-40%) so that i can leverage the rest for others? My logic sound?

4

Yessir. gpt-oss-20b does not need much.

5

VLLM will try to allocate all your available memory by default up to utilization setting, regardless of the model size to support maximum concurrency. To limit the memory footprint, you can set concurrency limit by providing --max-num-seqs 10 – this one will limit concurrency to 10 concurrent requests. If you are not planning to run multiple requests in parallel, set --max-num-seqs 1

6

Hello, i faced tiktoken encoding error too, after changed
from: image: nvcr.io/nvidia/vllm:25.09-py3
to: nvcr.io/nvidia/vllm:25.11-py3 problem has been solved.

i am running gpt-oss 20b too :)