DGX Spark RAG on Docker

[oussama.achhab]

Hello,

I am sharing my setup for deploying a RAG stack on the DGX Spark.

The stack is composed of Anything LLM as orchestrator, Ollama as LLM provider (with the possibility of configuring other local or cloud providers), and Qdrant DB as vector DB. The web search ability is achieved using SearxNG meta engine.

The deployment is achieved using docker compose.

GitHub - amasu/dgx-spark-rag: Retrieval Augmented Generation containerized for Nvidia DGX Spark · GitHub

Cheers !