Code assist and rag (instruct) in single node

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
1

Hi everyone,

I’m evaluating DGX Spark / GB10 to build a fully local developer assistant that combines:

  • Code assistance (code completion, refactoring, code understanding)

  • RAG (instruct) over internal corporate website

Questions:

  1. Is this combined code assist + RAG architecture feasible and efficient on GB10?

  2. Would you recommend running one single LLM or multiple specialized LLMs with routing?

  3. Which LLMs would you suggest today for:

    • coding-heavy tasks

    • instruct + RAG

  4. Any recommended inference stack (TensorRT-LLM, vLLM, Triton) for this use case?

Any experience, best practices, or references on DGX Spark would be very helpful.

Thanks in advance.

4

After several tests with vLLM and NIM, I had to go back to Ollama.

With Ollama, I’m able to run more usable models:

  • gpt-oss:120b-64k – inference and planner for Opencode

  • qwen3-coder:30b-64k – Opencode builder

  • nomic-embed-text:latest – embeddings (on Grace cpu)

  • qwen2.5-coder:1.5b – autocomplete

  • (nginx – gateway)

This way, I can achieve practical usage of a single Spark (Dell Pro Max GB10).

Giacomo

5

Thank for the information. I will give that setup a try with llama.cpp