Hello,
Has anyone successfully run Llama 405B nvfp4 on two sparks across CX7? I’ve tried a number of ways and failed with memory errors etc. I know there is a custom NIM container that is specifically made for SM 121 silicon. Any help or pointers greatly appreciated.
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| How do I download Llama 405b for Spark | 0 | 10 | April 20, 2026 | |
| Compiling llama.cpp | 14 | 1534 | February 7, 2026 | |
| NIM LLM Containers Fail on DGX Spark (GB10): Triton/vLLM Crash on sm_121 and NGC Permission Errors | 4 | 433 | February 6, 2026 | |
| Missing official native ARM64 NIM images for essential AI models | 6 | 515 | March 20, 2026 | |
| Marlin Fix: NVFP4 Actually Works on SM121 (DGX Spark) | 15 | 1565 | April 12, 2026 | |
| Two-Spark cluster with vLLM using tensor-parallel-size 2 causes one node to drop while the other's GPU goes 100% forever | 36 | 1235 | February 13, 2026 | |
| Help: Running NVFP4 model on 2x DGX Spark with vLLM + Ray (multi-node) | 18 | 2312 | December 25, 2025 | |
| Running Llama-3.1-8B-FP4 get triton error. Value 'sm_121a' is not defined for option 'gpu-name' | 2 | 684 | October 24, 2025 | |
| Single node and Dual node llama.cpp build flag | 5 | 152 | March 11, 2026 | |
| GLM-4.7-Flash-NVFP4 was just released, but for Transformers 5.0 + vLLM 0.14...? | 90 | 4234 | February 27, 2026 |