About a month ago, I heard that NVidia would be releasing an optimized model, but haven’t seen anything official. If someone knows of one, I would be appreciative if you can point me to it.
(Looking for a dual-spark solution)
Thanks,
John
There is already a NVFP4 quant they released: nvidia/DeepSeek-V4-Flash-NVFP4 · Hugging Face
Most folks are currently running the DSpark variant – it’s a mix of FP4 + FP8, so it’s already pretty optimized for platforms like the DGX Spark: deepseek-ai/DeepSeek-V4-Flash-DSpark · Hugging Face
I have tried but a lot of things are misbehaving.
went from deepseek-v4-flash → deepseek-v4-flash-dspark
eugrwith custom recipeaidendle94
More repo’s:
- GitHub - MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark · GitHub
- GitHub - tonyd2wild/DeepSeek-v4-Flash-DSpark-1M-NVFP4-KV-2x-DGX-Spark: DeepSeek V4 Flash DSpark 1M NVFP4 KV recipe for 2x DGX Spark · GitHub
PM me if assistance is required.
I’m using the Aiden 3.7 as my daily driver and works flawlessly. Don’t use spark run. Use the container here and build it directly from that.
Make sure you use the exact pinned versions too - when my agent helped me to put it together it tried to get the latest version from main instead of the version the container uses. Follow the container exactly.
I get up to 50+ t/s each for up to 3 concurrent agents at 500k context with this and could probably push to 1m - it is outstanding.
One or two sparks?
All the working recipes here use a two node cluster. There is a recipe out there that runs on a single spark, but I wouldn’t expect it to work well.
For 1 spark the only option is DwarfStar iq2xxs. It works surprisingly well for 2-bit quant, but not fully optimized for CUDA - initially built by famous Antirez for Metal (Apple silicon). It works on Spark but slow. On Apple it actually works well - 30 t/s, 500k context on 128GB ram M5 max - scored 86/100 on 168 point tool eval bench, insane for 2 bits! If you have to, you can but brace yourself for slow prefill and tg (tg is already better as MTP drafter comes from Antirez already).
80tps tg per stream or 80tps tg aggregate per 3 streams?
That IS insane for a 2 bit!
Slows down under concurrency, but still more than fast enough:
| Depth | Conc | Prefill (pp t/s) | Gen (tg t/s) | TTFT (ms) | Total (ms) |
|---|---|---|---|---|---|
| d0 | c2 | 1,744 | 60.5 | 1,723 | 4,970 |
| d0 | c4 | 1,746 | 67.9 | 3,239 | 8,389 |
| d4096 | c2 | 2,107 | 48.7 | 4,380 | 7,878 |
| d4096 | c4 | 1,927 | 38.7 | 7,659 | 14,481 |
| d8192 | c2 | 2,125 | 35.0 | 6,992 | 11,019 |
| d8192 | c4 | 2,156 | 29.6 | 11,744 | 19,293 |
that is nice.. i barely made it to 51 :( … oh wait.. you have c2 not c1, I get 59.4 peak at c1
Been ootl for a few weeks during crunch. How does it stack up for long agentic work loads and deep context erosion?
Currently using Qwen36 a3b nvfp4 at 120ctx but it gets nuts after 7-8 compactions.
Stu, I’m working with Claude to implement your suggestion on my 2-spark setup. It’s got a question for you:
The 3.7 image bakes --speculative-config method:dspark + FLASHINFER_MLA_SPARSE_DSV4, tuned for the DSpark checkpoint.
Which model does Stu serve with it (what should MODEL_PATH point at)?
-
The DSpark model
deepseek-ai/DeepSeek-V4-Flash-DSpark (the one we just deleted). Matches the baked dspark spec + sparse-DSV4 backend, but means re-downloading ~167 GB before launch. -
Our cached DeepSeek-V4-Flash
The plain fp8 model already on both nodes (no download). Risk: the baked dspark spec / sparse-DSV4 backend may reject it at startup. -
Ask Stu / not sure
You’ll confirm with Stu which checkpoint + any env overrides (max-model-len, MAX_NUM_SEQS, SPEC_TOKENS) so I follow his exact working setup rather than guess.
I really appreciate your help!
Thanks,
John
use the deepseek-ai/DeepSeek-V4-Flash-DSpark model with Aiden 3.7 .. working well enough over here
How well it is ? Any actual tool-call usages ? How is multi turn high context usages (150k+) ?
Does it work on single Spark?
All work fine. Check the last 10 posts in the thread
For single spark the only option is dwarf star 4