Sharing our engineering work on a single NVIDIA DGX Spark (GB10, 128 GB unified memory). We use ds4, a native C/CUDA inference engine, to run DeepSeek V4.1 Flash from an official checkpoint of roughly 510 GB on one device.
The core approach uses vector quantization to split the model into three files:
-
① 113.6 GB general base model (~1.6 bits/weight)
-
② ~40 MB domain sidecar files, switchable on demand for finance, code, law, medical, and science domains
-
③ Post-training file, re-solved nightly
The original FP8 203 GB n-gram memory table stays on SSD, and each token reads only about 6 KB from it.
Measured on a single DGX Spark:
| Metric | Result |
|---|---|
| Prefill (12.5k-token prompt) | 489 tokens/s |
| Decode (normal greedy, short context) | 30.7 tokens/s |
| Decode (speculative sampling, 14k context) | 43.0 tokens/s |
| Context length | 1M tokens |
| KV cache | 0.89 GB |
Distribution agreement with the original model (Restore rate Σmin):
| Domain | Restore rate |
|---|---|
| Finance | 0.745 |
| Code | 0.803 |
| Law | 0.760 |
| Medical | 0.739 |
| Science | 0.753 |
Same top-1 ratio ranges from 72.8% to 82.3%.
All CUDA kernels, quantization formats, and solvers are our own work. The model architecture belongs to DeepSeek. MIT license.
Original README: README.md · wenzhouwu/YoungAi-DeepSeek-V4.1-Flash at main