KRAFTON already provides a DGX Spark image for Raon-1-mini-preview NVFP4. I developed a separate image using a newer vLLM version and B12X kernels to explore overall performance improvements on a single DGX Spark.
For this image, I added a B12X_MLA adapter that connects Raon’s MLA decode and speculative verification path to existing B12X dense MLA kernels. The target model uses an FP8 KV cache, while MLA prefill uses FlashAttention.
I also quantized the DFlash2 draft model and its output head (lm_head) to NVFP4 W4A4 using NVIDIA Model Optimizer. The draft output head is calibrated using inputs from the quantized draft body.
Image configuration
The image uses:
- vLLM 0.32.0
- B12X Linear and MoE
- Custom
B12X_MLAadapter for target MLA decode and speculative verification - FP8 E4M3 target KV cache
- NVFP4 W4A4 DFlash2 draft, including its output head
- FLASH_ATTN draft attention with BF16 draft KV cache
- 4 speculative tokens
- Probabilistic drafting and block rejection sampling
- Async scheduling
- FULL decode CUDA graphs
- One DGX Spark, TP1/DCP1
Models:
Comparison with KRAFTON’s original DGX Spark image
The following llama-benchy results compare KRAFTON’s original image with my B12X image on a single DGX Spark.
The images use different vLLM implementations and serving configurations, so this is an image-level comparison rather than a measurement of one kernel or optimization in isolation.
Single-request tests
| Test | Original image | B12X image | Change |
|---|---|---|---|
| pp2048, tg128 run | 3,957.45 ± 37.69 tok/s | 6,614.65 ± 188.99 tok/s | +67.1% |
| tg128 | 57.54 ± 2.71 tok/s | 71.95 ± 3.21 tok/s | +25.0% |
| pp2048, tg1024 run | 4,737.45 ± 324.32 tok/s | 6,811.57 ± 57.61 tok/s | +43.8% |
| tg1024 | 63.10 ± 1.17 tok/s | 71.25 ± 4.64 tok/s | +12.9% |
| pp2048 @ d1024, tg128 run | 5,392.41 ± 136.57 tok/s | 6,269.53 ± 197.09 tok/s | +16.3% |
| tg128 @ d1024 | 51.31 ± 6.72 tok/s | 70.25 ± 6.97 tok/s | +36.9% |
| pp2048 @ d1024, tg1024 run | 4,657.61 ± 1,367.24 tok/s | 6,146.96 ± 170.35 tok/s | +32.0% |
| tg1024 @ d1024 | 58.22 ± 6.58 tok/s | 67.23 ± 2.90 tok/s | +15.5% |
| pp2048 @ d4096, tg128 run | 5,900.02 ± 284.13 tok/s | 6,541.13 ± 103.36 tok/s | +10.9% |
| tg128 @ d4096 | 51.70 ± 4.05 tok/s | 74.07 ± 6.89 tok/s | +43.3% |
| pp2048 @ d4096, tg1024 run | 6,165.15 ± 247.25 tok/s | 6,506.45 ± 100.87 tok/s | +5.5% |
| tg1024 @ d4096 | 54.17 ± 1.63 tok/s | 68.02 ± 2.39 tok/s | +25.6% |
For example, tg128 at depth 4096 increased from 51.70 to 74.07 tok/s, approximately 43.3% faster.
Concurrent requests
These figures show total output throughput. Concurrency is the number of requests submitted by the benchmark; it does not mean all requests were necessarily running on the GPU simultaneously.
| Test | Original image | B12X image | Change |
|---|---|---|---|
| tg256 @ d1024, c1 | 56.51 ± 2.00 tok/s | 78.64 ± 4.06 tok/s | +39.2% |
| tg256 @ d1024, c10 | 118.89 ± 2.14 tok/s | 179.39 ± 4.41 tok/s | +50.9% |
| tg256 @ d1024, c50 | 146.23 ± 2.03 tok/s | 254.79 ± 1.56 tok/s | +74.2% |
At 50 concurrent requests, total output throughput increased from 146.23 to 254.79 tok/s.
Long context: 100K depth
| Metric | Original image | B12X image | Change |
|---|---|---|---|
| pp2048 @ d100000 | 4,100.50 ± 7.39 tok/s | 3,931.13 ± 181.86 tok/s | −4.1% |
| tg512 @ d100000 | 33.23 ± 1.51 tok/s | 54.44 ± 5.06 tok/s | +63.8% |
| Time to first response | 24,893.14 ± 44.89 ms | 26,021.91 ± 1,205.14 ms | +4.5% |
The 100K-depth decode result improved substantially: generation throughput increased from 33.23 to 54.44 tok/s, approximately 63.8% faster, while the mean prefill throughput was approximately 4.1% lower.
With MLA, longer contexts require each decode step to read and process more cached data, which can reduce generation speed. In this test, the B12X image substantially improved decode throughput at 100K depth while keeping prefill throughput close to the original image.
Long context: 260K depth
I also tested the B12X image at 260K depth:
| Metric | B12X image |
|---|---|
| pp2048 @ d260000 | 2,463.53 ± 2.20 tok/s |
| tg512 @ d260000 | 39.25 ± 2.41 tok/s |
| Time to first response | 106,376.73 ± 95.11 ms |
There is no matching original-image result in this test, so these figures are shown without a direct comparison.
Some earlier long-context runs ended with out-of-memory errors or unexpected engine termination. I am continuing to check memory stability under long-context workloads.
Tool-eval-bench
I also ran tool-eval-bench v3.0.1.dev7+g887b53757 on both images.
| Metric | Original image | B12X image |
|---|---|---|
| Score | 81/100 | 84/100 |
| Points | 168/208 | 174/208 |
| Passed | 78 | 82 |
| Completion time | 887.4s | 776.7s |
The B12X run scored 3 points higher and completed 110.7 seconds sooner. The runs used different total token counts, so completion time should be read together with the workload.
I would appreciate any feedback or ideas for further improving long-context performance and stability on DGX Spark.