DeepSeek-V4-Flash (official FP8) running across 2x DGX Spark — TP=2, MTP, 200K ctx, recipe + numbers

Some initial llama-benchy results using recipe from PR Add DeepSeek V4 Flash recipe by arthur-drozdov · Pull Request #219 · eugr/spark-vllm-docker · GitHub, concurrent runs throttled pretty hard with context over 65535 so wasn’t able to test up to 100k context (however in practice it seems to only be able to handle a single concurrent request for large context).

Decode throughput, tg128, total tok/s:

depth c1 c2 c5 c10
0 37.32 54.16 43.64 43.09
4096 27.19 40.00 25.90 27.23
8192 35.21 30.82 26.72 20.52
16384 37.54 27.55 16.63 13.52
32768 33.51 19.01 12.64 2.25
65535 29.50 1.11 0.74 -

Prompt throughput, pp2048, total tok/s:

depth c1 c2 c5 c10
0 1097.6 977.9 1044.3 975.4
4096 738.9 598.3 571.3 552.6
8192 613.4 548.1 460.7 411.1
16384 471.1 467.9 307.7 280.2
32768 318.4 322.6 222.6 39.4
65535 176.0 17.2 11.9 -