4 Node Cluster with CRS504 and 100G Connection: Results

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

Just acquired 2 Acer Veriton and added them to my 2 GX10s. The Veritons had 4TB PCIE Gen 5 SSDs in them for only 300 over the GX10. I hate myself for not discovering this earlier, and now I’m stuck with the 2 shitty 2242 Gen 4 1TB drives in the GX10.

I run the cluster for myself, mainly for game dev stuff. Basically always at concurrency 1 (sometimes 2–3), so single-stream speed is all I care about. Before committing to another 2 nodes, I forced my 2-node direct link down to one 100G rail to see what I’d lose, then retested through the actual switch. I Tested Step-3.7-Flash and DeepSeek-V4-Flash, TP=2, 256k context. No change in decode, 5-10% loss in PP.

Got the 4-node cluster set up today and ran a benchmark with DSV4 Flash at TP=4 and 512k ctx. Here are the results:

DeepSeek-V4-Flash on aidendle B12X image — llama-benchy 0.3.7, c=1, pp2048, fp8 KV, MTP k=2

Depth PP TP=2 PP TP=4 TG TP=2 TG TP=4
0 1612 2236 35.8 53.6
8k 1894 2436 32.5 52.0
32k 2025 2452 29.9 52.3
64k 1947 2380 36.8 52.6

I don’t have a control, so I can’t speak to performance degradation vs a full 200G link node. Can someone post their results with DSV4 Flash and a 4-node cluster with CRS804 or 812?

2

I honestly don’t think there would be a huge difference, I’ve not tested either of those llm, but I can say that the traffic on the switch rarely does go over 100gig anyhow, so your unlikely missing out on much, I was considering testing that same switch myself, just because it would be quieter than the CRS804.

3

Yeah i measured the traffic and it was about 13g/s
I also swapped a pair of noctua 40mm in and it was significantly quieter, but the switch still forces them to run at ~5000rpm so there’s still a slight whine.

4

Which SSDs do you have on the Veriton nodes?

5

The Samsung ones, I forgot the name because it was gibberish and super long, but it was PCIE Gen 5. I got them today from Best Buy.

6

I was able to get Minimax M3 AWQ running with tp=4 and Eagle drafter. This is with 262k ctx length and mns=4. c=1. bf16 kv cache.

Context Decode Prefill
0 35.36 t/s 2,210.52 t/s
8k 32.75 t/s 1,820.79 t/s
32k 27.73 t/s 1,717.32 t/s
64k 28.44 t/s 1,684.39 t/s
7

Thank you for the useful information.

Would it be possible for you to test decoding performance under high concurrency as well?

I’m curious what kind of performance you would get with around 64 and 128 concurrent requests, using a workload of roughly 10k tokens for prefill and 1,000 tokens for decoding.

9

Necroing this thread to add that I got a Veriton GN100 right at the end of the June sale period (from Target but shipped direct from Acer) and it didn’t say what drive was inside but it is indeed the Samsung Gen5 - actually the exact same part as in my OEM DGX Spark.

Question for @CosmicRaisins - do your Veritons turn on automatically with power or not? I’m not sure if this is a quirk of the GN100 or something I can fix in the BIOS, but mine defaults to OFF until the button is pressed and I’d like to set it up to match the DGX Spark behavior.

BIOS is not easy for me to access as I’ve set these up headless, but I’ll find a way if it’s in there.

10

No they do not. That would’ve been really nice. The power button is hard to reach and harder to press…

11

I have same machine. There is a setting in the BIOS to change the power setting to what you want: when power comes it will turn on. I forget the exact name. I try to get WakeOnLan but not supported. This is the best I can get by using a smart plug to turn it on remotely.

12

I use a 100gb cable on my 2x sparks and I seem to be getting pretty much the same speed as I see everyone else with a 200gb gets. Nice thing is the cable was only $25 on amazon and works with 0 issues.

13

As 11pm noted, I can confirm this is indeed a BIOS setting.

It’s in Advanced → Power On Behavior and the GN100 defaults to requiring the button. Set it instead to Auto Boot and save!

14

When does the 200G cable really matter? Is it better to just buy the 100G cable for cheaper?

Also is CRS504 the best bet? CRS804 out of stock, only CRS812 available.

15

I’m not sure because I even do DDP training and I get about a 1.7x or 2x increase in throughput vs single node training. 10Gtek 100G QSFP28 DAC Cable -… Amazon.com: 10Gtek 100G QSFP28 DAC Cable - 100GBASE-CR4 QSFP28 to QSFP28 Passive Direct Attach Copper Twinax Cable for Cisco QSFP-100G-CU0.5M, Ubiquiti, Arista, Supermicro and More, 0.5-Meter(1.6ft) : Electronics is the cable I bought. Works flawlessly.