They only sell through distributors, but there is no inventory of the new model yet… and I only have 4x sparks at this point, so I’m just going to accept it since I’m already using them together–don’t want to do ship back, wait for inventory, shipping, exercise. …and if I’m willing to spend another $16k on DGX sparks + cabling, honestly I’ll cross the switch issue when I come to it. Not convinced what would be the most cost effective.
Plus… In practical use so far, the only way I’ve ever pushed to 200Gbps is during benchmarking network speed. I guess I’ll see when I do some larger training if I might need the bandwidth, but so far, I imagine I wouldn’t notice much change if I were to downgrade to 100Gbps per spark and then I could put 16x sparks on the CRS 812 DDQ!
Since I’ve been asked a few times, the cluster consists of:
8x DGX Spark (6x founder, 2x hp zgx nano)
2x CRS812 DDQ
1x DDQ+DA0001
3x 0.2m HP QSFP112
2x 1m 400G QSFP-DD to 2X200G QSFP56
1x NJAAKK-N911
The DDQ 400gbps cable links the switches, leaving 8 usable 200gbps connects. The only configurations needed to be done on the switches was enabling jumbo frames and turning off auto negotiation for the second DDQ port, so I could split it to 200gbps.
It pretty much just works.
Since that has 400Gbps interconnect between the switches, half your nodes effectively have max bandwidth of 100Gbps each (assuming equal traffic, 4x nodes on switch 1 trying to reach others on switch 2 will divide the 400Gbps switch interconnect) but… you have those 8x SFP56 ports that are doing nothing, right? You could connect all 8x of those SFP56 ports together and make the interconnect between the switches be QSFPDD + 8x SFP56 = 800Gbps meaning that it’s possible to get full bandwidth of 200Gbps per node. Realistically, I assume that 100Gbps is actually enough given what I’ve seen about my spark internode communication via NCCL, but… figured I’d ramble that out.
Have you done any tests to specifically see the effects of the bottleneck of your switch interconnect? Adding a hop adds latency no matter what, but should be relatively small here–but peak bandwidth should definitely show the effect in throughput benchmarks.
BTW, Christopher’s build is now part of my Docker repo too. You need to compile with a special flag, but other than that it will work with launch-cluster script the same way as a normal one.
Can Kimi-K2.5 (served with vLLM) be run on a 6x DGX Spark cluster? I believe that tensor parallelism requires that both the attention heads and hidden dimensions be divisible by tp. Perhaps with pipeline parallelism and tp=1?
Intersting project, I have 2 Asus Asxent GX10, now ordering extra 6 and custom made an all in one mirco rack includes 2 CRS804 DDQ (240W PD3.1 power strip module still sourcing in China). Waiting your x8 updates!
I love how reliable this model is with generating documentation for coding, reviewing my code, or generating reports on my code which I call “visual unit testing.”
Is there a way to cluster 4 Spark only using cables and no switch.
Is a problem of portability for me I can buy 4 of them and i will bring them with me in my Pelican bag but i will bever have space for switch or else at least when i tour.
You can cluster up to 3 Sparks, following a ring topology and without sacrificing interconnect bandwidth. From the fourth onwards you will need a switch. 🤷
I understand your preference for carrying the Sparks with you, but accessing the cluster via remote is far easier to do, unless you want to move from a Pelican case on the small end side, to a significantly larger one.
