I’m currently running a 2-node DGX Spark cluster connected via QSFP, running Qwen 122B and 397B at 128k-256k context. While the 122B runs well, inference on the 397B feels noticeably sluggish.
With hardware prices steadily climbing, I’m debating pulling the trigger on an expansion now. My use cases are still taking shape—primarily studying ML internals, exploring bioinformatics, and eventually spinning up agentic workflows—but I want to lock in the hardware before costs become too hard to justify, even if I don’t have a ton of free time to tinker just yet.
I am leaning toward a 3-node cluster because I can stick to simple QSFP direct-attach cables, completely avoiding the networking complexity and cost of a switch/router setup required for a 4-node cluster.
However, I noticed the Spark Arena leaderboard only shows benchmarks for 2-node and 4-node setups, leaving a blind spot for 3 nodes. For those with experience: what sort of real-world performance advantage does a 4-node cluster actually offer over a 3-node setup for models in the ~400B range? Does the performance jump to 4 nodes justify the added cost and complexity of the switch?
there is no such thing as tp 3 in vllm - so you will need 1 2 4 8 or more .. unless you decide to run a small model that can fit on 1 mashine and a bigger one on 2 - there is very little reason for 3 nodes for LLM workloads
Like a gentlemen above said - you cannot use tensor parallel for 3, only pipe parallel - you can run larger models with it but with a speed of single spark. Hardly a good idea.
IMO 2 is a sweet spot for an individual. I am very tempted to buy 2 more - price is climbing every day here too. Insanity. However you must answer honestly to yourself - does extra 9K for 2 sparks plus and router and cables justify added capabilities? What model can you run on 4 that you cannot run on 2 that justifies the expense? 397b - no noticeable jump over 122b and deepseek v4 is arguably much better (1M toks) that 397b and runs on 2 comfortably? Minimax M3? If you critically need it and you sure it does what 122b or DS4F cannot do. I am skeptical about it after using this model in the cloud. We cannot run truly frontier models like DeepSeek v4 Pro or GLM 5 or Kimi K on 4 sparks. The gains seem to become marginal for 2->4 jump.
The capability jump must be of consistent use to be a sensible investment. If you need to fine-tune or LoRA train a model - you will be better off spending 50-100 bucks on Lambda or RunPod and run the task on industrial gear like H200 or GH200, B200. Much faster. Much cheaper for one-off task.
If you just WANT it - its a different matter and no calculations can disprove it :) Do whatever you like to.
Just IMO.
Ps right as it was posted there are interesting developments surfaced that can make 3rd spark suddenly very useful option!
the problem is the that you need to divide the tensor shapes / 3 then - and that will kill the model as you did not train it on that shape - but im happy to eat my words
I liked that CRS804too but did not find it in stock at B&H so i might get the CRS812 one.
I feel the cable to connect to the switch seems more limited option from big online retailers. I am looking at this cable Mikrotik 3M Passive Breakout DAC ｜NADDOD
The Mac Studio 512 GB memory option is not available in stock right now to pair with DGX spark and I don’t want to play the waiting game to find the pricing is not favorable for the new launch if and when. Also hard to know how long the wait time can be for that hardware even.
The only thing longer than the wait for a Mac Ultra 512GB to ship is the wait for it to complete prompt processing on tasks with large contexts (translation: actual work)
I would go 4 for flexibility as you already felt limit on 397b’s speed. Ornith 397b model seems nice fro m what I have heard so far, I got 4 sparks and switch etc at once, not because of FOMO, more like giving me freedom and flexibility to choose models I like to get.
It seems recent open weight models are under 35B (dense or moe) but Qwen seems abit lost their passion, so Gemma might still do.
120B models are dying except nemotron, and Ornith did not even try to renewing qwen’s 122b, only 9B, 31B, 35B, 397B. When I looked back Qwen3.5 or 3.6 family’s renewed model, 122B is forgotten.
Visible 2 sparks range has deepseek v4 flash, stepfun, mimo which are good but Minimax went to above 256GB field. Most likely deepseek v4.1 flash will be similiar as v4 flash, But recently after fable appeared, Liang Wenfeng decided to get external big inventment to catch up 1T model size.
Ppl are guessing next deepseek would be bigger than current.
glm 5.2 appeared nicely and kinda doable on 4 sparks, still it seems room to get some speed improvement. As mentioned Minimax m3 joined 3 or 4 sparks field, and at 4, faster.
Ornith 397b would work better in 4 sparks.
For me, my payment for 2 extra 8.7k euros is fine for my freedome and flexibility, but when budget is tight, I would add just 1 or 2 rtx 3090 egpu on 2 sparks to have another flexibility for 27B, 31B ish dense models as reviewer or cross check.
Yes, Mac totally suck on long context, m5 max sweet spot under 128k. It’s GPU, 40 GPU of m5 max is much faster than half of m3 ultra (which is 2xm3 max) but still 70 tflops, vs realistic 220 tlops of single spark. Gen is faster but again even with prefix cache reprocessing cached medium size request kills any work flow. Ultra is not a competitor for two sparks, let alone 4. Look past the you tubers who who you one shot prompts to build a flappy bird from 0 context. Real work 8s nothing like that. Mac is great for subagents doing piecemeal tasks with short context, then it’s very fast and lives up to the hype.
Thanks for the suggestion. I wanted to update the thread - I got the 4 node setup with a router and the cables. I was able to run the nemotron ultra from eugr’s recipe to test the setup.
I will give the deepseek a try. I noticed it is added recently after u commeneted. I dont get to play with the cluster as much I would have hoped for at this point.
Also the Mac Studio 512 GB would likely cost ~15k USD if it comes out after the recent price hike. I was able to get the 4 node for ~4k each with the router ~1200$ and the cables ~600$.
I’m just telling myself that I made some money on MU(Micron) on the stock market, and using the gains for this cluster and making peace with the cost burden.