I’m planning to connect two DGX Spark nodes to run distributed LLM workloads.
In that case, besides just using a larger combined RAM pool, if I distribute thousands of jobs, will they actually be divided between the two nodes for parallel execution?
If so, would that provide a significant speed advantage?
Fundamentally, aside from expanding RAM capacity, what other benefits are there to connecting two DGX Spark systems together?
The main benefit would be that you can run a large model. The inference would still be capped at theoretical maximum 273 GB/s.
If you want speed use two smaller models and serve them as replicas. It won’t speed up a single inference, but it will enable you run run two concurrent inferences before a queue builds.
In general though DGX spark is not a production environment. It’s goal is to provide stability for dev environments with the intention to deploy into larger DGX environments.
For setting up as a cluster where you do have replicas, DGX spark supports that but it’s up to you to write the code to deploy in that way. I.e K3s/k8s with a load balancer
Thanks for your reply.