Why we need 8 leaf switchs and 4 spine switchs in one su DGX superpod

Infrastructure & Networking Switches and Gateways Ethernet Switches
1

why we need 8 leaf switchs and 4 spine switchs in one su DGX superpod?

2

DGX SuperPOD is 20 DGX servers.
Each has 8 HCAs.

8 leaf switches with 20 downlinks each accommodate the 20*8 dgx ports

4 * 40 spine ports connect the 8*20 uplinks from the leaf switches.

4

Thank you for your answer
we used h100 server and QM9700switch
96 DGX servers.
so,96*8 dgx port need 24 leaf switches? I don’t know if I’m right.
and How many spine switches do we need? 12 ?

5

Yeah, your math is basically right.

96 DGX × 8 ports = 768 server links.
With QM9700 (64p), if you keep it non-blocking, you usually split it ~32 down / 32 up.

768 ÷ 32 = 24 leaf switches, so that part checks out.

For spines:
24 leaves × 32 uplinks = 768 leaf-to-spine links total.
768 ÷ 64 = 12 spine switches.

So a 24 leaf / 12 spine fabric is exactly what you’d expect for 96 DGX with full bandwidth.

7

Hey🙂, in a DGX SuperPOD the leaf switches connect to the nodes and the spine switches connect the leafs together. 8 leaf + 4 spine just makes sure all the nodes can talk fast without bottlenecks. Mostly it’s about speed and scaling.

I happened to come across a pretty good article that explains the system layout: DGX B300 SuperPod System Architecture Introduction: Based on Ethernet and InfiniBand Networking