Nvidia GB200 NVL72

Gaming and Visualization Technologies General Topics and Other SDKs GPU - Hardware
1

Hi,

How many GPUs can be accessed from a single process on the Nvidia GB200 NVL72? Can a single process access all 72? I’ve written some test code for this - can someone from NVIDIA please run this on one of these machines?

curl -sSL https://raw.githubusercontent.com/MattMcL4475/gpu/refs/heads/main/test_gpus.py | python3 -

Thank you!
Matt

2

Hi @MattMcLoughlin and welcome to the NVIDIA developer forums.

I am afraid we cannot download this kind of content and run it on one of our internal GB200s.

And since GB200s are currently only supplied through Cloud and Service providers I suggest you contact their support directly.

In terms of implementation you sure should be able to access all GPUs from one process. That is the whole idea of NVLINK. Sorry to quote marketing, but

The GB200 NVL72 is a liquid-cooled, rack-scale solution that boasts a 72-GPU NVLink domain that acts as a single massive GPU.

How you parallelize or distribute workloads is completely up to the implementation.

3

Hi Markus, thanks for the reply. NVLink will ensure memory access, but more specifically, can a single process launch a kernel on all 72? Are there any technical documents besides the marketing collateral that might confirm this?

I wrote the simplest possible code to test this using PyTorch: here is the GitHub link.

Can you confirm that this same code (or equivalent) would successfully execute matmul on all 72? That would definitively answer the question.

Thank you,
Matt