(DGX Spark, ARM64, CUDA 13) pathological slowdown for many-small H2D copies from pageable CPU memory (≈50× vs pinned); impacts PyTorch model load patt

[alan.dang]

Out of curiosity, what is the performance when you use the PyTorch container?

PyTorch | NVIDIA NGC

My CUDA13 benchmark showed that CPU to GPU was still decent.

Enabling GPU Direct RDMA for DGX Spark Clustering

You don’t benefit from GPU Direct if you’re writing your own code because of unified memory. The memory pointers for host and device are identical. So if a GPU writes to itself, and the CPU just pulls that memory pointer directly without using cudaMemcpy, you get maximum bandwidth. Then over 200GbE, you’re basically saturating the network connection already and can push data from GPU2 to GPU1 at >22GB/sec. If you’re using legacy software that falls down to cudaMemcpy as fallback when it does…

I only pasted the relevant stuff to the thread but the benchmark did test local CPU to GPU