Request for sm_121-tuned kernels in cuDNN/cuBLAS — DGX Spark training throughput gap

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
,
1

Hi NVIDIA team,

I’m running training workloads on a DGX Spark (GB10, compute capability 12.1 / sm_121) and observing a significant per-epoch throughput gap compared to a discrete consumer GPU (RTX
4080, sm_89) on the same code, dataset, and PyTorch version. I’d like to formally request that cuDNN/cuBLAS ship sm_121-tuned kernel paths, and ask whether there is any public
roadmap commitment for this.

Hardware and software

  • DGX Spark — NVIDIA GB10 Blackwell, compute capability 12.1 (sm_121), 119 GB unified LPDDR5X
  • Reference comparator — NVIDIA GeForce RTX 4080, compute capability 8.9 (sm_89), 16 GB GDDR6X
  • Driver: NVIDIA 570.x series
  • CUDA toolkit: 12.8
  • PyTorch: stable channel (built with TORCH_CUDA_ARCH_LIST ending at sm_120 / compute_120 — no sm_121 binaries shipped as of this writing)
  • cuDNN: 9.x bundled with PyTorch
  • Workload: medium-scale audio classification training (multi-label CNN, EfficientNet-B0 backbone via timm, bfloat16 amp, batch size 64)

Observed behavior

┌─────────────────────────┬──────────────────────┬─────────────────────┐
│ GPU │ Per-epoch wall-clock │ Ratio vs sm_89 4080 │
├─────────────────────────┼──────────────────────┼─────────────────────┤
│ RTX 4080 (sm_89) │ ~18.5 min │ 1.0× (reference) │
├─────────────────────────┼──────────────────────┼─────────────────────┤
│ DGX Spark GB10 (sm_121) │ ~52 min projected │ ~2.8× slower │
└─────────────────────────┴──────────────────────┴─────────────────────┘

A second prior workload (EfficientNetV2-S, same family) measured an even wider gap of ~4.2× slower on the same DGX Spark (13.4 min/epoch vs 3.2 min/epoch on the same 4080, same
code, same seed, same data).

The GPU itself appears healthy — nvidia-smi shows brief utilization spikes to 96% during kernel execution windows, indicating the GPU IS doing real work when batches arrive. The gap
is not a hardware fault.

What I believe is happening

Per public discussion (and PyTorch’s own arch-list at build time), PyTorch stable currently ships sm_120 binaries. GB10 is sm_121 — a different SM revision from sm_120 (RTX
50-series consumer Blackwell). On GB10:

  • cuDNN convolution kernels and cuBLAS GEMMs fall back to generic / sm_120 paths rather than sm_121-tuned ones.
  • NVRTC JIT compilation targeting sm_121 fails outright (no compiler support visible at our toolchain version).
  • Triton heuristics use generic paths.

The combination of these — plus the GB10’s lower memory bandwidth (~273 GB/s LPDDR5X vs ~717 GB/s GDDR6X on the 4080) — appears to dominate throughput on memory-bound depthwise /
inverted-bottleneck convolutions, which are common in modern CNN backbones.

My ask

  1. Is there a committed roadmap for sm_121-tuned kernels in cuDNN/cuBLAS? Public posts I’ve seen from NVIDIA representatives acknowledged a DGX Spark training-throughput regression
    but stopped short of committing a timeline. A roadmap commitment — even a vague quarter — would help DGX Spark customers plan around this.
  2. Is there a near-term workaround on the toolchain side? For example: an NGC PyTorch container with internal cuDNN tunings that aren’t yet in the public release, a CUDA 13.x
    preview with broader compute-capability coverage, or a recommended environment configuration.
  3. Should DGX Spark customers expect the existing sm_120 fallback path to be the long-term plan? If yes, that’s a useful clarification — it would inform purchasing and capacity
    decisions. If no, what’s the rough horizon?

Context

I understand DGX Spark is positioned as a unified-memory development workstation and large-model inference rig, not primarily as a training accelerator. That’s a reasonable product
framing, but the current gap means many real-world training workloads (CV backbones, audio classifiers, anything memory-bound) run materially slower than on a consumer 4080 — and a
sm_120-tuned cuDNN release would presumably close most of that gap without any hardware change.

A formal roadmap commitment, or an NGC-container path, would let customers like me plan deployment and budgeting decisions with less uncertainty.

Thanks in advance for any update you can share.

2

We run FSDP training on GB10 (sm_121) as well, so following this — a roadmap answer on sm_121-tuned cuDNN/cuBLAS paths would be genuinely useful.

One thing worth ruling out before pinning the whole gap on kernels: how much of your per-epoch delta is compute-bound vs memory-bound? GB10’s unified LPDDR5X has substantially lower memory bandwidth than the 4080’s GDDR6X, and a lot of a training step (optimizer state, activations, normalization, elementwise ops) is bandwidth-bound rather than FLOP-bound. If your workload leans that way, sm_121-tuned GEMM/conv kernels would speed up the compute-bound fraction but wouldn’t touch the memory-bound fraction — so the gap might not close as much as the kernel request implies.

Quick way to tell: profile one step in Nsight Compute and check whether the hot kernels are hitting SM/tensor-core throughput limits or DRAM-bandwidth limits. If it’s mostly DRAM-bound, that reframes both the ask and the expectation. Not arguing against the kernel request at all — just that on a unified-memory part the bandwidth ceiling is often the bigger lever, and it’s worth knowing which one you’re actually hitting before comparing against a discrete-VRAM card.

3

Good point — you’re right that the bandwidth math (LPDDR5X at ~273 GB/s vs GDDR6X at ~717 GB/s, ratio 2.6×) lines up suspiciously well with my observed 2.7–2.8× slowdown, and even
closer on V2-S where depthwise is more bandwidth-bound. I should disambiguate that before pinning the gap on kernels.
Reframed ask to NVIDIA: can someone confirm whether the throughput gap on memory-bound CNN training workloads on GB10 is dominated by LPDDR5X bandwidth (i.e., a hardware tradeoff
customers should plan around) or by missing sm_121-tuned cuDNN/cuBLAS paths (i.e., a software fix that could close most of the gap)? Either answer is actionable. The current
uncertainty is what’s hard to plan around.

4

Measured GB10 LPDDR5X bandwidth under real workload conditions:

  • GPU read idle: 161 GB/s

  • GPU read under inference load: 90 GB/s (−44%)

  • GPU write idle: 116 GB/s

  • GPU write under inference load: 62 GB/s (−46%)

The 273 GB/s figure is peak theoretical. Under sustained load the effective bandwidth collapses significantly. For memory-bound training workloads — depthwise convolutions, optimizer state, activation’s — this is the real ceiling, not the marketing number.

sm_121-tuned kernels would help the compute-bound fraction but would not recover the memory bandwidth headroom lost under load pressure. Both gaps are real and compound each other.

Full measurement methodology and baseline data: https://forums.developer.nvidia.com/t/gb10-hardware-baseline-first-direct-measurements-and-findings/367851

5

someone claims they fixed slowness when training qwen 3.5 in unsloth GitHub - albond/DGX_Spark_Unsloth_Lossless_Speedup: 7.67× LoRA / 8.35× Full FT speedup for Qwen3.5 (0.8B–27B) on NVIDIA DGX Spark — wall-clock parity with rented H100. Lossless within BF16. Three-command interactive wizard handles model picker, data validator, training, and merge. · GitHub