Hard power-off under sustained GPU load at ~90W, persists after full platform firmware update

My DGX Spark powers off completely (not a reboot, not a hang, a hard power-off) whenever sustained GPU load pushes power draw above roughly 90 W. The unit is rated at ~140 W. It is fully reproducible, and it is stable if I cap the GPU clock.

System

Field Value
Product NVIDIA_DGX_Spark
Serial 1983825002579
GPU NVIDIA GB10
BIOS 5.36_0ACUM018 (2025-08-06)
SOCFW 2.155.11
EC 3.5.8
USBPD 0.5.22
OTA OTA2607 (July 2026)
OS Ubuntu 24.04.4 LTS
Kernel 6.17.0-1021-nvidia
Driver 580.159.03

Reproduction

A stepped GPU load: FP16 matrix multiplications of increasing size (4096 up to 20480), run inside a container with PyTorch 2.11/cu130, logging temperature, power draw and clocks_event_reasons.active to disk every 15 s so the data survives the power-off.

Run A, default clocks:

step 4096     76 C   91.13 W   throttle 0x0
step 8192     79 C   91.96 W   throttle 0x0
step 8192 OK  81 C   88.61 W   throttle 0x0
step 12288    81 C   92.08 W   throttle 0x0
step 12288    83 C   90.28 W   throttle 0x0
step 16384    82 C   91.81 W   throttle 0x4    <-- SwPowerCap
--- SYSTEM POWERS OFF ---

Run B, GPU clock capped at 2200 MHz via nvidia-smi -lgc 300,2200:

step 4096     69 C   85.21 W   throttle 0x0
step 8192     81 C   90.53 W   throttle 0x0
step 12288    82 C   89.69 W   throttle 0x0
step 16384    82 C   90.98 W   throttle 0x4
step 16384    83 C   85.45 W   throttle 0x20   <-- SwThermalSlowdown
step 20480    83 C   82.64 W   throttle 0x20
=== all steps completed, no crash ===

Run B completed the larger workload that Run A died on, plus one step beyond it.

Platform firmware is current, and the failure persists

I applied every available platform firmware update via fwupdmgr (all flagged High urgency):

Component Before After
SOCFW 2.152.15 2.155.11
EC 3.3.2 3.5.8
USB-C PD n/a 0x00000516 applied

nvidia-spark-ota-check now matches OTA2607. The unit still hard powers-off under load after the update, and sooner than before:

before firmware update, free clocks:  died at step 16384  (91.81 W, GPU 82 C)
after  firmware update, free clocks:  died at step  8192  (88.82 W, GPU 83 C, CPU 97 C)

This matters because the Embedded Controller is precisely the component that manages power delivery and thermal response, and bringing it to the current release changed nothing.

Key observations

  1. The unit dies before thermal protection can engage. At free clocks no thermal reason bit is ever asserted: it goes from 0x0 straight to power-off, and in Run A it died at 82 C, cooler than the 83 C step it had just completed. With the clock capped, 0x20 (SwThermalSlowdown) does get asserted and the unit survives by throttling from 92 W down to about 82 W. Same peak power, opposite outcome. At free clocks the ramp appears too fast for the protection to react.

  2. CPU/SoC temperature reaches 92 to 97 C under load while the GPU reads 78 to 83 C. The GPU sensor never looks abnormal.

  3. No orderly shutdown. journalctl for the failing boots shows zero shutdown markers (Reached target Shutdown, systemd-shutdown[1], Powering off). The log simply stops mid-operation.

  4. No kernel panic captured. /sys/fs/pstore is empty after every crash.

  5. No hardware errors logged. rasdaemon (ras-mc-ctl --errors) reports nothing, and GPU ECC counters are clean.

  6. DCGM cannot stress this platform. dcgmi diag -r 3 reports Skip for targeted_power, targeted_stress, memory_bandwidth, memory, pcie and diagnostic on GB10; only the software/deployment group runs. nvidia-smi reports power.limit, power.max_limit and all temperature thresholds as N/A, so there is no documented envelope to compare against.

  7. Memory is not the constraint. The unit died with 36.7 GB used of 124.6 GB in one run and 99.5 GB in another, never out of memory. This is a pure matmul workload, so there is no KV cache growth or UVM livelock involved.

Environment, already ruled out

  • Enclosure: the unit was in a cabinet. Moving it into open air changed load temperatures by about 1 C and did not prevent the crash.
  • Mains supply: crashes occur identically on wall power and on an EcoFlow River 2 Pro (800 W output, mains-connected). Three electrical configurations, same ~90 W ceiling.
  • Ambient: normal office conditions, idle temps 41 to 48 C.

Questions

  1. Is the ~90 W power-off threshold a known defect on this unit or batch?
  2. Since platform firmware is now current and the failure persists, what is the next diagnostic step?
  3. Could you provide access to DGX Spark Field Diag? I have seen it requested in similar threads and would like to supply that data.
  4. If this is a hardware fault, I would like to start an RMA. The unit was purchased at Micro Center (Miami).

Happy to attach nvidia-bug-report output from both before and after the firmware update, plus the per-minute temperature/power/throttle logs covering each crash.

It looks like you are experiencing a know issue. The current workaround is to lower your GPU clock max as you described.
You can install and run the DGX Spark Fieldiag by following the user guide: https://docs.nvidia.com/pdf/userguide-dgx-spark-fieldiag.pdf