Hello,
NVIDIA Customer Care case: 261005-000268
I purchased three NVIDIA Jetson AGX Thor Developer Kits on August 21. NVIDIA Customer Care asked me to post the technical issue here so the Jetson/developer team can review it before they proceed with the RMA process.
All three units were flashed using NVIDIA SDK Manager with the same L4T image and configured for essentially identical workloads. Two are operating normally. The third has a persistent unit-specific issue where it becomes extremely hot under sustained GPU inference load and eventually hard-powers-off.
Problem
The affected Thor abruptly loses power after several minutes of heavy, sustained GPU utilization.
This is not a normal reboot or application crash. The entire system powers off, and the logs do not show a kernel panic, OOM event, orderly shutdown, or other normal OS-level explanation immediately before the failure.
This has now occurred three times:
-
September 24: hard power-off under sustained GPU inference load
-
September 25: hard power-off again under sustained GPU inference load
-
October 3: hard power-off again under sustained GPU inference load, after the unit had been completely reflashed
The September 28 reflash used the same L4T release used successfully on the other two Thor units. I then reinstalled the workload and reproduced the same failure.
Workload
The systems are being used primarily for local AI inference. The setup includes Docker and workloads such as Ollama, Whisper, and Flux.
The crashes occur during prolonged, sustained GPU utilization rather than during idle or light use.
The affected unit is running in the 120 W power mode.
Thermal behavior
Immediately before each hard power-off, the affected unit enters repeated GPU thermal-throttle states and logs:
thermal cooling_device9: gpu-throttle-alert cooling state: 0 -> 1
For the three failure events, I recorded approximately:
-
3,615 throttle-state transitions before the first shutdown
-
566 before the second shutdown
-
1,855 before the post-reflash shutdown
During the post-reflash incident, GPU temperature reached approximately 109°C, with the last telemetry before shutdown at approximately 108°C.
The other two Thor units are performing the same class of sustained workload without exhibiting this extreme thermal behavior or shutting down.
GPU / driver errors
During one of the pre-reflash incidents, the kernel logged the following GPU memory-mapping error approximately 31 minutes into sustained GPU load:
Cannot map memory with base addr 0xfffce0000000 and size of 0x2aa08 pages
This is the only occurrence of this type of GPU memory-mapping fault that I found in the persisted kernel record.
The affected unit also produces the following GPU initialization signatures on every boot:
xidfc_flesa | 0x00000040 |
and repeated:
NVRM_RPC_DCE: Failed RM ctrl call cmd:0x731341
including both:
NV_ERR_GENERIC
and
NV_ERR_TIMEOUT
These signatures were present on 14/14 recorded boots before the reflash and are still present on 3/3 recorded boots after the reflash.
Reflash test
To eliminate the existing installation/configuration state as much as possible, I completely reflashed the affected Thor and reinstalled the environment.
The other two Thor units use the same software/image successfully.
After the reflash:
-
The GPU initialization errors remained.
-
The affected unit continued to run significantly hotter than the other systems.
-
Under sustained GPU load, it again entered repeated
gpu-throttle-alertstates. -
The entire unit again abruptly powered off.
There was no kernel panic, OOM event, or normal shutdown associated with the post-reflash failure.
System information
Affected unit:
-
NVIDIA Jetson AGX Thor Developer Kit
-
GPU part number:
2B00--A1 -
L4T: R39.2
-
Kernel:
6.8.12-1021-tegra -
Ubuntu 24.04.4 LTS / aarch64
-
NVIDIA driver: 595.78
-
CUDA: 13.2
-
NV Power Mode: 120 W
I am not posting the unit’s full serial number publicly, but I can provide it directly to NVIDIA through the support case if needed.
Host-visible ECC/retired-page counters appear to be unavailable on this platform; the relevant nvidia-smi fields report N/A.
Fleet comparison
I have three Thor Developer Kits configured for the same project and running the same general workload.
The other two systems are working normally.
Only this physical unit:
-
reaches these extreme GPU temperatures,
-
repeatedly enters prolonged GPU thermal throttling,
-
experiences the hard power losses,
-
and reproduced the same behavior after a complete reflash.
I have retained the kernel logs, system logs, thermal telemetry, and pre- and post-reflash diagnostic captures and can provide them to NVIDIA if useful.
At this point the unit is not usable for sustained workloads.
Could the NVIDIA team please advise whether there are any additional diagnostics you would like me to run, or whether the observed behavior is sufficient to proceed with hardware replacement/RMA through Customer Care?
Thank you.