RTX 5090 on Windows — three faults logged at 200 ms, with +12V measured at both ends of the 12VHPWR cable
I’ve been chasing this since August and I’ve got three captures of it now, so I’ll put the data up in case it’s the same thing you’re all seeing.
The short version of what happens: I’m in a game, the display goes black, the audio carries on playing normally. Windows is still alive at that point — it answers pings, CPU is idling along. Then sometimes it freezes completely a few seconds later and the audio starts looping a short buffer, and sometimes it just carries on running for as long as I leave it, with the card dark. When it does freeze, nothing but a power button hold gets it back regardless.
It’s a Windows box so there’s no Xid in any of this, which I know makes it less useful to you. What I do have is current and voltage at both ends of the 12VHPWR cable, sampled every 200 ms, through the fault.
The machine
|
|
| GPU |
MSI GeForce RTX 5090 32G SUPRIM LIQUID SOC, PCI 10DE:2B85, subsystem 1462:5300 |
| VBIOS |
98.02.2E.40.9F |
| Clocks |
stock, 600 W board limit, 2565 MHz boost |
| Driver |
616.92 (32.0.16.1692) Game Ready WHQL |
| OS |
Windows 11 24H2, build 10.0.26100.9457 |
| CPU |
i9-13900KF, microcode 0x12F |
| Board |
ASUS PRIME Z790-P WIFI, BIOS 1825 |
| PCIe |
x16 slot at Gen4, link trained 16 GT/s x16 throughout |
| RAM |
4 x 16 GB DDR5 at 4200, XMP off |
| PSU |
MSI MEG Ai1300P 1300 W — peak system draw I’ve seen is 930 W |
| Displays |
3 x LG 24GL600F on DisplayPort, NV Surround, one 5760x1080 @ 143 Hz surface |
| Cooling |
AIO, 360 mm radiator chassis-mounted |
This machine also runs Hyper-V, VBS, WSL2 and VMware, so GPU paravirtualisation is active. I haven’t tested with all that ripped out.
The oldest crash dump I still have is 12 August. Three driver builds since then.
How I got the data off the machine
Since the box can freeze and take the event log with it, some of this is read off the machine afterwards — but mainly it’s all streamed to a second machine while it’s happening.
HWiNFO64 publishes its sensor tree into shared memory. A service reads that block and pushes every sensor over TCP to a Linux box on the LAN at 5 Hz, which writes it to SQLite at its end. A second agent streams new event log records, process lists and the foreground window title over the same connection as they’re written. And the Linux box pings the Windows box once a second, which is the only witness in the set that isn’t running on the machine under test — that’s what tells me whether the logger died or the machine did.
Two of the sensors do the heavy lifting on the power side, and they’re independent of each other and of the GPU:
+12V5 current, voltage and power, reported by the Ai1300P over its own USB link. That rail is the PSU’s 12VHPWR feed. It’s measured at the PSU, and it doesn’t care whether the GPU, the driver or PCIe are alive.GPU 12VHPWR Voltage, reported by the card, measured on the card at its own connector. Far end of the same cable.
So I can watch both ends of the cable at once, which is the bit I thought might be useful here.
First one — 00:53:43
Cyberpunk 2077 running. 1856 consecutive polls at 4.92/s across the event, no gap anywhere over a second.
It goes in one 365 ms step:
|
00:53:43.327 |
00:53:43.692 |
+12V5 Current |
46.75 A |
3.75 A |
+12V5 Power |
558.1 W |
45.1 W |
+12V5 Voltage |
11.938 V |
12.016 V |
| PSU total |
805.5 W |
301.5 → 203.5 W |
Here’s GPU 12VHPWR Voltage — measured on the card — for the last eleven samples before it stops updating:
11.711 11.713 11.691 11.715 11.701 11.701 11.696 11.704 11.705 11.698 11.694
For that whole window the PSU end is sitting at 11.92–11.97 V while delivering 47 A.
Core was 56.88 °C, hot spot 68.59 °C on the death sample. GPU PCIe +12V Input Power — the slot — is about 11 W the whole time.
I’ve got PresentMon in the same stream. GPU Wait (avg) reads 0.112 ms at the death sample, and 0.124, 0.117, 0.133, 0.132, 0.120, 0.145, 0.118, 0.131, 0.112, 0.129, 0.112 across the two seconds before it. Frame time 3.79–3.86 ms. Frames kept presenting for about a second after the card stopped pulling power.
Every PCIe error counter in the HWiNFO tree stayed at 0 and never moved: Receiver Errors, Replay Count, Bad TLP, LCRC, Correctable, Non-Fatal, Fatal. WHEA Total Errors 0.
After that: display dead, audio still playing, Windows responsive, CPU 7–16%, polls still arriving on time. 96.5 seconds later it stopped answering. Last twenty poll intervals before it went:
227 199 198 231 189 200 198 235 229 225 190 198 238 185 228 229 200 200 189 240 ms
Then nothing.
Second one — 19:50:48
Cyberpunk again, but this one died sitting on a loading screen, which surprised me. I’ve got the instant bracketed to 500 ms.
time +12V5 V +12V5 W 12VHPWR W 12VHPWR V CoreLd GPU W GPU C
19:50:47.9 11.938 552.109 531.020 11.708 97 546.508 56.086
19:50:48.0 11.969 544.578 543.314 11.702 94 -- --
19:50:48.5 12.031 33.086 537.026* 11.706* 96 546.927* 56.117*
19:50:48.8 12.047 51.199 (card-sourced readings never update again)
19:50:49.2 12.031 63.164
19:50:53.9 12.031 42.109
19:51:15 12.047 21.082 (+12V5 Current 1.75 A)
* = last latched value, already stale
Everything the card reports — power rails, clocks, core load, bus load, GPU 12VHPWR Voltage, GPU PCIe +12V Input — freezes at 19:50:48 and never updates again. Everything the PSU reports keeps updating for as long as the machine is up. That’s why the starred values look like the card is still drawing 537 W after it’s dead; it isn’t, that’s just the last number it ever gave me.
Both ends of the cable over the 47 seconds before it, at 546 W:
|
|
+12V5 at the PSU |
11.906 – 11.969 V |
GPU 12VHPWR Voltage at the card |
11.695 – 11.718 V |
| difference |
0.233 V at ~45.6 A, so ~5.1 mΩ across the cable and both connectors |
At the death instant the PSU rail reads 12.031, then 12.047 V.
Core 56.1 °C, memory junction 68 °C on the last healthy sample. I’ve also got a thermistor taped to the 12VHPWR cable reported by a Corsair Commander, so not on the card — it peaked at 46.3 °C in the first 5-second sample after the fault and then decayed steadily.
Event stream for this one, streamed off as it was written:
19:50:48.0-48.5 the card stops
19:50:50 nvlddmkm id 14 x21 <- first error, 2.0 s later
19:50:50-51:14 id 153 at ~7.5/s, id 14 x2 every ~6.5 s
19:51:15 id 153 rate rises to ~13/s
19:51:24 id 14 x22
19:51:47 flood ends. Totals: 60 x id 14, 613 x id 153
19:51:49 card answers one last NVAPI query, then never again
19:51:56 WER flush: LiveKernelEvent 0x141, LiveKernelEvent 0x1B8, 0x116
That last reply at 19:51:49 came back with 470.0 °C on all sixteen memory dies, 0.0 °C hot spot, 0.0 °C memory junction, 0 RPM on both fans, which I assume is what a dead device returns rather than anything real.
WHEA-Logger produced nothing at all through the whole event.
Power settles in stages afterwards: 544 W to 33 W in one sample, then about 42–45 W for thirty seconds, then 21.08 W at 1.75 A.
This is the one that never froze. Still running normally seven minutes later with the card dark.
Load going in, PresentMon again:
19:49:40-53 ~400 FPS GPU 21% 143 W Busy 0.91 ms Wait 1.60 ms menu
19:49:54-55 48 FPS GPU 62% 97 W Busy 9.55 ms Wait 7.80 ms load hitch
19:49:56 208 FPS GPU 81% 333 W Busy 4.18 ms Wait 0.55 ms loading screen
19:50:00-48 ~258 FPS GPU 95% 546 W Busy 3.80 ms Wait 0.08 ms 52 s, uncapped
19:50:48 258 FPS GPU 96% 547 W Busy 3.85 ms Wait 0.07 ms last sample
GPU Wait sat at 0.07–0.10 ms for 52 seconds straight, uncapped, at 91% of board power — and the game was on a loading screen for all of it, not in gameplay.
Third one — 20:47:00
Cyberpunk again.
20:46:59.536 GPU Power 545.234 W, GPU Clock 2842 MHz <- last change of anything on the card
20:46:59.988 nvlddmkm id 153, level 2 <- the only one this time
20:47:00.135 +12V5 Current 47.25 A -> 3.50 A <- one 200 ms sample
+12V5 Power 563 W -> 42.055 W
GPU 12VHPWR Voltage 11.686 -> 11.681 V
Total CPU Usage 31% -> 13.84%
20:47:00.360 last HWiNFO poll received
20:47:02.693 agent's last transmission — it died mid-loop
Current drops 93% inside one 200 ms sample. The voltage at the card goes 11.686 → 11.681 V, which is inside the 11.687–11.727 V band it had been in for the preceding seconds.
Machine froze 2.56 seconds after the card went.
Going in: 37.2 seconds continuously at 80%+ GPU load, frame time about 3.78 ms, so ~263 FPS, uncapped. No WHEA records anywhere in the session.
What’s the same in all three
- Current at the PSU drops about 93% inside one sample, and in the sample after, the voltage at both ends is flat or higher.
- No WHEA-Logger records at all, any severity, in any of the three.
- PCIe error counters polled at 5 Hz right up to the death sample — all zero, no increments.
- Link trained at 16 GT/s x16 at the last reading.
- 56–57 °C core, about 68 °C memory junction at the death sample.
- Frame time and
GPU Wait unchanged in the samples before it goes.
- The card’s own telemetry and the card’s current stop in the same 200 ms sample. On the second one, where I got the full event sequence, the driver’s first log entry comes 2.0 s after that.
One long session that didn’t crash
For whatever it’s worth as a comparison — a 7 hour 16 minute Valheim session, frame-capped, no fault. From that log, 13,133 samples:
Framerate [FPS] p50 60.0 p99 60.0 max 61.0
Frame Time Presented p50 16.7 ms
GPU Wait (avg) p10 2.10 ms p50 5.19 ms p90 12.40 ms
GPU Busy (avg) p50 11.32 ms
GPU 12VHPWR Power p10 422 W p50 504 W p90 556 W max 642 W
GPU Core Load p50 71% 13.7% of samples >= 80%
Next to the three faults:
|
clean 7 h session, capped |
the three faults, uncapped |
GPU Wait (avg) |
5.19 ms median |
0.07 – 0.11 ms, held 37–52 s |
| sustained 12VHPWR power |
504 W median, 642 W peak |
546 – 563 W at death |
| min 12VHPWR voltage |
11.534 V |
11.681 – 11.708 V |
| peak PSU +12V current |
75.0 A |
~47 A |
| peak PSU output |
930 W |
805 W |
| min free VRAM |
1,295 MB |
1,302 MB |
| duration |
7 h 16 m, no fault |
dead in 37–52 s |
Different game though, so it’s not a controlled comparison and I’m not going to pretend it is. I haven’t yet run the same title capped and uncapped back to back, which is obviously the thing to do.
The bugchecks
When it bugchecks instead of freezing, the parameters come out the same every time:
0x116 VIDEO_TDR_FAILURE
arg3 = 0xffffffffc000009a (STATUS_INSUFFICIENT_RESOURCES)
arg4 = 0x0000000000000004
Five dumps, identical in arg3 and arg4. Uptime at bugcheck runs from 7 minutes to 6 hours 21 minutes, so it’s not an uptime thing.
arg2 lands on the same code location inside nvlddmkm across two different driver builds — the module grew by exactly 0x5000 between them and the offset moved by the same 0x5000.
Whether it takes the machine with it
| capture |
what happened |
| 1 |
froze 96.5 seconds later |
| 2 |
never froze, still running normally 7+ minutes later |
| 3 |
froze 2.56 seconds later |
Same game, same sort of load, same card. When the freeze does come, poll cadence and CPU load are completely normal right up to the last sample. Once it’s frozen, only a power button hold gets it back — a warm reboot has never worked.
Things I’ve changed that didn’t stop it
- PCIe Gen4 — happens at Gen4.
- Driver — three builds now.
- VBIOS — already on the newest one that exists for this subsystem ID, so there’s nothing to update to.
- Hardware-accelerated GPU Scheduling — happens with it on and with it off.
And some config I’ve checked but haven’t changed, in case it saves anyone asking:
- ASPM is Off on AC in the OS. Both the GPU and its root port report
DEVPKEY_PciDevice_SupportedLinkSubState = 0, so no L1 substates supported at all.
- Root port AER is fully armed and nothing’s masked —
Uncorrectable_Error_Mask 0, Correctable_Error_Mask 0, Error_Reporting 7, RootError_Reporting 7.
TdrLevel, TdrDelay and TdrDdiDelay are all absent from the registry, so TDR is at Windows defaults.
Why I’m putting it here
Most of what’s in this thread is Linux, mostly 5080s, and measured at the PSU. Mine’s Windows, a 5090, and measured at both ends of the cable. The current and voltage behaviour at the moment it dies looks the same to me as what’s been described here, which is why I’m adding to this rather than starting another thread.
Happy to hand over the raw data to anyone who wants it: per-sample SQLite of every HWiNFO sensor at 200 ms across all three faults, the streamed event log records, and the five 0x116 minidumps.