Quadro RTX 6000 (Turing) — DPR on WDDM: programmatic page retirement for known stuck-at-0 VRAM defect?

Gaming and Visualization Technologies General Topics & Other SDKs GPU - Hardware
1

I have a Quadro RTX 6000 (Turing, sm_75, 24 GB GDDR6, driver 580.88, CUDA 13.0, Windows 11 23H2) with a confirmed physical GDDR6 defect: multiple stuck-at-0 cells in a localized ~64 KB region at ~1.6 GB VRAM offset, confined to a single GDDR6 chip (chip #16).

Current state:

  • ECC enabled (stable, no TDR after TdrDelay=10 / TdrDdiDelay=10 registry adjustments)
  • 8 Double Bit ECC pages already retired (512 KB coverage via organic DBE detection during prior CUDA workloads)
  • DPR counter: 8 DBE, 0 SBE, Pending Page Blacklist: No
  • Aggregate ECC: 3323+ DRAM Uncorrectable (historical), 112+ DRAM Correctable
  • Compute mode: WDDM (Default) — TCC not possible despite multiple attempts

TCC attempt (failed):

  • nvidia-smi -i 1 -dm 1 → reboot → Code 10 in Device Manager
  • The Quadro RTX 6000 refuses to initialize in TCC mode on this system
  • Reverting to WDDM (-dm 0) restores normal operation
  • This blocks nvidia-smi -r (unsupported under WDDM) and may affect DPR reliability per the v560 documentation

What I attempted for programmatic DPR under WDDM:

  1. 0xFFFFFFFF SBE pattern — all-ones write to the bad region. Single stuck-at-0 cell = 1 bit wrong per ECC word → correctable. Result: uncorrectable ECC error — multiple stuck-at-0 cells in the same ECC word produce 2+ bit errors.

  2. Walking-1 bit pattern (32 passes) — exactly 1 bit written per word → at most 1 bit wrong. Result: bit 0 detects 11 SBE pages, but DPR counter does NOT increment even after 2 passes. Bit 1 write triggers uncorrectable ECC error — the hardware memory scrub reads the region between controlled passes, detects multi-bit errors on unretired pages, and poisons the CUDA context.

  3. devcon disable/enable — resets the driver but does NOT clear ECC error state or allow nvidia-smi -r. The scrub resumes immediately and re-poisons the context within seconds.

  4. nvidia-smi -e 0 → reboot → vram_mapper_detailed (ECC OFF) → nvidia-smi -e 1 → reboot — maps the defect reliably (16 bad 4 KB pages confirmed at chunk 55), but provides no path to retire them.

Questions:

  1. Is Dynamic Page Retirement officially supported on WDDM? The v560 DPR documentation states “Windows TCC platforms” as a supported OS — does this mean DPR retirement mechanics are disabled or unreliable under WDDM, even though ECC error counters are visible in nvidia-smi?

  2. Is there any NVAPI function (documented or internal/OEM) that allows targeting a specific physical VRAM address for forced page retirement, bypassing the organic ECC detection path? Something analogous to nvmlDeviceGetRetiredPages but in the write direction.

  3. Is the hardware memory scrub configurable or disable-able at any level (driver registry, NVAPI, vbios)? It currently prevents any SBE-driven DPR approach by triggering uncorrectable errors between controlled writes.

  4. For Quadro GPUs that hit Code 10 in TCC, is there any supported WDDM path to achieve a full GPU reset (equivalent to nvidia-smi -r) without an OS reboot? The devcon disable/enable cycle is insufficient.

Any insight into NVAPI internals, WDDM DPR behavior, or alternative retirement paths would be greatly appreciated — even if the answer is “not possible,” that’s valuable data for the community.

Regards

2

Hi, do you have any other gfx device to run the windows desktop on, in the same system? Windows mandatory needs a GPU, in WDDM mode, to host its desktop, no way to run the single GPU in TCC mode… so maybe, adding another GPU will allow for setting the RTX into TCC really…?

-Frank

3

Hi Frank,

Yes, I already have an RTX 3070 Ti (main GPU) handling the Windows desktop in WDDM mode. The Quadro RTX 6000 is a secondary GPU. Even with this setup, nvidia-smi -dm 1 followed by reboot results in Code 10 on the Quadro.

Regards

Tsitola