Hello,
My Gigabyte GeForce RTX 4070 12 GB intermittently becomes inaccessible over PCIe. I am trying to distinguish a GPU/PCIe/power problem from a driver or GSP issue, without assuming a hardware diagnosis.
System used in the documented tests:
-
Linux Mint 22.3; kernel 6.8.0-142-generic.
-
NVIDIA 615.71.09, open kernel module with GSP.
-
ASUS ROG STRIX B760-F GAMING WIFI; BIOS 1836.
-
Corsair RM750e (2023), 750 W.
-
Desktop/display on Intel integrated graphics.
-
Recent compute tests: PyTorch 2.14.1+cu130 (CUDA runtime 13.0); ComfyUI 0.41.0 and 0.42.0.
Failure signature:
The kernel reports a correctable PCIe Physical Layer RxErr on the upstream root port, followed in log order by Xid 79 (GPU has fallen off the bus), then Xid 154 requesting OS reboot. nvidia-smi can no longer access the GPU. The log ordering does not establish the root cause. A complete power-off/on restores access in recent incidents; some software reboot attempts have hung.
Important results:
- Failures occurred in both motherboard slots: main slot 01:00.0 / root port 00:01.0, and lower slot 03:00.0 / root port 00:1c.0 (Gen3 x4). Forcing Gen3 in the main slot did not eliminate the issue.
- ComfyUI is not required. A standalone PyTorch script reproduced two failures in the main slot. It keeps approximately 4 GiB allocated, performs FP16 matrix multiplication for 5 minutes with another 1 GiB temporary allocation, releases the temporary allocation, then pauses for 120 seconds. In both runs the kernel recorded PCIe loss during the first pause; the script detects the CUDA error on resuming. Source and one failed run are attached.
- The same standalone scenario passed twice in the lower slot. This does not establish long-term stability: ComfyUI subsequently lost the GPU in that slot too.
- Disabling DynamicVRAM and cudaMallocAsync did not eliminate the problem. Some 100-generation sessions passed, but failures recurred. The latest lower-slot incident on October 19 at 17:05:13 +02:00 has RxErr/Xid 79/154 confirmed in the previous-boot journal. Both options were disabled according to my launch configuration, but I do not have a verified startup capture for that session.
- A 140 W power limit did not prevent an earlier incident. A short Vulkan VRAM test and a separate CUDA host/device copy verification passed, so failures are workload-dependent/intermittent, not a proven permanent memory corruption.
- Driver branches 595/610/615 were tried, all open/GSP. These were not identical prolonged A/B tests; the short successful 610 test does not establish that branch as unaffected.
- Cleaning/unblocking PSU airflow and changing the modular PCIe output did not eliminate the fault. A replacement PSU with its own cables is planned but has NOT yet been tested. I cannot currently cross-test this GPU in another suitable PC.
Diagnostic collection:
- nvidia-bug-report.sh hung during an earlier incident. The blocked process was reading /proc/driver/nvidia-modeset/dpys, before its nvidia-debugdump step. The partial report is retained.
- Direct nvidia-debugdump --ioctl --nvlogonly collection succeeded after GPU loss on several incidents. Original binary archives are retained unchanged. They are not in the public ZIP because I cannot audit all internal fields; I can provide them through a private channel designated by NVIDIA staff.
- nvidia-smi event-log and cper could not communicate with the driver after a previous incident.
- Attached kernel extracts are selected/redacted excerpts, not full bug reports. Hostname and GPU UUID are removed. Original logs are retained.
Could NVIDIA engineering advise:
- Can you analyze the retained NVLOG archives, and how should I submit them privately?
- Is there a more informative supported collection method before reboot when the GPU is already inaccessible and nvidia-bug-report.sh hangs?
- Is a controlled proprietary-module/GSP comparison supported and useful for this Ada GPU and driver stack, or is there a better diagnostic comparison?
- support-nvidia-20261019-public.zip (7.3 KB)
Thank you.