PCI PM fails with modesetting enabled in multi-GPU config in a console/tty env

Graphics / Linux Linux
,
1

NVIDIA Open GPU Kernel Modules Version

550.67

I confirm that this does happen with the proprietary driver package as well

Operating System and Version

Arch Linux

Kernel Release

Kernel 6.8.4 from Arch Repositories

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

  • I am running on a stable kernel release.

Hardware: GPU

GPU 0: NVIDIA GeForce GTX 1660 Ti (UUID: GPU-fd77e18f-2c78-5a1c-48f8-f7869e44aa8e)

Describe the bug

On a Multi-GPU system, where the NVIDIA GPU is NOT the primary GPU, PCI power management fails if modesetting is enabled on nvidia_drm in a console/tty environment. However, PCI power management functions correctly if modesetting is disabled

To Reproduce

  1. Have the following options in /etc/modprobe.d/nvidia.conf:
options nvidia_drm modeset=1
options nvidia NVreg_DynamicPowerManagement=0x02
options nvidia NVreg_EnableGpuFirmware=1
  1. Have a multi-GPU setup, such as a laptop, where the NVIDIA GPU is not the primary GPU.
  2. Disable your Display Manager/boot to multi-user.target
  3. Run echo auto > /sys/bus/pci/devices/0000\:01\:00.0/power/control as root
  4. Run grep . /sys/bus/pci/devices/*/power/runtime_status a few seconds later. The GPU state shows as active.
  5. Change options nvidia_drm modeset to 0 and reboot to multi-user.target/tty
  6. Repeat steps 3 and 4. The GPU state shows suspended

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

Ideally this should not be happening since the NVIDIA GPU is not driving the tty either. There are no processes on the NVIDIA card, as verified by nvidia-smi, as well as lsof /dev/dri/render*

The GPU also suspends correctly in an X11 environment.

Full output of dmidecode: dmidecode.log (12.0 KB)
Laptop model: Acer Nitro 7 AN715-51 (2019 - i5-9300H, GTX 1660Ti)

Moved here from open kernel module github issues

2

for anyone coming across this, the Turing GSP does not and probably will not support suspending the GPU entirely