Cannot enter D3 state with dynamic power management on hybrid setup

I’m using the open driver version 590.48.01. I’m on an asus laptop with hybrid setup, using prime offload. I want the dGPU to sleep when not used, but I always see runtime_usage=2 or 1.

Some commands output:

cat /sys/class/drm/card0/device/power/runtime_status /sys/class/drm/card0/device/power_state /sys/class/drm/card0/device/power/runtime_usage
active
D0
2
sudo lsof | grep nvidia
lsof: WARNING: can’t stat() fuse.portal file system /run/user/1000/doc
Output information may be incomplete.
nvidia-mo  1026                             root cwd       DIR              254,0      4096          2 /
nvidia-mo  1026                             root rtd       DIR              254,0      4096          2 /
nvidia-mo  1026                             root txt   unknown                                         /proc/1026/exe
nvidia-mo  1027                             root cwd       DIR              254,0      4096          2 /
nvidia-mo  1027                             root rtd       DIR              254,0      4096          2 /
nvidia-mo  1027                             root txt   unknown                                         /proc/1027/exe
nvidia     1051                             root cwd       DIR              254,0      4096          2 /
nvidia     1051                             root rtd       DIR              254,0      4096          2 /
nvidia     1051                             root txt   unknown                                         /proc/1051/exe
cat /proc/driver/nvidia/gpus/0000:01:00.0/information
Model: 		NVIDIA GeForce RTX 4060 Laptop GPU
IRQ:   		232
GPU UUID: 	GPU-4e7c7272-6a73-5be0-7ab2-f49225bbeeca
Video BIOS: 	95.07.21.00.0e
Bus Type: 	PCIe
DMA Size: 	47 bits
DMA Mask: 	0x7fffffffffff
Bus Location: 	0000:01:00.0
Device Minor: 	0
GPU Firmware: 	590.48.01
GPU Excluded:	No
cat /proc/driver/nvidia/gpus/0000:01:00.0/power
Runtime D3 status:          Enabled (fine-grained)
Tegra iGPU Rail-Gating:     Disabled
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Supported
 Video Memory Off:          Supported

S0ix Power Management:
 Platform Support:          Supported
 Status:                    Disabled

Notebook Dynamic Boost:     Supported
sudo nvidia-smi
Sat Jan  3 19:07:41 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.48.01              Driver Version: 590.48.01      CUDA Version: 13.1     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4060 ...    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   40C    P8              1W /   35W |       2MiB /   8188MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
lsmod | grep nvidia
nvidia_uvm           2568192  0
nvidia_drm            143360  0
nvidia_modeset       2113536  1 nvidia_drm
nvidia              16248832  2 nvidia_uvm,nvidia_modeset
drm_ttm_helper         20480  2 nvidia_drm,xe
ecc                    49152  2 ecdh_generic,nvidia
video                  81920  5 asus_wmi,asus_nb_wmi,xe,i915,nvidia_modeset
cat /sys/bus/pci/devices/0000:01:00.0/power/runtime_active_kids

0
sudo cat /sys/kernel/debug/dri/0/clients
             command  tgid dev master a   uid      magic                                                             name                   id

I’m running with NVreg_DynamicPowerManagement=0x02, NVreg_TemporaryFilePath=/var/tmp, NVreg_OpenRmEnableUnsupportedGpus=1, nvidia-drm.modeset=1 nvidia-drm.fbdev=1.

It looks like no one is using the dGPU, but it doesn’t go to sleep, also the memory should be put in sleep or self refresh state but it doesn’t.

The only strange thing I noticed is that nvtop shows an higher memory usage: 0.373Gi/7.996Gi but I don’t understand who allocated that memory (if it’s right). I suspet it’s nvidia_drm.

Looking at the D3 sleep documentation there is no reference to drm and modesetting issue.

nvidia-bug-report.log.gz (356.6 KB)

I managed to get a constant runtime_usage=1 configuring the nvidia audio device to sleep: 0000:01:00.1 Audio device: NVIDIA Corporation AD107 High Definition Audio Controller (rev a1)

Also, with the closed driver and

options nvidia "NVreg_DynamicPowerManagement=0x02"
options nvidia NVreg_DynamicPowerManagementVideoMemoryThreshold=1024
options nvidia NVreg_S0ixPowerManagementVideoMemoryThreshold=1024
options nvidia NVreg_EnableS0ixPowerManagement=1
options nvidia NVreg_TemporaryFilePath=/var/tmp
options nvidia NVreg_EnableGpuFirmware=0

and `NVreg_PreserveVideoMemoryAllocations=1`it worked for a second, but only one time.

Hi @sfacenda , thanks for reporting this issue.
Could you try the following -

  1. Remove the nvidia-drm.fbdev=1 param
  2. Edit options nvidia "NVreg_DynamicPowerManagement=0x02" to 0x03 or remove it and re-install the driver.
  3. Verify that both /sys/bus/pci/devices/0000:01:00.0/power/control and /sys/bus/pci/devices/0000:01:00.1/power/control are set to auto instead of on.

Also, I don’t think you need the nvidia.NVreg_OpenRmEnableUnsupportedGpus=1 param in your cmdline arguments.
Please help to upload the bug report again if it doesn’t enter into D3 after the above steps, thanks.

Ok, I tried with both the open and closed drivers (and GSP disabled), no D3 sadly.

❯ grep . /sys/bus/pci/devices/0000:01:00.*/power/control
/sys/bus/pci/devices/0000:01:00.0/power/control:auto
/sys/bus/pci/devices/0000:01:00.1/power/control:auto
❯ sudo grep . /sys/module/nvidia_drm/parameters/*
/sys/module/nvidia_drm/parameters/fbdev:N
/sys/module/nvidia_drm/parameters/modeset:Y

nvidia-bug-report.log.gz (351.2 KB)

The report is done when using the open drivers.

NVreg_OpenRmEnableUnsupportedGpus is set by my distro when I use the open drivers and it’s not trivial to remove it.

Thanks @sfacenda , could you please share the output of cat /proc/fb from your system?

cat /proc/fb
0 i915drmfb

Thanks @sfacenda , I think we have one more thing left to check, could you please send the output of :

cat /sys/bus/pci/devices/0000:01:00.0/boot_vga
cat /sys/bus/pci/devices/0000:00:02.0/boot_vga
cat /sys/bus/pci/devices/0000:01:00.0/boot_vga
cat /sys/bus/pci/devices/0000:00:02.0/boot_vga
0
1

Have you already tried this? (worked for me)

Yes, no luck for me.

Hi @sfacenda , could you try after adding this kernel commandline option initcall_blacklist=sysfb_init ?

Just tried, with all the possible combinations of NVreg_DynamicPowerManagementVideoMemoryThreshold=0/1024, NVreg_PreserveVideoMemoryAllocations=1/0, still Runtime D3 status: Enabled (fine-grained) but

grep . /sys/bus/pci/devices/0000:01:00.0/power/*
/sys/bus/pci/devices/0000:01:00.0/power/async:enabled
grep: /sys/bus/pci/devices/0000:01:00.0/power/autosuspend_delay_ms: Input/output error
/sys/bus/pci/devices/0000:01:00.0/power/control:auto
/sys/bus/pci/devices/0000:01:00.0/power/runtime_active_kids:0
/sys/bus/pci/devices/0000:01:00.0/power/runtime_active_time:273785
/sys/bus/pci/devices/0000:01:00.0/power/runtime_enabled:enabled
/sys/bus/pci/devices/0000:01:00.0/power/runtime_status:active
/sys/bus/pci/devices/0000:01:00.0/power/runtime_suspended_time:0
/sys/bus/pci/devices/0000:01:00.0/power/runtime_usage:1
/sys/bus/pci/devices/0000:01:00.0/power/wakeup:disabled

Sometimes I see runtime_suspend_time:560 or others number <1000.

Hi @sfacenda , thanks for the update, could you attach a bug report after setting initcall_blacklist=sysfb_init?

nvidia-bug-report.log.gz (431.3 KB)

Hello @vanditd , any update?

Hi @sfacenda, sorry for the delay, I will try to repro this locally and provide an update.