NVidia dGPU in hybrid/optimus laptop not powering down after unplugging external monitor

Graphics / Linux Linux
1

As a quick summary before I give the detailed information, on my hybrid/optimus nvidia graphics laptop, I am not having any kind of graphics issues, including when I use an external monitor. However, if I plug in then unplug an external monitor, the nvidia gpu does not return to the power saving state as I would like it to.

Now for the more in-depth explanation:
Setup:

  • ASUS ROG Zephyrus G14 laptop, which has both an integrated AMD graphics card as well as a discrete nvidia GeForce RTX 4070 card, which the HDMI port on my laptop is connected to.
  • Latest versions of Arch linux (6.12.6) and nvidia/nvidia-utils (565.77) drivers, all freshly installed with no additional configuration or packages.
  • I am using the latest version of Hyprland and SDDM with no additional configuration for testing graphic programs. I do not think this is relevant, since the problem happens even without launching Hyprland or SDDM.
  • Using PRIME rendering offloading (prime-run command to specify programs to use the nvidia gpu) for usage of the nvidia gpu and RTD3 for power management
  • The power state of the nvidia gpu can be queried from /sys/class/drm/card0/device/power_state, and it returns D0, D1, D2, D3hot, or D3cold depending on the power state. D0 means the card is fully powered, and D3cold means it is fully unpowered

All graphics are functioning correctly, and I am able to use the nvidia gpu for processes as I wish with prime-run, but my end goal is for the nvidia gpu to remain in the D3cold power saving state as much as possible, since I would like to preserve battery life on the go. The issue I am having happens when I unplug an external monitor, and the nvidia card remains in it’s highest power state (D0), but I want it to return to D3cold (lowest power state). This transition is suppsoed to happen automatically when there are no processes running on the nvidia gpu, and this does work correctly in several scenarios:

  • Booting directly into a tty. The card starts off in D0, but transitions to D3cold after a few seconds. This is acceptable.
  • Once the card is in D3cold, run nvidia-smi. This briefly brings the nvidia gpu back into the D0 power state, but it returns to D3cold after a few seconds. This is also ok, since nvidia-smi needs to wake the card to query it.
  • Launching Hyprland from the tty. The nvidia gpu stays in the D3cold state. I can run prime-run glxinfo, which will briefly activate the card, after which it returns to the D3cold state.

Interestingly enough, in the instructions on nvidia RTD3 (linked above), it says that this kind of power management is disabled by default, however I have not specified the NVreg_DynamicPowerManagement parameter anywhere like instructed yet it sill works, and if I do add it (either to my linux command line or as a .conf file in /etc/modprobe.d), this feature does not work anymore in the test scenarios above.

The problematic sequence I am trying to solve is as follows:

  • Boot into the tty. Nvidia gpu is briefly in the D0 state, but returns to D3cold after a few seconds.
  • Plug external monitor into HDMI port. Monitor turns on and functions correctly, and dgpu goes into power state D0. This is expected since the HDMI port works through the nvidia gpu.
  • Unplug external monitor. The nvidia gpu stays in the D0 state unless I reboot my computer, but I want it to return to the D3cold state to save power without having to do this.

The interesting thing about this scenario is that if I look at nvidia-smi after unplugging the monitor, I can see there are no processes running on the nvidia gpu, yet it stays powered.

If anyone has any other resources I could look at that might be relevant or any ideas as to a solution I would be glad to hear them. Thank you for reading!

Edit: formatting

1 Like
2

Multiple users report this:
565 release feedback & discussion - #154 by ievgenp (me), 565 release feedback & discussion - #41 by xplodgui and this thread maybe related 4070 / 555 and 560 drivers wont stay in D3cold, Lenovo Legion Slim 5 :(

1 Like
3

To me, the cause is [BUG] linux driver fails to remove framebuffer device when HDMI cable plugged out - #2 by Aetherall

where the nvidia-drm is either not receiving or handling hotplug events.

It seems that the linux kernel had some changes regarding framebuffers lifecycle with drm, maybe changes required to handle hotplug events and cleanup framebuffers have yet to be implemented / released

EDIT: issue opened at RTD3 dont allow gpu to sleep after a monitor has been plugged and unplugged on prime reverse sync · Issue #759 · NVIDIA/open-gpu-kernel-modules · GitHub

1 Like
4

Well I’ve been investigating a bit why this is happening, and part of it seems to be related to the fact the NVIDIA GPU somehow shows as card0/main card when the NVIDIA driver is in use.

Using the Nouveau open-source driver, /dev/dri/card0 is the Intel card (card1 being the nvidia card), while using the Nvidia driver, the /dev/dri/card0 is the Nvidia card (and card1 is the Intel card). This somehow seems to confuse Gnome/mutter/… as it then loads the desktop session on the Nvidia card, instead of the Intel card, meaning it needs the Nvidia card to stay powered on.

There have been changes recently in Mutter to do that, but I haven’t had the chance to test it yet so I’m not sure if this will affect us.

As Aetherall mentioned, there might be other things going wrong that prevents proper GPU power management.

It’s really frustrating, considering the Nouveau open-source driver works soooo much better in those regards, except that the performance is poor and stability is hit or miss. The nvidia driver is so close, I don’t know why the devs aren’t spending just a bit of time to fix that. Or, you know, Nvidia could use some of those $3 trillion to hire one more Linux developer to tackle this, it’s not like there’s not a lot of work remaining on the Linux drivers in general.

5

I think this issue is not related to the compositor/window manager/desktop environment, since I see this happening when booting to a tty and not using any kind of graphical interface. In addition, Hyprland (the WM I am using) does not have any issue using only the integrated AMD card and letting the nvidia card go into power saving mode.

I can plug in my external monitor and see a hyprland process appear on the nvidia gpu, then drop off the list when the monitor is unplugged, so it seems that is not the problem, but rather there is some kind of driver issue preventing the gpu from shutting down.