UPDATE: I can now drop to the lowest power level by changing my 3 aux monitors from 75hz to 60hz. So now:
1440p@170hz + three 1440p@75hz → stuck at highest power, idles at 90w
1440p@170hz + three 1440p@60hz → works, idles at 34w
If I bump just one of the 60hz monitors up to 75hz, then I get stuck at the middle power level using 66w
This is an improvement because on previous drivers, dropping the aux monitors to 60hz did not help and now it does. I would still like to be able to run 75hz on the aux monitors, but for now I can live with 60hz since they are not the primary display.
@amrits
Removing the UseNvKmsCompositionPipeline “false” line did not help. All it did was make my GPU Utilization jump to 10% while doing nothing. I’m still stuck at P0 using 90w. When I add the line back, GPU Util drops to 2%.
I also tried with UseNvKmsCompositionPipeline “true” and ForceFullCompositionPipeline turned on for all monitors. This brought my GPU Util up to 18% while idle.
I have spent hours trying out various settings and turning things on/off, compositing on/off, etc, and the only way I have been able to reach lower power levels is by disconnecting monitors and/or dropping resolution/refresh. It almost feels like when the total pixel clock on all the outputs reaches some threshold, it forces the highest power level.
Fri Sep 23 09:31:03 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.76 Driver Version: 515.76 CUDA Version: 11.7 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:01:00.0 On | N/A |
| 51% 45C P0 91W / 390W | 929MiB / 12288MiB | 11% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 100199 G /usr/lib/Xorg 543MiB |
| 0 N/A N/A 100354 G /usr/bin/kwin_x11 158MiB |
| 0 N/A N/A 100371 G /usr/bin/plasmashell 157MiB |
+-----------------------------------------------------------------------------+
515.76 is much improved for me. I have 2 27" 4K@60 Hz monitors connected via DP to an RTX 3090.
With 515.61.01 and earlier, it’s stuck in P0 with both monitors at 60 Hz and using over 100 W at idle. Setting one at 59.94 Hz and the other at 60 Hz would drop it to P5 in this configuration and draw about 57 W at idle.
With 515.76, the GPU drops to P8 at idle regardless of the refresh rate. Idle power draw is from 37 W to 48 W with both at 60 Hz. GPU-Util is 0-1%.
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.76 Driver Version: 515.76 CUDA Version: 11.7 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | N/A |
| 0% 58C P8 37W / 420W | 1764MiB / 24576MiB | 1% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 2713 G /usr/lib/xorg/Xorg 961MiB |
| 0 N/A N/A 3223 G /usr/bin/kwin_x11 176MiB |
| 0 N/A N/A 3255 G /usr/bin/plasmashell 119MiB |
| 0 N/A N/A 3477 G ...AAAAAAAAA= --shared-files 96MiB |
| 0 N/A N/A 3990 G ...779513106121182875,131072 401MiB |
+-----------------------------------------------------------------------------+
As one of the first people to report on this, I was ecstatic when I heard the newest driver contained a fix.
Alas, it literally changed nothing. To reiterated my setup, I have an RTX 3090 that I’ve had since like 930 AM on launch day (camped at Micro Center), and I have two identical model monitors. Both are Gigabyte G32QCs, 2560x1440 @ 165Hz (technically xrandr reports both run at 164.80Hz but still).
After installing the new driver and rebooting, power draw is exactly the same, and the source of the problem - the Memory clock being pegged at its highest available clock speed - is still present.
That’s what I don’t understand. We know the problem, why the power draw is so high. It’s because the memory clock gets pegged at its max frequency. And yet it’s still not fixed.
I’ll try the latest Vulkan Beta driver and see if anything changes, but I doubt it.
If I lower one of the monitors to 60Hz, the problem goes away. But that’s preposterous to expect people to run their gaming monitors at basically 1/3 of their refresh rate because of a known bug in the driver.
Same, it’s only partially fixed. I can hit lowest power level now but have to lower all my aux monitors to 60hz. Seems like they need to take whatever they changed for 515.76 and adjust it a little more.
You might not have to drop one monitor all the way to 60hz, you might be able to do something like 165 + 144, 165 + 120, 144 + 144, etc.
Unfortunately this bug still occurs if the card begins GPU decoding of any video.
Nvidia-SMI output while decoding a single 1080p x264 video:
Fri Oct 7 18:56:49 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.76 Driver Version: 515.76 CUDA Version: 11.7 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | Off |
| 31% 60C P2 111W / 450W | 1418MiB / 24564MiB | 7% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1405 G /usr/libexec/Xorg 210MiB |
| 0 N/A N/A 1511 G /usr/bin/kwin_x11 93MiB |
| 0 N/A N/A 1579 G /usr/bin/plasmashell 112MiB |
| 0 N/A N/A 1937 G /usr/bin/krunner 17MiB |
| 0 N/A N/A 2033 C+G mpv 538MiB |
| 0 N/A N/A 2106 G ...817190634816839652,131072 384MiB |
+-----------------------------------------------------------------------------+
When decoding a video with nvenc, it sets the core clock to 1920mhz and the memory clock to 1025 on my 3090ti with just 7% gpu usage while drawing 111W. I don’t get this same operation under windows.
It appears that the issue is still in 520.56.06 (using a 4090)
$ xrandr -q
Screen 0: minimum 8 x 8, current 7680 x 1440, maximum 32767 x 32767
HDMI-0 disconnected (normal left inverted right x axis y axis)
DP-0 connected 2560x1440+0+0 (normal left inverted right x axis y axis) 598mm x 336mm
2560x1440 59.95 + 144.00* 120.00 99.95 84.98 23.97
1024x768 60.00
800x600 60.32
640x480 59.94
DP-1 disconnected (normal left inverted right x axis y axis)
DP-2 connected primary 2560x1440+2560+0 (normal left inverted right x axis y axis) 598mm x 336mm
2560x1440 59.95 + 144.00* 120.00 99.95 84.98 23.97
DP-3 disconnected (normal left inverted right x axis y axis)
DP-4 connected 2560x1440+5120+0 (normal left inverted right x axis y axis) 598mm x 336mm
2560x1440 59.95 + 144.00* 120.00 99.95 84.98 23.97
1024x768 60.00
800x600 60.32
640x480 59.94
DP-5 disconnected (normal left inverted right x axis y axis)
I have 3 screens running at 2560x1440 144hz, and the interesting output from nvidia-smi -q looks like this:
Power Readings
Power Management : Supported
Power Draw : 56.30 W
Power Limit : 450.00 W
Default Power Limit : 450.00 W
Enforced Power Limit : 450.00 W
Min Power Limit : 10.00 W
Max Power Limit : 450.00 W
Clocks
Graphics : 240 MHz
SM : 240 MHz
Memory : 10501 MHz
Video : 1530 MHz
When one of the screens is at 60hz, the memory clock speed look more sane for an idle state.
Power Readings
Power Management : Supported
Power Draw : 38.13 W
Power Limit : 450.00 W
Default Power Limit : 450.00 W
Enforced Power Limit : 450.00 W
Min Power Limit : 10.00 W
Max Power Limit : 450.00 W
Clocks
Graphics : 210 MHz
SM : 210 MHz
Memory : 810 MHz
Video : 1545 MHz
Can you please confirm if you have GSYNC/Adaptive sync enabled monitors when you see high power Draw and the behavior on windows if you disable gsync via nvidia control panel. Do you see high power draw then.
However, its not like it matters. @amrits I’m certain you already know that no one here can actually use GSYNC while having more than one monitor connected. It’s flat-out impossible for everyone on X11, and only AMD on Sway (and I believe now Plasma) can use any VRR on Wayland, including while using a multi-monitor setup.
So even if GSYNC is “enabled,” every single monitor @0115 has connected is running at the exact same refresh rate - specifically, whichever refresh rate his slowest monitor is running at (unless you use Plasma and force KWin to run at a specific refresh rate, in which case you can “cheat” around this X11 limitation, but the monitors will still all be running at fixed refresh rates).
And I didn’t even have to install Windows on bare metal to prove this issue is limited to Linux. My single-GPU-passthrough VFIO VM that I had set up with my 3090 never once showed the behavior when the Windows VM was running and taking over the GPU. At idle, power draw was in the 35-45W range and I would stay at extremely low memory clocks. Same exact machine after the VM shuts off has always run at the maximum memory clock 100% of the time unless I drop one of my 1440p 165Hz monitors to 60Hz, at which point the clock speeds and power draw act normal.
And in Windows, you actually CAN use GSYNC while having more than one monitor, so I even actually had GSYNC running on both monitors, still no insane power draw (and that didn’t change with GSYNC disabled either).
This has likely been a bug for years and years, but I personally have had it from the morning the 3090 launched over two years ago (got mine in person on launch day) and there has never been a moment I’ve been able to run both monitors at 165Hz in Linux and not have insane power draw.
Its so bad that it makes a rather large difference in idle temperatures/fan speeds. Especially with a 3090 which has half its memory on the back of the PCB. In Windows at idle my fans would only have to run at 15-20% and I’d be in the 30-35° range. On Linux the fans have to stay at 35-40% minimum just to stay below 45°.
My setup is 3 monitors, 1x60hz and 2x144hz. Changing one of them to 120hz fixed it for me, but that was before I enabled HDR and now it’s constantly using performance level 0 pulling 105W at idle.
I don’t have high hopes that it would help to switch g-sync off, but I can’t even do it from the Linux Nvidia X control panel. I disabled G-sync in there but one of the monitors is still stuck with G-sync enabled. Maybe I’ll reboot.
Sadly I would choose the extra power draw over having to drop any of these 3 monitors down from 120hz. Because once that happens NONE of the applications on ANY monitor will be able to refresh at 120.
Nvidia, I am in the market for a 3090 on one machine and a 4090 on another. This bug is preventing me from buying either today. Can you please confirm the issue with power draw when using multiple monitors has been fixed on linux?
You charge a premium for these products so I expect a premium experience without additional energy costs.
@amrits I have a RTX 3070Ti on 525.78.01. I can confirm using two identical monitors both at 165hz with GSYNC enabled shows high power draw. The performance level is pegged at P0. If I drop both of them to 144hz with GSYNC still enabled, the performance level gets back to P5. If one of them is 165hz and the other is 144hz, the performance level at idle is P8.
However, if I disable GSYNC from both monitors, they can idle at P8 even with both of them at 165hz.
If anything, it looks like they made it worse.
My 3090 sits at 120W idle, 1860 MHZ GPU clock and 9751 MHz Memory clock. Even if I disconnected every display other than my main 4K 120 LG C1, even if I enable/disable HDR and G-Sync.
Have the latest driver available today.
Watching videos with RTX Video Enhancement on level 2 means I’m using up 150W JUST on the GPU almost at all times, which is quite ridiculous.
