Has anyone been able to run an RTX 3060 laptop GPU at more than 80W on Linux?

Not rebooting, only restarting the Xserver.

Tried restarting Xorg, didn’t help. But I tested a bit more with and without nvdia-powerd running. 90W is really only available when powerd is running, so it seems it actually works. But when daemon is running it wakes nvidia GPU often without a good reason (for example, I launched browser, GPU wakes up though browser doesn’t need it). Eventually it goes back to D3Cold but it takes some time. When nvidia-powerd is not running, GPU wakes up only if some application really renders on it (i.e. render offload variables are set accordingly).

Sounds like you’re in offload mode, does switching to nvidia as primary gpu (aka performance mode) make this available?

When nvidia is a primary GPU benchmark results are the same within the margin of error. nvidia-powerd allows 90W just like in offload mode. So no reason to use nvidia as a primary GPU, offload mode gives same results. The only thing I don’t like is that nvidia-powerd wakes GPU without obvious reasons, I hope it will be fixed in the future. But at least power boost works.

Hi again, I need some help understanding how nvidia-powerd works. I have a script that runs applications in offload mode:


export VDPAU_DRIVER=nvidia
export __NV_PRIME_RENDER_OFFLOAD=1
export __GLX_VENDOR_LIBRARY_NAME=nvidia
export __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0
export __VK_LAYER_NV_optimus=NVIDIA_only
#export SDL_VIDEO_X11_VISUALID=

cmd=$(printf "%q" "$1")

for (( i=2; i<=$#; ++i ))
do
	args_array[$(( i-2 ))]=$(printf "%q" "${!i}")
done

sh -c "$cmd ${args_array[*]}"

I decided to add start and stop commands for nvidia-powerd so it’s only started when needed, so I added commands before running application in this script:

sudo systemctl start nvidia-powerd
sh -c "$cmd ${args_array[*]}"
sudo systemctl stop nvidia-powerd

Daemon reporting that it’s started successfully, systemctl status indicates it is running, but power boost does not work (80W only). For some reason daemon only does it’s job if started somewhere else but not from this very script. I even tried to add sleep 1 after start, but it did not help. Even simply restarting daemon from another terminal (sudo systemctl restart nvidia-powerd) while application is already running boosts power to 90W. So why daemon does not want to boost power when started from script, what difference does it make?

No idea.
Something different: the doc and stuff only talk about “DynamicBoost” which is AFAIK only the 10-15 extra Watts that the gpu is allowed to take while the general Power limit is set by the profile. The nvidia-settings GUI should have a drop-down for this, can you find it or is it hidden?

There are powermizer settings there, I tried to set ‘prefer maximum performance’ instead of ‘auto’, makes no difference. Other than that there are no power settings on nvidia-settings window.

Greetings!

I have been able to confirm this issue on my laptop as well.
The computer in question is a Lenovo Legion 5 Pro 16ACH6H (AMD Ryzen 7 5800H with Radeon Graphics, NVIDIA RTX 3060 Mobile).

I am using the latest driver available in my linux package manager, and the latest Nvidia Studio driver on Windows.

the GPU does report a power usage way above 110W under gaming load, using HWInfo64 as the monitoring tool.

Running the same game under Linux (through Wine/Proton, via Lutris) shows a 100% GPU utilization and a maximum measured power usage of 80W.

The nvidia-powerd service fails to load for me as well, reporting the same error: Failed to read the data for calculating CPU power.

I have tried to use the zenpower3 driver, with no success. the service still fails to detect power usage.

Let me know if I can provide any missing information to help fix this issue.

nvidia-settings -q DynamicBoostSupport

  Attribute 'DynamicBoostSupport' (legion5p:0[gpu:0]): 1.
    'DynamicBoostSupport' is a boolean attribute; valid values are: 1 (on/true) and 0 (off/false).
    'DynamicBoostSupport' is a read-only attribute.
    'DynamicBoostSupport' can use the following target types: GPU.

A small update on the matter:

TDP Boost should be working on the latest driver and Linux Kernel for Intel CPU owners, but it is still broken on AMD cpu laptops.

Using Final Fantasy XIV Online’s main menu as a metric, and using MangoHUD to monitor the hardware usage, there is a lot of performance left to be had by unlocking the TDP as it can do on Windows and Intel CPUs:

I am using the zenpower3 DKMS module to obtain the CPU wattage monitoring, and most applications are able to confirm that the driver is indeed working, and some measurement is taking place.

However, the NVIDIA user-space utility nvidia-powerd does not appear to be looking in the right place, as it continues reporting that monitoring is unavailable, and therefore will not unlock the TDP:

legion5p ~ » sudo systemctl restart nvidia-powerd

Job for nvidia-powerd.service failed because the control process exited with error code.
See "systemctl status nvidia-powerd.service" and "journalctl -xeu nvidia-powerd.service" for details.
legion5p ~ » systemctl status nvidia-powerd
                   1 ↵
× nvidia-powerd.service - nvidia-powerd service
     Loaded: loaded (/usr/lib/systemd/system/nvidia-powerd.service; disabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Thu 2022-05-26 18:33:41 EDT; 1s ago
    Process: 15621 ExecStart=/usr/bin/nvidia-powerd (code=exited, status=1/FAILURE)
   Main PID: 15621 (code=exited, status=1/FAILURE)
        CPU: 5ms

May 26 18:33:41 legion5p systemd[1]: Starting nvidia-powerd service...
May 26 18:33:41 legion5p /usr/bin/nvidia-powerd[15621]: nvidia-powerd version:1.0(build 1)
May 26 18:33:41 legion5p /usr/bin/nvidia-powerd[15621]: Failed to read the data for calculating CPU power
May 26 18:33:41 legion5p /usr/bin/nvidia-powerd[15621]: Failed to initialize GPU Boost controller
May 26 18:33:41 legion5p systemd[1]: nvidia-powerd.service: Main process exited, code=exited, status=1/FAILURE
May 26 18:33:41 legion5p systemd[1]: nvidia-powerd.service: Failed with result 'exit-code'.
May 26 18:33:41 legion5p systemd[1]: Failed to start nvidia-powerd service.

I hope we can see this issue resolved soon as using NVIDIA graphics card on Linux outside the datacenter space is currently a big downgrade due to being capped at half the power usage it is capable of using. (this is a laptop with a 300W power adapter).

1 Like

The same thing happened on my 5900h+3080 notebook, which can only reach 115W at most,

cat /proc/driver/nvidia/gpus/0000:01:00.0/power

Runtime D3 status:          Enabled (fine-grained)
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Supported
 Video Memory Off:          Supported

Power Limits:
 Default:                   115000 milliwatts
 GPU Boost:                 4294967295 milliwatts

cat /proc/driver/nvidia/gpus/0000:01:00.0/information

Model: 		 NVIDIA GeForce RTX 3080 Laptop GPU
IRQ:   		 104
GPU UUID: 	 GPU-aeb2e7e5-50cb-6519-b948-aaf554397d01
Video BIOS: 	 94.04.3e.00.99
Bus Type: 	 PCIe
DMA Size: 	 47 bits
DMA Mask: 	 0x7fffffffffff
Bus Location: 	 0000:01:00.0
Device Minor: 	 0
GPU Excluded:	 No

And i got error from $ sudo systemctl start nvidia-powerd.service

× nvidia-powerd.service - nvidia-powerd service
     Loaded: loaded (/lib/systemd/system/nvidia-powerd.service; disabled; vendo>
     Active: failed (Result: exit-code) since Sun 2022-06-05 12:25:32 CST; 6s a>
    Process: 5300 ExecStart=/usr/bin/nvidia-powerd (code=exited, status=1/FAILU>
   Main PID: 5300 (code=exited, status=1/FAILURE)
        CPU: 8ms

Jun 05 12:25:32 jl systemd[1]: Starting nvidia-powerd service...
Jun 05 12:25:32 jl /usr/bin/nvidia-powerd[5300]: nvidia-powerd version:1.0(buil>
Jun 05 12:25:32 jl /usr/bin/nvidia-powerd[5300]: Failed to read the data for ca>
Jun 05 12:25:32 jl /usr/bin/nvidia-powerd[5300]: Failed to initialize GPU Boost>
Jun 05 12:25:32 jl systemd[1]: nvidia-powerd.service: Main process exited, code>
Jun 05 12:25:32 jl systemd[1]: nvidia-powerd.service: Failed with result 'exit->
Jun 05 12:25:32 jl systemd[1]: Failed to start nvidia-powerd service.
lines 1-14/14 (END)...skipping...
× nvidia-powerd.service - nvidia-powerd service
     Loaded: loaded (/lib/systemd/system/nvidia-powerd.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sun 2022-06-05 12:25:32 CST; 6s ago
    Process: 5300 ExecStart=/usr/bin/nvidia-powerd (code=exited, status=1/FAILURE)
   Main PID: 5300 (code=exited, status=1/FAILURE)
        CPU: 8ms

Jun 05 12:25:32 jl systemd[1]: Starting nvidia-powerd service...
Jun 05 12:25:32 jl /usr/bin/nvidia-powerd[5300]: nvidia-powerd version:1.0(build 1)
Jun 05 12:25:32 jl /usr/bin/nvidia-powerd[5300]: Failed to read the data for calculating CPU power
Jun 05 12:25:32 jl /usr/bin/nvidia-powerd[5300]: Failed to initialize GPU Boost controller
Jun 05 12:25:32 jl systemd[1]: nvidia-powerd.service: Main process exited, code=exited, status=1/FAILURE
Jun 05 12:25:32 jl systemd[1]: nvidia-powerd.service: Failed with result 'exit-code'.
Jun 05 12:25:32 jl systemd[1]: Failed to start nvidia-powerd service.

165W power can be achieved under window test. I prefer the power up to 160W for deep learning, rather than the 115W with weak performance.

AMD Zen3 Cpu Power Consumption monitoring is added to kernel 5.19 please someone warn nvidia developers to adapt nvidia powerd to work with that driver. I can see that option in menuconfig AMD Processor Power Reporting Mechanism. I think ive overexcited and make a mistake on understanding the new mechanism its just for family 15h processors and doesnt support 5800H and 5900HX CPUs.

Throwing my hat into the ring too, I am seeing all the same problems as many others in this thread on my differently spec’d laptop.

ROG Zephyrus M16 2022 (i9 12900H, 3080Ti Mobile)
NVIDIA Driver 510.73.05 installed via ubuntu-drivers
Ubuntu 22.04 LTS 5.15 kernel / Also tried on mainline 5.18 Kernel

As stated by Asus here this laptop should be running with a 120w power budget. Instead I’m seeing a limited 80w. I don’t have windows to test, but I am confident that it would run at 120w based on others findings.

As I’ve got an Intel CPU, I had a go at running the nvidia-powerd.service extracted from the runfile for my drivers and I was able to get a 10w increase in the power budget, but that has had little to no impact on my performance in CUDA workloads.

When checking the power limits in the driver’s system files, I get much the same as others.

➜ cat /proc/driver/nvidia/gpus/0000:01:00.0/power      
Runtime D3 status:          Enabled (fine-grained)
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Supported
 Video Memory Off:          Supported

Power Limits:
 Default:                   80000 milliwatts
 GPU Boost:                 80000 milliwatts

And the following on the GPU information file…

➜ cat /proc/driver/nvidia/gpus/0000:01:00.0/information 
Model: 		NVIDIA GeForce RTX 3080 Ti Laptop GPU
IRQ:   		225
GPU UUID: 	GPU-3ba4078f-8640-0378-c143-65114149ce58
Video BIOS: 	94.03.17.00.0a
Bus Type: 	PCIe
DMA Size: 	47 bits
DMA Mask: 	0x7fffffffffff
Bus Location: 	0000:01:00.0
Device Minor: 	0
GPU Excluded:	No

I’ve tried a few combinations of kernels (LTS 5.15 and mainline 5.18) to no avail as well as prime-select nvidia and prime-select on-demand but as per usual, 80w limit remains.

My use-cases are slightly different to others here as I mainly use the GPU for deep-learning inference which is a much more unbalanced load (lower CPU usage, 100% GPU usage) compared to gaming which would affect the expected boost behavior. I would expect with this laptop to get at least close to Jetson Xavier levels of performance, but I’m around 2x slower than the 3080Ti mobile probably should be.

However, I have also confirmed in 3d applications (superposition) that the GPU never goes above 80w or 90w with nvidia-powerd running.

Just wanted to provide a perspective with a different GPU/CPU configuration, and that it doesn’t seem that nvidia-powerd is a magic bullet that fixes this issue, and that there is something probably more serious going on here that NVIDIA will need to fix at a driver level. It feels like all our GPU’s, no matter which model or power limit VBIOS we all have, the GPU’s never exit their defined lowest power state.

I do hope this is something NVIDIA is looking into. Seeing others mention of ‘quiet, balanced and performance’ options in nvidia-settings (and seeing it in the code on GitHub) is promising, but those settings haven’t appeared for me on my machine as of yet…

Was there any update related to this issue?
I have recently tried a Lenovo Legion 5 with a RTX 3070 on Ubuntu and I couldn’t get the gpu to run at more than 40W (AC plugged).
I tried the latest nvidia drivers and found the same issue as you.

I’m in the boat now. Please do your math, it’s a bad intel joke. Put shortly, I have a power budget of 150W, the intel cpu (with a TDP of 35W, haha) takes 80W constant, 100W on boost. Leaving 50W for the nvidia.
Windows standard install uses 50W on idle, vendor specific software puts it down to 13W if set to “powersave” profile.
Depending on profile, the nvidia gpu can be switched to 50,60 or (70W+rare15W boost).
Obviously, the intel cpu has to be constricted in its power usage first before the nvidia gpu’s power can be raised.
I guess that’s the reason the options are disabled.

Only amd does not support it, it is already available on intel laptop

https://download.nvidia.com/XFree86/Linux-x86_64/515.57/README/dynamicboost.html

Hardware Requirements
Dynamic Boost is supported only on platforms that meet all of the following requirements:

Notebook form factor.

Ampere or newer GPUs.

Intel CometLake or newer Intel chipsets.

SBIOS support for Dynamic Boost

Run the following command to check if the SBIOS supports Dynamic Boost:

nvidia-settings -q DynamicBoostSupport

We are actively working upon it and have a bug 3674976 filed internally for tracking purpose.

2 Likes

Nvidia-powerd is only for dynamic boost, that’s the occasional 15W extra. The switch between the base power limit of (in my case) 50,60 or 70W is a “platform profile”.
You can check the nvidia-settings code, there’s an extra pane for it that’s hidden. As you can see from my measurements, this can’t be enabled per default since it has to be made sure the intel cpu is restricted in its power usage first. Unless you want to blow up your power regulators.

So any guesstimates about when we are be able to use our gpus at full power on linux with an amd cpu and nvidia gpu? i remember optimus is implemented on nvidia drivers took like 3 years while implementing that there were newer technologies to adapt on linux. Im waiting for a year already.

Hi, I hope you are doing Ok.
I was wondering if you known if there’s been a new development regarding been able to use Dynamic Boost Feature with Nvidia 3XXX cards and AMD Cpus. I’m kinda been unable to make it work at full TDP. Thanks a lot. Have a great Day.

Is there any update about the power limit? I have a Lenovo Legion 5 15ACH6H AMD Ryzen 5 5800H RTX 3060 and I am still not able to use my GPU beyond 80W.