GPU power limit at 80W in VM with gpu-passthrough

[big_miki]

Hi! I made a pretty nice setup with my laptop and only thing that i couldnt find the answer to is that my gpu is stuck at 80W when passed through to VM.

neofetch

OS: Arch Linux x86_64 
Host: 82RF Legion 5 Pro 16IAH7H 
Kernel: 6.4.7-arch1-2 
Uptime: 4 hours, 46 mins 
Packages: 1372 (pacman) 
Shell: bash 5.1.16 
Resolution: 2560x1600 
DE: Plasma 5.27.7 
WM: KWin 
Theme: [Plasma], Breeze [GTK2/3] 
Icons: [Plasma], breeze-dark [GTK2/3] 
Terminal: konsole 
CPU: 12th Gen Intel i7-12700H (20) @ 4.600GHz 
GPU: NVIDIA GeForce RTX 3060 Mobile / Max-Q 
GPU: Intel Alder Lake-P 
Memory: 6959MiB / 15719MiB 

Laptop is in performance mode.

In arch, everything works flawlessly, i installed both intel and nvidia drivers.
When i start nvidia-powerd.service nvidia card is working as expected (140W max)

I enabled all virtualization options in UEFI, setup the kvm and passed the gpu to the VM and installed looking glass.
For it to work when laptop boots up, the nvidia gpu is controlled by vfio-pci driver.

To reattach it to linux i used these commands

sudo virsh nodedev-reattach pci_0000_01_00_0
sudo rmmod vfio_pci vfio_pci_core vfio_iommu_type1
sudo modprobe -i nvidia_modeset nvidia_uvm nvidia

To detach it, for use in VM

sudo rmmod nvidia_modeset nvidia_uvm nvidia
sudo modprobe -i vfio_pci vfio_pci_core vfio_iommu_type1
sudo virsh nodedev-detach pci_0000_01_00_0

In a VM i tried installing drivers from Lenovo website, i tried latest nvidia drivers, and i tried default driver that windows installs automatically, and all of them are stuck at 80W under load.

Is it possible to achieve maximum performance without custom Vbios?
Is there something like nvidia-powerd but in windows that i can activate?

[big_miki]

I solved it “kinda”. It is not perfect but it works.

Inside the windows VM i had to download and install the nvidia driver 528.49 for notebooks.

I installed it with NVCleanstall, selected only Driver, Audio, physx, and NV platform controller (but i think it doesn’t matter because in the end Platform controller still shows up with error in device manager, so the driver can probably be installed like normal directly from installer that has been downloaded)

After installation open cmd as admin, and you can type

nvidia-smi

And you can see current power limit.

Now you can change the power limit with this command (95 is power in watts, i don’t think is a good idea to go for more than what you GPU is rated for)

nvidia-smi -pl 95

This driver version (528.49) had to be used because newer ones doesn’t allow you to change the power limit.

My gpu in my laptop can go up to 140w, so i tried it and it worked

*Couple of notes

I don’t know if this will damage your components in your laptop. So do it on your own risk.
My gpu can go up top 140w, and i don’t know what would happen if power limit is set to more than it can handle.
In nvidia control panel → system information, dynamic boost shows off, but from what i can see, is might be working because when i put the load on cpu too, GPU power drops to ~100w. Maybe dynamic boost works on a firmware level? I don’t know just guessing…

This in not the best solution because it requires this specific driver version, but i will mark it as SOLVED so others can find it useful.

If you find a more legit way of unlocking this power limit please do let me know, thanks.