Strange behaviour of low power states on Ubuntu 22 LTS

I have a 3080 Ti that I used on Ubuntu 22 desktop for quite some time but i transitioned to Ubuntu server 22. Then I became aware that my machine idles at a higher wattage than before. P-state was always P8.
I noticed on the desktop version is that if I kill the X and Gnome the power consumption goes up by more than triple it was before, which is huge! It only went from 5 W to 15-20 W but in my opinion its just wasted power.
I dismissed it as I believed I messed up something in the config but now as I have a server, and it has no window manager and gui I have the same problem. Installing x / gnome did solve the problem.
I tried all the power settings I could, and I also read the readme here:
https://download.nvidia.com/XFree86/Linux-x86_64/525.105.17/README/
I installed with apt install from the with
sudo apt install nvidia-driver-525-server"
But I could never get the power down to 5 watts as it was on desktop. Another difference that I noticed that on desktop is that connecting a monitor to the GPU increases slightly the power draw but by not much, and on server the power draw is unaffected. I tried to disable the framebuffer by adding specific parameters to disable it and also added some i found on the web to “fool” the drier into better power management but none of these settings had any effect:
GRUB_CMDLINE_LINUX_DEFAULT="fbcon=map:0 consoleblank=1 quiet acpi_osi=! acpi_osi=\"Windows 2009\""

Any other suggestions?

On headless systems, please make sure nvidia-persistenced daemon is running.

I checked and it was running, but it was set to “–no-persistence-mode”, so i added --persistence-mode. This way i no longer have to add sudo nvidia-smi -pm 1 after every boot.
However, the idle power draw didn’t change.

Please run nvidia-bug-report.sh as root and attach the resulting nvidia-bug-report.log.gz file to your post.

Note that I’ve updated cuda yesterday, which bumped the driver to 530.
nvidia-bug-report.log.gz (493.4 KB)

Some extra info:
I added modprobe nvidia "NVreg_DynamicPowerManagement=0x02"
But when i do
cat /proc/driver/nvidia/./gpus/0000:09:00.0/power
The return is that the D3 status is disabled by default(it should be Enabled (fine grained) i think)

Runtime D3 status:          Disabled by default
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Not Supported
 Video Memory Off:          Supported

Found some more info here, but this is related to GUI and laptops
https://www.reddit.com/r/archlinux/comments/ov2o86/runtime_d3_status/

EDIT: After I executed sudo update-initramfs -u the output changed to

Runtime D3 status:          Not supported
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Not Supported
 Video Memory Off:          Supported

Why is the D3 power setting not supported? Maybe its related to my cpu, but it should be possible. Maybe the driver needs xorg to interface with the proper settings or maybe way to block metacentre usage? Windows control panel won’t open/driver cannot be installed with no monitor attached. I also stumbled upon a lot of optimus related discussions where there was problem with the switch between the igpu and dgpu but I’m not sure how they are related to this problem.