Power State P12 with Nvidia RTX A1000 on Unraid Server

[david.becker1]

Hello everyone.
I recently built an Unraid home server and purchased an Nvidia RTX A1000 8GB for it.
The graphics card seems to successfully switch to persistence mode, but it only reaches a maximum of P8 status. What do I need to do to achieve P12?

Here are my specs:

• MPG Z790I EDGE WIFI
• Intel i5 13500
• G.Skill DIMM 32 GB DDR5-6400 (2x 16 GB) Dual Kit
• be quiet! SFX Power 3 (450 W)
• 2x WD BLACK SN770 NVMe SSD
• 2x Toshiba MG09 18 TB
• Nvidia RTX A1000 8GB
• Driver version: 560.35.03
• Unraid system: 7.0.0-beta.3

Thank you in advance!

[shelter]

I have never seen a card use P12, lowest I’ve seen is P8, but maybe the A1000 is different.

Maybe this post might help you understand what’s going on.

GPU Performance

run the command with: nvidia-smi -a and check the “Clocks Throttle Reasons” line to see if any reasons are reported (although I think there won’t be). gromacs is not putting the card under much load (this is evident by the fact that the power only goes from 80W to 90W under load, and the GPU temp is the same at 46C) so it’s possible that the clock management system is alternating between P0 and P2 states as the gromacs load dynamically varies. It may be difficult to monitor this using nvidia…

[david.becker1]

Thanks, but since Nvidia itself claims, that the power stages 10 & 12 exist, i expect, that there are gpu´s out there, which can reach these stages.

This are the official list of stages i found:
P0/P1 - Maximum 3D performance
P2/P3 - Balanced 3D performance-power
P8 - Basic HD video playback
P10 - DVD playback
P12 - Minimum idle power consumption

[rs277]

Perhaps try disabling Persistence. My antiquated card is idling at P12 with persistence off:

±-----------------------------------------------------+
| NVIDIA-SMI 352.39 Driver Version: 340.104 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Quadro FX 1800 Off | 0000:03:00.0 N/A | N/A |
| 30% 40C P12 N/A / N/A | 48MiB / 767MiB | N/A Default |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 Not Supported |
±----------------------------------------------------------------------------+

[shelter]

Yeah, persistent seems to make the card go into a higher p-state, at least that’s how i understood it from the post I linked. I think… ?

Besides, your card got a 50W cap so it’s not going to use more than 50W.
Tbh I think P8 is the lowest your card can do.

[david.becker1]

Thanks.
I tried that, but it seems like there is no difference. The GPU goes normally from P0 to P8 after disabling persistance mode…

[david.becker1]

Hi there.
So there is really no way to get this GPU into P12?
It is hard to believe since this should be a power efficient GPU.
Probably this needs to be fixed within the gpu driver.

[shelter]

I think the Nvidia documentation is slightly misleading, they might have removed P-states lower than P8 on newer cards?

Your card doesn’t seem to report how much wattage it uses in P8 so it’s obviously impossible to know how much it draws.

My RTX 4070 reports it uses 12W in P8 state.

[BlueGoliath]

You’re starring into the abyss. GPUs can have whatever PStates it was programmed to have. That PState level meaning is old and irrelevant for modern GPUs.

What matters is whether GPU/Memory clocks are at the lowest frequency and your PCIe generation is at 1.

[david.becker1]

In Unraid the PCIe Generation switches to 1 in idle. If the GPU is in use, it goes up to Gen 4.

But seems like it was a bad idea to buy this GPU for my Unraid Server-System, since

• it prevents my system to reach higher c-states than C3
• you can not connect a monitor cable with the GPU, because it increases the energy drain permanantly by 10-15 Watt
• it is not able by design to reach the lowest power state P12 within Unraid

I did not expect, that it would double my energy drain in idle.

[adolfotregosa]

Hi. I’m curious. In my limited testing with some more modern nvidia cards, they all block the cpu package to C3 in the pcie connected to the cpu or chipset. What route did you choose to “fix” this ?