NVIDIA driver has a habit of keeping my GPU at the highest performance level

After upgrading to an RTX 3080 Ti, I am also affected by this and it’s very annoying because it causes my GPU fans to be loud during desktop usage. It does seem to be related to multiple monitors and/or refresh rate. I have four 1440p monitors and tested various configurations:

RTX 3080, driver 465.24.02, 4 monitors: worked, powermizer would bounce between level 0 and 1 on idle desktop. (no longer have this card)

RTX 3080 Ti, driver 470.57.02, 4 monitors (1@144hz, 3@75hz): stays at highest level, uses 93w on idle desktop, GPU temp 45c, fans loud

RTX 3080 Ti, driver 470.57.02, 1 monitor at 144hz: works, drops to level 0 on desktop, uses 28w, GPU temp 32c, fans silent

RTX 3080 Ti, driver 470.57.02, 2 monitors at 144hz and 75hz: stuck at highest level / 93w / 45c / loud

RTX 3080 Ti, driver 470.57.02, 2 monitors at 60hz: works, drops to level 0 on desktop

RTX 3080 Ti, driver 470.57.02, 3 monitors at 60hz: back to being stuck at highest level

So it seems that somewhere around 2 1440p monitors at 60hz is about the highest I can go before powermizer stops working. The 3080 Ti should have no problem idling 4 monitors on the desktop at the lowest power level. I work at my computer all day and it’s wasting 65w + putting out more noise and heat for no reason.

Edit: If I let the computer idle until the displays turn off and then SSH into the box, the GPU is at the lowest power level and using 30w. It’s only stuck at the highest level when the monitors are on.