I’ve completed my tests (quite a while back) regarding this issue and I seem to observe a pretty simple pattern.
PCI PM state changes doesn’t seem to occur instantly after it is requested meaning a state change from D0 → D3cold or vice versa may take a few seconds.
During that time, any state changes requested will result to “can’t change power state from D3cold to D0”. This could be that the PCI device cannot stop midway on the transition.
In my system (a laptop), I have two ways to trigger this error:
First way is to trigger a transition by writing ‘auto’ or ‘on’ to the PCI device’s PM depending on which is already set. Then, try loading the module.
For example in my system the result of
cat /sys/bus/pci/devices/0000:01:00.0/power/control is on so I do a
echo auto > /sys/bus/pci/devices/0000:01:00.0/power/control.
After which, I try to load the nvidia module then the system freezes which that error.
The second way is using the fact that a udev rule sets the PCI device’s PM to “on” when unloaded. You just have to unload then reload the module.
The udev rule is from PCI-Express Runtime D3 (RTD3) Power Management in the Automated Setup section.
For example. After making sure that the nvidia module is loaded, I wait for about 2 minutes then run modprobe -r nvidia; modprobe nvidia. This also freezes the system with that error.
So my conclusion? I think the foolproof way here is to never set the PCI device’s runtime PM control to ‘auto’ during or on boot up and only set it after the system has initialized. Or, to manually load the module after a successful boot or pretty much delay loading it as much as it can be delayed.