Nvidia-smi Unable to determine the device handle for GPU0000:01:00.0 Unknown Error, No device were found

Hello,
Here at our tech support we have received a call about a dell precision 3660 system with a nvidia geforce rtx 4090. Where they try to run some stuff on ubuntu bur are receiving the following errors:
Nvidia-smi Unable to determine the device handle for GPU0000
Since the system is located on a different campus i have only attempted to do some googling and fount that it could be related to a thermal issue. So first i asked whether they could modify the thermal setting when running their command and they replied that they already use certain settings, these to be precise:
echo 80 >/sys/devices/system/cpu/intel_pstate/max_perf_pct
nvidia-smi -pl 405
Also i have asked which version of the driver they use because somewhere was suggested to choose not the free version i think, this is what they are using the 520 one.
In the end when this suggestions did not help i then suggested to contact the my colleagues that are on site to run the hardware diagnostics what they did and also did update the bios but with no change.
So then i suggested to generate the logs that are asked for here:
»»»»»»»»»» If you have a problem, PLEASE read this first «««««««««« - Graphics / Linux / Linux - NVIDIA Developer Forums
So now they have added the logs in the ticket. Here are the logs.
What do you think is there something to see in the logs? Any other suggestion?
I am myself somewhat familiar with linux so theres that.
What else can we do?
nvidia-installer.log (4.3 KB)
nvidia-bug-report.log (686.0 KB)