Dual 3090 crashes randomly

I have a dual 3090 PC with 1600W PSU.

It recently randomly crashed when I ran some deep-learning code based on PyTorch.

It shows "
Unable to determine the device handle for GPU 0000:01:00.0: Unknown Error" on nvidia-smi page.

I have searched for similar issues, and tried to update my driver.

Here is the log file.
nvidia-bug-report.log.gz (274.8 KB)

The gpu 0000:01:00.0 is always shutting down. Please check/change its power connectors.

Thanks,I will have a try now!