Hi, I am using the Gforce GTX and have run several DL jobs on it. Suddenly it is showing an error on the nvidia-smi, “ERR!” shown in both the fan and powerusage from the nvidia-smi. I have driver version 515.65 and running on CUDA 11.7. Any idea what’s going on?
Below is the output of nvidia-smi:
±----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01 Driver Version: 515.65.01 CUDA Version: 11.7 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA TITAN X … On | 00000000:03:00.0 Off | N/A |
|ERR! 87C P0 ERR! / 250W | 8MiB / 12288MiB | 0% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+
±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
±----------------------------------------------------------------------------+
nvidia-bug-report.sh command output:
nvidia-bug-report.log.gz (379.5 KB)