Unable to determine the device handle for GPU 0000:0A:00.0: GPU is lost. Reboot the system to recover this GPU

marc.lelarge September 16, 2020, 4:41pm 1

Doing some deep learning experiments with PyTroch 1.6 crashed my TITAN V. Anyone else having similar problems? Any suggestion to solve it?
Thanks,
Marc
The nvidia bug report is attached.
nvidia-bug-report.log.gz (237.0 KB)

Topic		Replies	Views
1 out of 4 GPUs is suddenly gone Linux	0	347	November 17, 2020
Unable to determine the device handle for GPU. GPU is lost. Reboot the system to recover this GPU GPU - Hardware cuda , cudnn	0	516	November 11, 2020
Unable to determine the device handle for GPU 0000:21:00.0: GPU is lost. Reboot the system to recover this GPU Linux	4	904	January 18, 2022
GPU is lost. Reboot the system to recover this GPU CUDA Setup and Installation	1	3998	October 1, 2019
Unable to determine the device handle for GPU 0000:05:00.0: GPU is lost. Reboot the system to recover this GPU Linux cuda , ubuntu	2	988	October 28, 2022
Titan Xp: Unable to determine the device handle Linux	0	457	February 12, 2020
Unable to determine the device handle for GPU 0000:02:00.0: GPU is lost. Reboot the system to recover this GPU Linux	3	4204	April 6, 2020
Ubuntu16.04 with 1080Ti x 4, encounters this problem once every week. <Unable to determine the device handle for GPU 0000:05:00.0: GPU is lost> Linux	0	491	August 26, 2020
Nvidia-smi Unable to determine the device handle for GPU0000:01:00.0: Unknown Error GPU - Hardware kernel , nvbugs	0	187	April 9, 2024
gpu always loss CUDA Setup and Installation	1	418	February 25, 2020

Unable to determine the device handle for GPU 0000:0A:00.0: GPU is lost. Reboot the system to recover this GPU

Related topics