Couldn't communicate with the NVIDIA driver on Ubuntu 18.04

[sebastien.lemetter]

Hello,
I have been using a NVIDIA graphic card as hardware accelerator for my deep learning experiments for a moment. It worked for months, but since today, I get the following error:

nvidia-smi
NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

I didn t change anything myself. Maybe it is the result of an automatic update. After checking some answer of this forum, secure boot is disabled and no file /lib/modprobe.d/blacklist-nvidia.conf is available on my desktop.

HW: Quadro M2000 4GB.
OS: Ubuntu 18.04
OS kernel: 4.15.0-123-generic
Nvidia driver: 418.39
Cuda version: 10.1

nvidia-bug-report.log.gz (73.8 KB)

[sebastien.lemetter]

Hi,
I just uninstalled and reinstalled the same driver, and now it works. However, it is really not satisfactory, because I still don t know why it happened, and it might happen again in the future.
It seems I have still an old config available on my machine, the driver 455. I first installed cuda 11, and probably these drivers, but Torch was not supporting it yet, so I rolled back to 418. Maybe some kind of clean up happened in my PC, and the 418 got partly errased. We can see in the screenshot that on 418 file was still there in my buggy config.