NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver

GPU Unix Graphics Linux
#1

I previously was using CUDA 10 and NVIDIA-410 for running 2080 ti. Everything was good.

Yesterday, when I started to run a experiment, it showed the problem that cuda was failed. After I type “nvidia-smi”, the error message came out:
“NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.”

Then I follow the steps to reinstall the cuda and the driver, but it does not work. I tried almost all the ways I can find online. None of them works.

My nvidia-bug-report is listed below:

https://drive.google.com/file/d/1WblwvXlK9G937hfD-gPvoblJKkrJv7FH/view?usp=sharing

Thank you very much!

#2

BTW, my OS is Ubuntu 16.04.

#3

There doesn’t seem to be a kernel driver installed. Your (stock 16.04) kernel was compiled with gcc 5.4.0 yet your current compiler is gcc 4.8. Please use update-alternatives to switch back to the standard gcc. Then use the graphics ppa to install the current 430 driver. If necessary, purge anything nvidia/cuda beforehand and uninstall any nvidia .run installer using the --uninstall option.
Afterwards, create a new nvidia-bug-report.log.

#4

Thanks. It has been solved following this method.