Nvidia-smi failed after reboot with driver 515.48.07 on Ubuntu 20.04

Graphics / Linux Linux
#1

I have a P2000 GPU and trying to install driver on Dell precision 3650 tower.
We were using Ubuntu 16.04 with Nvidia driver 430.26. Everything was fine.
Now, we have trouble upgrading the system to Ubuntu 20.04 with latest hwe kernel 5.13.0-51.

Step to install driver:
Download run file for 515.48.07
sudo apt install build-essential libglvnd-dev pkg-config
sudo reboot (Install will fail if I don’t reboot here, will try to add log when I run into it again)
sudo rmmod nouveau
sudo bash NVIDIA-Linux-x86_64-515.48.07.run

nvidia-smi works after installation. But it stops working after reboot.
Giving error “NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.”

nvidia-bug-report.log.gz (64.4 KB)

#2

After reading some other posts. I excluded some common issue.

  • I don’t have prime-select installed
  • grep nvidia /etc/modprobe.d/* /lib/modprobe.d/* only returns blacklist nvidiafb
  • sudo modprobe nvidia gives error modprobe: ERROR: could not insert 'nvidia': No such device
#3

I cannot install with driver now. Is there some wrong with kernel parameter? Can’t figure out what went wrong in the bug report.
nvidia-bug-report.log.gz (53.7 KB)

#4 
[  135.056899] nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
[  135.056916] NVRM: The NVIDIA GPU 0000:01:00.0
               NVRM: (PCI ID: 10de:1430) installed in this system has
               NVRM: fallen off the bus and is not responding to commands.

Please check for a bios update first. If that doesn’t help, please check if downgrading to kernel 5.4 works.
sudo apt install --install-recommends linux-generic
https://wiki.ubuntu.com/Kernel/LTSEnablementStack