NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Given that I already have nvidia-prime 8.16; Do you think that there might still a bug in that package? Maybe there is some conditions on my system that the fix did not account for?

Please post the output of
grep 10de /lib/udev/rules.d/*
so I can see the filenames.

I deleted the file I found under /lib/udev/rules.d that contained a reference to 10de (nvidia) as per instruction. It was never recreated, so a grep 10de /lib/udev/rules.d/* does not return any matches.

Anything in
/etc/udev/rules.d/
?

There’s 3 .rules files in /etc/udev/rules.d/ - but none that contains “10de”. The rules seems to be for chrome and snap.

/etc/udev/rules.d/* :

-rw-r--r-- 1 root root  7904 mars   4 08:10 70-snap.chromium.rules
-rw-r--r-- 1 root root 58549 okt.  22 16:18 70-snap.snapd.rules
-rw-r--r-- 1 root root  2272 mars   4 08:10 70-snap.snap-store.rules

Update:
lsinitramfs /boot/initrd.img-$(uname -r) |grep .rules:

usr/lib/udev/rules.d
usr/lib/udev/rules.d/50-firmware.rules
usr/lib/udev/rules.d/50-udev-default.rules
usr/lib/udev/rules.d/55-dm.rules
usr/lib/udev/rules.d/60-block.rules
usr/lib/udev/rules.d/60-persistent-storage-dm.rules
usr/lib/udev/rules.d/60-persistent-storage.rules
usr/lib/udev/rules.d/61-persistent-storage-android.rules
usr/lib/udev/rules.d/70-snap.chromium.rules
usr/lib/udev/rules.d/70-snap.snap-store.rules
usr/lib/udev/rules.d/70-snap.snapd.rules
usr/lib/udev/rules.d/71-seat.rules
usr/lib/udev/rules.d/73-special-net-names.rules
usr/lib/udev/rules.d/75-net-description.rules
usr/lib/udev/rules.d/80-drivers.rules
usr/lib/udev/rules.d/80-net-setup-link.rules
usr/lib/udev/rules.d/95-dm-notify.rules

lsinitramfs /boot/initrd.img-$(uname -r) |grep nvidia:

usr/lib/modprobe.d/blacklist-nvidia.conf
usr/lib/modprobe.d/nvidia-graphics-drivers.conf
usr/lib/modprobe.d/nvidia-kms.conf
usr/lib/modprobe.d/nvidia-runtimepm.conf
usr/lib/modules/5.8.0-44-generic/kernel/drivers/i2c/busses/i2c-nvidia-gpu.ko
usr/lib/modules/5.8.0-44-generic/kernel/drivers/net/ethernet/nvidia
usr/lib/modules/5.8.0-44-generic/kernel/drivers/net/ethernet/nvidia/forcedeth.ko

I have also added the full result of lsinitramfs as initrd.txt (120.2 KB)

Then check if it’s still in the initrd

lsinitramfs /boot/initrd.img-$(uname -r)

Hey all, I’m having a very similar problem and have tried most of the items in this thread. I’d love a second set of eyes on my machine. Was working just fine with a solid install of nvidia’s 440 driver. We rebooted the system and started to receive this message. Have reinstalled clean the nvidia drivers (now running 460), cuda software several times, and tried out a new kernel even. Please find attached our nvidia-bug-report.log.gz and hopefully help spot what I’ve missed. Thanks in advance for any suggestions. Hopefully the below helps to answer some questions!
nvidia-bug-report.log.gz (1.2 MB)

wradmin@emboleye:~/Desktop$ lspci -v | grep VGA
0000:04:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P620] (rev a1) (prog-if 00 [VGA controller])
0000:b3:00.0 VGA compatible controller: NVIDIA Corporation Device 1e04 (rev a1) (prog-if 00 [VGA controller])

wradmin@emboleye:~/Desktop$ dkms status
nvidia, 460.32.03: added
veeamsnap, 5.0.0.4318, 4.15.0-136-generic, x86_64: installed
veeamsnap, 5.0.0.4318, 5.3.0-66-generic, x86_64: installed
wradmin@emboleye:~/Desktop$ nvidia-smi

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

wradmin@emboleye:~/Desktop$ whereis nvidia
nvidia: /usr/lib/x86_64-linux-gnu/nvidia /usr/lib/nvidia /usr/share/nvidia /usr/src/nvidia-460.32.03/nvidia

wradmin@emboleye:~/Desktop$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic


wradmin@emboleye:~/Desktop$ lsmod | grep nvidia
i2c_nvidia_gpu         16384  0

wradmin@emboleye:/etc/X11$ uname -a
Linux emboleye 5.4.0-050400-generic #201911242031 SMP Mon Nov 25 01:35:10 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

I did find a rule: 71-nvidia.rules at grep 10de /lib/udev/rules.d/*
that indicated 10de. I've commented it out, updated initramfs and restarted --> no luck.
nothing in /etc/udev/rules.d/*

We've toggled the nvidia-prime select to intel and back to nvidia.

wradmin@emboleye:~$ dpkg -l |grep ubuntu-drivers-common
ii  ubuntu-drivers-common                                 1:0.5.2.5                                        amd64        Detect and install additional Ubuntu driver packages

I tried GRUB_CMDLINE_LINUX_DEFAULT="quiet splash pci=realloc"

updated grub, restarted. No change.

no /etc/X11/xorg.conf file on our system.

Unable to init server: Could not connect: Connection refused

wradmin@emboleye:/etc/X11$ sudo nvidia-settings
ERROR: The control display is undefined; please run `nvidia-settings --help` for usage information.

You have set your system compiler to gcc 6.5 but 7.5 ist needed. Please set gcc and cc to gcc 7.5 using update-alternatives.

Good morning! Thanks so much for your swift reply. I’ve updated GCC to 7.5, set to update-alternatives 7 with:

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 1

update-alternatives --query gcc:
wradmin@emboleye:~$ update-alternatives --query gcc
Name: gcc
Link: /usr/bin/gcc
Status: auto
Best: /usr/bin/gcc-7
Value: /usr/bin/gcc-7

Alternative: /usr/bin/gcc-7
Priority: 1

wradmin@emboleye:~$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.5.0-3ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04) 

wradmin@emboleye:~$ sudo update-alternatives --config gcc
There is only one alternative in link group gcc (providing /usr/bin/gcc): /usr/bin/gcc-7
Nothing to configure.

issue persists after initramfs update + restart:
wradmin@emboleye:~$ nvidia-smi
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

I re-ran nvidia-bug-report, and have re-attached here in case it’s got new data now that the compiler is different.

Thanks for your help!

nvidia-bug-report.log.gz (1.1 MB)

You also need to trigger dkms to compile the driver
sudo dkms install nvidia/460.32.03 --all
and afterwards check if the driver has status “installed”
dkms status

wradmin@emboleye:~$ sudo dkms install nvidia/460.32.03

Kernel preparation unnecessary for this kernel.  Skipping...
applying patch disable_fstack-clash-protection_fcf-protection.patch...patching file Kbuild
Hunk #1 succeeded at 85 (offset 14 lines).


Building module:
cleaning build area...
unset ARCH; [ ! -h /usr/bin/cc ] && export CC=/usr/bin/gcc; env NV_VERBOSE=1 'make' -j8 NV_EXCLUDE_BUILD_MODULES='' KERNEL_UNAME=5.4.0-050400-generic IGNORE_XEN_PRESENCE=1 IGNORE_CC_MISMATCH=1 SYSSRC=/lib/modules/5.4.0-050400-generic/build LD=/usr/bin/ld.bfd modules..........
Signing module:
 - /var/lib/dkms/nvidia/460.32.03/5.4.0-050400-generic/x86_64/module/nvidia-uvm.ko
 - /var/lib/dkms/nvidia/460.32.03/5.4.0-050400-generic/x86_64/module/nvidia-modeset.ko
 - /var/lib/dkms/nvidia/460.32.03/5.4.0-050400-generic/x86_64/module/nvidia.ko
 - /var/lib/dkms/nvidia/460.32.03/5.4.0-050400-generic/x86_64/module/nvidia-drm.ko
Secure Boot not enabled on this system.
cleaning build area...

DKMS: build completed.

nvidia.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.4.0-050400-generic/updates/dkms/

nvidia-modeset.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.4.0-050400-generic/updates/dkms/

nvidia-drm.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.4.0-050400-generic/updates/dkms/

nvidia-uvm.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.4.0-050400-generic/updates/dkms/

depmod...

DKMS: install completed.
wradmin@emboleye:~$ sudo dkms install nvidia/460.32.03 --all
Module nvidia/460.32.03 already installed on kernel 5.4.0-050400-generic/x86_64
wradmin@emboleye:~$ dkms status
nvidia, 460.32.03, 5.4.0-050400-generic, x86_64: installed
veeamsnap, 5.0.0.4318, 4.15.0-136-generic, x86_64: installed
veeamsnap, 5.0.0.4318, 5.3.0-66-generic, x86_64: installed

rebooted… and it bloody works. THANK YOU!! 10/10, 5-stars, smashed like button. I really appreciate your assistance here, and swift attention.

Cheers!

Hi generix,
I also experienced the same problem after update today. Could you please have a look on my log file.
Thanks in advance!
Nenad

nvidia-bug-report.log.gz (108.7 KB)

Please check if you can manually load the modules:
sudo modprobe nvidia

Hi, unfortunately I also get almost the same problem. Nvidia is not working! I installed Kubuntu 20.04.2 with automated updating, my GPU is Geforce GTX 1060 6GB. Nvidia-smi is not working. After booting nothing happend, display does not show anything. I finally got managed to boot in recovery mode of GRUB and executed “nvidia-bug-report.sh”.

Thanks in advance for any suggestion!

nvidia-bug-report.log.gz (58.9 KB)

Thank you for your fast response!
sudo modprobe nvidia
returns:
modprobe: ERROR: could not insert 'nvidia': Key was rejected by service.

I tried also with the nvidia 450 driver. It doesn’t work as well; same problem.
I have run the sudo ubuntu-drivers autoinstall without success.

@polomacnenad Please try this:
https://forums.developer.nvidia.com/t/nvidia-smi-has-failed-because-it-couldn-t-communicate-with-the-nvidia-driver-after-updating-ubuntu-20-04/170985/7?u=generix

@a.javan.gh Please get a log from the previous boot (to black screen)
sudo journalctl -b-1 >journal.txt
and attach that.

@generix thanks for your reply. Just to be sure that everything is well in my PC i just installed latest fedora (v33) with nvidia driver (460.56), everything worked fine. I just reinstalled Ubunut 20.04.2 again, but still the same result. No login splash and no desktop. So i logged in termin (fn+f2) and started journalctl and nvidia-bug-report. Please find the results attached.

Btw journalctl -b-1 gives:
data from the specified boot (-1) is not available no such boot id in journal

After removing nvidia-driver with:
sudo apt remove --purge '^nvidia-.*'
I could login, but w/o nvidia :(.

Is something wrong with the latest ubuntu version?

Thoughts?

Thanks in advance

nvidia-bug-report.log.gz (46.8 KB)
journal.log (150.3 KB)

Many users are running into this since a few days, this must be some weird Ubuntu bug introduced with an update. It’s always the same issue, as soon as the nvidia driver is installed, Gnome is not even trying to start. No error is ever logged. Please check if X starts up when changing to lightdm.
Overall, this has to be reported to Ubuntu.

@generix thanks for your answer. After all now i just feel reassured, since i am not the only one with this weired issue. Btw i think it is not realy a Gnome problem, since i am actually using kubunut (kde-version of ubuntu).