Ubuntu 20.04: Driver apt packages broken by unattended-upgrade after 2 months

Graphics / Linux Linux
1

On April 28, I installed a brand new Ubuntu Server 20.04 system with Nvidia drivers from the https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/repository. I have barely used the system at all since then.

Today I attempted to apt install a completely unrelated package and was greeted with:

The following packages have unmet dependencies:
 nvidia-dkms-510 : Depends: nvidia-kernel-common-510 (>= 510.73.05) but 510.47.03-0ubuntu1 is to be installed
 nvidia-driver-510 : Depends: nvidia-kernel-common-510 (>= 510.73.05) but 510.47.03-0ubuntu1 is to be installed
                     Recommends: libnvidia-compute-510:i386 (= 510.73.05-0ubuntu0.20.04.1)
                     Recommends: libnvidia-decode-510:i386 (= 510.73.05-0ubuntu0.20.04.1)
                     Recommends: libnvidia-encode-510:i386 (= 510.73.05-0ubuntu0.20.04.1)
                     Recommends: libnvidia-fbc1-510:i386 (= 510.73.05-0ubuntu0.20.04.1)
                     Recommends: libnvidia-gl-510:i386 (= 510.73.05-0ubuntu0.20.04.1)
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

I see that there was an unattended upgrade that was attempted and failed on May 25:

Start-Date: 2022-05-25  06:58:42
Commandline: /usr/bin/unattended-upgrade
Upgrade: libnvidia-compute-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), libnvidia-encode-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), nvidia-kernel-common-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), xserver-xorg-video-nvidia-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), libnvidia-gl-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), libnvidia-fbc1-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), libnvidia-decode-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), libnvidia-cfg1-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), nvidia-utils-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), nvidia-dkms-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), nvidia-compute-utils-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), nvidia-driver-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), libnvidia-extra-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1), nvidia-kernel-source-510:amd64 (510.47.03-0ubuntu1, 510.73.05-0ubuntu0.20.04.1)
Error: Sub-process /usr/bin/dpkg returned an error code (1)
End-Date: 2022-05-25  06:59:11

I’ve attached all relevant logs. It would be nice to know how to avoid this situation in the future.

logs.tar.gz (130.1 KB)

2

This happened again. There was an unattended upgrade:

2022-08-04 06:08:27,730 INFO Packages that will be upgraded: libnvidia-cfg1-510 libnvidia-common-510 libnvidia-compute-510 libnvidia-decode-510 libnvidia-encode-510 libnvidia-extra-510 libnvidia-fbc1-510 libnvidia-gl-510 nvidia-compute-utils-510 nvidia-dkms-510 nvidia-driver-510 nvidia-kernel-common-510 nvidia-kernel-source-510 nvidia-utils-510 xserver-xorg-video-nvidia-510

Afterwards Nvidia tools are broken, even after rebooting:

$ nvidia-smi 
Failed to initialize NVML: Driver/library version mismatch

And packages have unmet dependencies:

The following packages have unmet dependencies:
 nvidia-dkms-510 : Depends: nvidia-kernel-common-510 (>= 510.85.02) but 510.73.08-0ubuntu1 is installed
 nvidia-driver-510 : Depends: nvidia-kernel-common-510 (>= 510.85.02) but 510.73.08-0ubuntu1 is installed
                     Recommends: libnvidia-compute-510:i386 (= 510.85.02-0ubuntu0.20.04.1)
                     Recommends: libnvidia-decode-510:i386 (= 510.85.02-0ubuntu0.20.04.1)
                     Recommends: libnvidia-encode-510:i386 (= 510.85.02-0ubuntu0.20.04.1)
                     Recommends: libnvidia-fbc1-510:i386 (= 510.85.02-0ubuntu0.20.04.1)
                     Recommends: libnvidia-gl-510:i386 (= 510.85.02-0ubuntu0.20.04.1)

I ran sudo apt --fix-broken install && sudo apt upgrade and rebooted and afterwards things seemed OK.

3

You can turn off the auto upgrade:

sudo nano /etc/apt/apt.conf.d/20auto-upgrades

and set to APT::Periodic::Unattended-Upgrade "0"; from "1".

4

This happened yet again… It would be nice if Nvidia weighed in on possible causes here.

@Naofumi That’s an imperfect solution. Is it Ubuntu’s fault for upgrading packages automatically (which helps keep systems secure, for one thing)? If it is something about Nvidia’s apt packages/repos that causes this issue, Nvidia should address it.

1 Like
5

Looks like this is still a problem. For example, this error occurs when a node attempts to upgrade the chrome browser.

"chrome_browser[stable] (google_chrome::install line 79) had an error: Mixlib::ShellOut::ShellCommandFailed: apt_package[google-chrome-stable] (google_chrome::install line 31) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '100'
---- Begin output of [""apt-get"", ""-q"", ""-y"", ""--allow-downgrades"", ""-o"", ""Dpkg::Options::=--force-confdef"", ""-o"", ""Dpkg::Options::=--force-confold"", ""install"", ""google-chrome-stable=117.0.5938.149-1""] ----
STDOUT: Reading package lists...
Building dependency tree...
Reading state information...
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 nvidia-dkms-535 : Depends: nvidia-kernel-common-535 (>= 535.113.01) but 535.104.12-0ubuntu1 is to be installed
 nvidia-driver-535 : Depends: libnvidia-gl-535 (= 535.113.01-0ubuntu0.20.04.1) but 535.104.12-0ubuntu1 is to be installed
                     Depends: nvidia-kernel-common-535 (>= 535.113.01) but 535.104.12-0ubuntu1 is to be installed
                     Recommends: libnvidia-fbc1-535:i386 (= 535.113.01-0ubuntu0.20.04.1)
                     Recommends: libnvidia-gl-535:i386 (= 535.113.01-0ubuntu0.20.04.1)
STDERR: E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
---- End output of [""apt-get"", ""-q"", ""-y"", ""--allow-downgrades"", ""-o"", ""Dpkg::Options::=--force-confdef"", ""-o"", ""Dpkg::Options::=--force-confold"", ""install"", ""google-chrome-stable=117.0.5938.149-1""] ----
Ran [""apt-get"", ""-q"", ""-y"", ""--allow-downgrades"", ""-o"", ""Dpkg::Options::=--force-confdef"", ""-o"", ""Dpkg::Options::=--force-confold"", ""install"", ""google-chrome-stable=117.0.5938.149-1""] returned 100"

Is there any way to avoid this?

6

I have the same exact issues. Its quite annoying as a reboot is the only solution for this.

this is a similar issue.

7

If you have performed a network repo installation, try adding a new configuration file for unattended-upgrades:

echo "Unattended-Upgrade::Allowed-Origins:: \"NVIDIA:\";" | sudo tee -a /etc/apt/apt.conf.d/55unattended-upgrades-nvidia > /dev/null

Also, check if the cuda-repository-pin-600 file is present as mentioned on the installation guide:

cat /etc/apt/preferences.d/cuda-repository-pin-600 
Package: nsight-compute
Pin: origin *ubuntu.com*
Pin-Priority: -1

Package: nsight-systems
Pin: origin *ubuntu.com*
Pin-Priority: -1

Package: *
Pin: release l=NVIDIA CUDA
Pin-Priority: 600