Hi @steven.ramboer and @xkszltl
I have filed an internal task to notify the cuDNN, NCCL, and TensorRT teams about this request for Ubuntu 22.04 .deb packages to be available (hopefully sooner than later). Also will ask about .deb packages for the Debian 11 repo too.
Hi all. For anyone annoyed by the apt-key warning :)
Easy fix …
Thanks! Had to keep stealing things from 20.04 to 11 and hope that’ll be fixed soon.
For whoever had the same challenge, here’s how you can do it properly, with priority to avoid overwriting debian 11’s own deb: Roaster/repo.sh at 460b2e126909448e0c285b3de4eaeb4ce5cdb009 · xkszltl/Roaster · GitHub
I’ve discovered this discussion while trying to update some CentOS 7 severs.
I have tried to follow the instructions here but they simply do not work. There is some circular logic happening and I have no idea how to break it:
I remove the bad key using sudo rpm -e gpg-pubkey-7fa2af80-576db785
I verify that it is gone with rpm -q gpg-pubkey --qf ‘%{NAME}-%{VERSION}-%{RELEASE}\t%{SUMMARY}\n’
Yet when I try to install the latest nvidia driver, the bad key is RE-DOWNLOADED!
=============================================================================================================================================================================================================================================================
Package Arch Version Repository Size
=============================================================================================================================================================================================================================================================
Updating:
nvidia-driver-latest-dkms x86_64 3:515.48.07-1.el7 cuda 23 M
Installing for dependencies:
egl-wayland x86_64 1.1.6-1.el7 epel 29 k
Updating for dependencies:
kmod-nvidia-latest-dkms x86_64 3:515.48.07-1.el7 cuda 30 M
nvidia-driver-latest-dkms-NVML x86_64 3:515.48.07-1.el7 cuda 468 k
nvidia-driver-latest-dkms-NvFBCOpenGL x86_64 3:515.48.07-1.el7 cuda 59 k
nvidia-driver-latest-dkms-cuda x86_64 3:515.48.07-1.el7 cuda 291 k
nvidia-driver-latest-dkms-cuda-libs x86_64 3:515.48.07-1.el7 cuda 54 M
nvidia-driver-latest-dkms-devel x86_64 3:515.48.07-1.el7 cuda 19 k
nvidia-driver-latest-dkms-libs x86_64 3:515.48.07-1.el7 cuda 177 M
nvidia-modprobe-latest-dkms x86_64 3:515.48.07-1.el7 cuda 34 k
nvidia-persistenced-latest-dkms x86_64 3:515.48.07-1.el7 cuda 36 k
nvidia-xconfig-latest-dkms x86_64 3:515.48.07-1.el7 cuda 95 k
Transaction Summary
=============================================================================================================================================================================================================================================================
Install ( 1 Dependent package)
Upgrade 1 Package (+10 Dependent packages)
Total size: 286 M
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/7/cuda/packages/nvidia-driver-latest-dkms-cuda-515.48.07-1.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d42d0685: NOKEY
Retrieving key from https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/7fa2af80.pub
Importing GPG key 0x7FA2AF80:
Userid : "cudatools <cudatools@nvidia.com>"
Fingerprint: ae09 fe4b bd22 3a84 b2cc fce3 f60f 4b3d 7fa2 af80
From : https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/7fa2af80.pub
Is this ok [y/N]: y
Public key for nvidia-driver-latest-dkms-cuda-515.48.07-1.el7.x86_64.rpm is not installed
Failing package is: 3:nvidia-driver-latest-dkms-cuda-515.48.07-1.el7.x86_64
GPG Keys are configured as: https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/7fa2af80.pub
How can I get the correct key? RPM is supposed to download it according to the documentation yet it’s getting the old key. Please help!
Hi @lh2332
Yes, you will need to update the .repo
file for RHEL-based, Fedora, and SUSE distros. The cuda-rhel7.repo file on your system is pointing at the old GPG key location, which is why it is re-downloading it.
This is mentioned very briefly in the CUDA Installation Guide
- Install the new CUDA public GPG key: The new GPG public key for the CUDA repository (RPM-based distros) is d42d0685.On a fresh installation of RHEL, the yum package manager will prompt the user to accept new keys when installing packages the first time. Indicate you accept the change when prompted.
For upgrades, you must also also fetch an updated .repo entry:
sudo yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/$distro/$arch/cuda-$distro.repo
It should like look this
$ cat /etc/yum.repos.d/cuda-rhel7.repo
[cuda-rhel7-x86_64]
name=cuda-rhel7-x86_64
baseurl=https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64
enabled=1
gpgcheck=1
gpgkey=https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/D42D0685.pub
Hi @kmittman
Thanks so much for the prompt reply.
I actually did that step, and the file is correct but the error persists.
-Lokke
Hi @kmittman
I got it to work by deleting the key, using wget to download the key, and then using rpmkeys to install the key manually.
I just wanted you to know the other method was failing
-Lokke
Do you happen to have other cuda repos listed under /etc/yum.repos.d
?
Yours called cuda in log, but these days it’s called cuda-rhel7, so probably you have an old version as well and that’s the one complaining.
Ah yes there are other cuda repos. Since my fix worked, I’m not going to mess with that however but will keep it in mind.
Thanks for the help!
Go into /etc/apt/sources.list.d and delete cuda.list and cuda_learn.list (there should be another file there e.g. cuda-ubuntu2004-x86_64.list)
I’ve solved it with “gpgcheck” off :
vim /etc/yum.repos.d/cuda-rhel7.repo
[cuda-rhel7-x86_64]
name=cuda-rhel7-x86_64
baseurl=Index of /compute/cuda/repos/rhel7/x86_64
enabled=1
gpgcheck=0
…
#yum update
Hi @schwab1976
I do not recommend disabling the GPG signature check. This is an important security feature.
Updating the local gpg-pubkey on RHEL7-like distros can be achieved by refreshing the .repo
file like so:
sudo yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-rhel7.repo
Below is a demonstration of the migration from the old GPG key to the new GPG key.
Please do not follow these steps.
$ podman run -it centos:7 /bin/bash
# yum-config-manager --disable updates >/dev/null
# curl https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-rhel7.repo -o cuda-rhel7.repo
# sed -i 's|D42D0685\.pub|7fa2af80\.pub|' cuda-rhel7.repo
# yum-config-manager --add-repo cuda-rhel7.repo
# yum makecache
The deprecated GPG pubkey does not match the RPMs in the repository …
# yum install libnvjpeg-11-0
[...]
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/7/cuda-rhel7-x86_64/packages/libnvjpeg-11-0-11.1.1.245-1.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d42d0685: NOKEY
Public key for libnvjpeg-11-0-11.1.1.245-1.x86_64.rpm is not installed
libnvjpeg-11-0-11.1.1.245-1.x86_64.rpm | 2.0 MB 00:00:00
Retrieving key from https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/7fa2af80.pub
Importing GPG key 0x7FA2AF80:
Userid : "cudatools <cudatools@nvidia.com>"
Fingerprint: ae09 fe4b bd22 3a84 b2cc fce3 f60f 4b3d 7fa2 af80
From : https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/7fa2af80.pub
Is this ok [y/N]: y
Public key for libnvjpeg-11-0-11.1.1.245-1.x86_64.rpm is not installed
Failing package is: libnvjpeg-11-0-11.1.1.245-1.x86_64
GPG Keys are configured as: https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/7fa2af80.pub
Then force refresh the .repo
file
# yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-rhel7.repo
Loaded plugins: fastestmirror, ovl
adding repo from: https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-rhel7.repo
grabbing file https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-rhel7.repo to /etc/yum.repos.d/cuda-rhel7.repo
repo saved to /etc/yum.repos.d/cuda-rhel7.repo
Now packages can be installed
# yum install libnvjpeg-11-0
[...]
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/7/cuda-rhel7-x86_64/packages/libnvjpeg-11-0-11.1.1.245-1.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d42d0685: NOKEY
Retrieving key from https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/D42D0685.pub
Importing GPG key 0xD42D0685:
Userid : "cudatools <cudatools@nvidia.com>"
Fingerprint: 610c 7b14 e068 a878 070d a4e9 9cd0 a493 d42d 0685
From : https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/D42D0685.pub
Is this ok [y/N]: y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libnvjpeg-11-0-11.1.1.245-1.x86_64 1/1
Verifying : libnvjpeg-11-0-11.1.1.245-1.x86_64 1/1
Installed:
libnvjpeg-11-0.x86_64 0:11.1.1.245-1
Complete!
it seems the keys changed today. it is working!
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
sudo apt-get update
sudo apt-get -y install cuda
Your instructions to remove the old and install the new key lead to the same old key, at least for ubuntu2204/x86_64
Can you please correct your article under:
Chain of commands instructed there for Ubuntu:
$ sudo apt-key del 7fa2af80
$ sudo dpkg -i cuda-keyring_1.0-1_all.deb
triggers then the result:
Selecting previously unselected package cuda-keyring.
(Reading database … 254715 files and directories currently installed.)
Preparing to unpack cuda-keyring_1.0-1_all.deb …
Unpacking cuda-keyring (1.0-1) …
Setting up cuda-keyring (1.0-1) …
A deprecated public CUDA GPG key appear to be installed.
To remove the key, run this command:
sudo apt-key del 7fa2af80
Hi @philipp4
Please use cuda-keyring version 1.1
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
This has been updated in the CUDA Installation Guide https://docs.nvidia.com/cuda/cuda-installation-guide-linux/#network-repo-installation-for-ubuntu
Has this procedure been updated? I have run into the same error message:
W GPG error: file:/var/cudnn-local-repo-ubuntu2204-8.9.1.23 InRelease: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY 26253C7BE7A7D88D
and I followed the procedure in the post, however the same GPG error is occurring.
The cuda-keyring
package enables the “online” repository. It does not enroll a GPG public key for the “offline” local repos.
The local repo installers use ephemeral “one-time” keys, so please follow the copy instruction provided at install time, i.e. for CUDA 12.1
$ sudo dpkg -i cudnn-local-repo-ubuntu2204-8.9.1.23_1.0-1_amd64.deb
Selecting previously unselected package cudnn-local-repo-ubuntu2204-8.9.1.23.
(Reading database ... 5258 files and directories currently installed.)
Preparing to unpack cudnn-local-repo-ubuntu2204-8.9.1.23_1.0-1_amd64.deb ...
Unpacking cudnn-local-repo-ubuntu2204-8.9.1.23 (1.0-1) ...
Setting up cudnn-local-repo-ubuntu2204-8.9.1.23 (1.0-1) ...
The public cudnn-local-repo-ubuntu2204-8.9.1.23 GPG key does not appear to be installed.
To install the key, run this command:
sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.1.23/cudnn-local-E7A7D88D-keyring.gpg /usr/share/keyrings/
As such, simply adapt 1. Introduction — cuda-installation-guide-linux 12.2 documentation
sudo cp -v /var/cudnn-local-repo-ubuntu2204-8.9.1.23/cudnn-local-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
Any chance at taking compliance seriously after 4 years? Especially with CMMC 2.0 here? Yes, FIPS matters.
@BJSmithIEEE
A few possible workaround I’ve been using for similar things:
- Use some custom scripts to query the 2nd/3rd level of dependency tree and kick out unwanted pkgs (e.g. nsight in this case).
- Mirror their repo through https to ensure it’s authentic in transit, and repack/re-sign with compliant algo.