Apt-get update is failing

The apt-get update command related to NVIDIA is failing on my Linux server.
Please fix it as soon as possible.


Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB]
Hit:2 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal InRelease                                 
Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease                                               
Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB]        
Get:5 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  InRelease [1581 B]
Get:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  Packages [2729 kB]
Err:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64  Packages
  File has unexpected size (2731565 != 2729198). Mirror sync in progress? [IP: 23.62.21.174 443]
  Hashes of expected file:
   - Filesize:2729198 [weak]
   - SHA256:8016001c50e96aff2e638b7664e312e0fe2a51efeafd36aae798c1d443603fc8
   - SHA1:15dd8e4febc82691fae53c423ce1ec7d3cd61b01 [weak]
   - MD5Sum:01cf0833f2bea362efc1a4150e68087f [weak]
  Release file created at: Tue, 22 Jul 2025 19:06:30 +0000
Get:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB]
Fetched 385 kB in 2s (170 kB/s)  
Reading package lists... Done
E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/Packages.gz  File has unexpected size (2731565 != 2729198). Mirror sync in progress? [IP: 23.62.21.174 443]
   Hashes of expected file:
    - Filesize:2729198 [weak]
    - SHA256:8016001c50e96aff2e638b7664e312e0fe2a51efeafd36aae798c1d443603fc8
    - SHA1:15dd8e4febc82691fae53c423ce1ec7d3cd61b01 [weak]
    - MD5Sum:01cf0833f2bea362efc1a4150e68087f [weak]
   Release file created at: Tue, 22 Jul 2025 19:06:30 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.
8 Likes

Same here, please fix the cuda_keyring package. It seems to have been updated yesterday but the checksums now fail, causing all of our workflows that rely on it to fail.

As a suggestion to nVidia it may be beneficial to include a test when releasing a new version that the released package works as intended.

Edit: temporary fixed by pinning the installation to an explicit version link

4 Likes

Same here.

1 Like

Same problem

1 Like

Same here…

Debian 12 is broken too

root@Debian-bookworm-latest-amd64-base:~# apt update
…
Err:10 Index of /compute/cuda/repos/debian12/x86_64 Packages
File has unexpected size (1044929 != 1042645). Mirror sync in progress? [IP: 23.53.42.130 443]
Hashes of expected file:

  • Filesize:1042645 [weak]
  • SHA256:13bf68c41dbdaf4a5e1047d3c50e3e468134d478352eeb3c97abf7d241e9a237
  • SHA1:237a19117dc8cb3375cb6ef1abc1f05c408525ce [weak]
  • MD5Sum:6fba555e59d2f81669dd2194a8d2c6ee [weak]
    Release file created at: Thu, 17 Jul 2025 16:33:11 +0000
    Fetched 1,581 B in 2s (763 B/s)
    Reading package lists… Done
    E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/Packages.gz File has unexpected size (1044929 != 1042645). Mirror sync in progress? [IP: 23.53.42.130 443]
    Hashes of expected file:
    - Filesize:1042645 [weak]
    - SHA256:13bf68c41dbdaf4a5e1047d3c50e3e468134d478352eeb3c97abf7d241e9a237
    - SHA1:237a19117dc8cb3375cb6ef1abc1f05c408525ce [weak]
    - MD5Sum:6fba555e59d2f81669dd2194a8d2c6ee [weak]
    Release file created at: Thu, 17 Jul 2025 16:33:11 +0000
    E: Some index files failed to download. They have been ignored, or old ones used instead.

Similar issue here. The following is blocking our CI currently (on Ubuntu 24.04):

  wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
  sudo dpkg -i cuda-keyring_1.1-1_all.deb
  sudo apt-get -y update
  sudo apt-get -y install \
    cuda-cudart-12-6 \
    cuda-cupti-12-6 \
    libcublas-12-6 \
    libcudnn9-cuda-12 \
    libcufft-12-6 \
    libcurand-12-6 \
    libcusparse-12-6 \
    libcusparselt0 \
    libnccl2 \
    libnvjitlink-12-6
  shell: /usr/bin/bash -e {0}
--2025-07-25 08:16:01--  https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
Resolving developer.download.nvidia.com (developer.download.nvidia.com)... 23.219.160.15, 23.219.160.11
Connecting to developer.download.nvidia.com (developer.download.nvidia.com)|23.219.160.15|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4328 (4.2K) [application/x-deb]
Saving to: ā€˜cuda-keyring_1.1-1_all.deb’

     0K ....                                                  100%  221M=0s

2025-07-25 08:16:02 (221 MB/s) - ā€˜cuda-keyring_1.1-1_all.deb’ saved [4328/4328]

Selecting previously unselected package cuda-keyring.
(Reading database ... 219605 files and directories currently installed.)
Preparing to unpack cuda-keyring_1.1-1_all.deb ...
Unpacking cuda-keyring (1.1-1) ...
Setting up cuda-keyring (1.1-1) ...
Get:1 file:/etc/apt/apt-mirrors.txt Mirrorlist [144 B]
Hit:2 http://azure.archive.ubuntu.com/ubuntu noble InRelease
Get:3 http://azure.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB]
Get:4 http://azure.archive.ubuntu.com/ubuntu noble-backports InRelease [126 kB]
Get:5 http://azure.archive.ubuntu.com/ubuntu noble-security InRelease [126 kB]
Hit:6 https://packages.microsoft.com/repos/azure-cli noble InRelease
Get:7 https://packages.microsoft.com/ubuntu/24.04/prod noble InRelease [3600 B]
Get:8 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64  InRelease [1581 B]
Get:9 http://azure.archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [1282 kB]
Get:10 http://azure.archive.ubuntu.com/ubuntu noble-updates/main Translation-en [260 kB]
Get:11 http://azure.archive.ubuntu.com/ubuntu noble-updates/main amd64 Components [163 kB]
Get:12 http://azure.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [1113 kB]
Get:13 http://azure.archive.ubuntu.com/ubuntu noble-updates/universe Translation-en [284 kB]
Get:14 http://azure.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Components [377 kB]
Get:15 http://azure.archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Packages [1572 kB]
Get:16 http://azure.archive.ubuntu.com/ubuntu noble-updates/restricted Translation-en [341 kB]
Get:17 http://azure.archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Components [212 B]
Get:18 http://azure.archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Components [940 B]
Get:19 http://azure.archive.ubuntu.com/ubuntu noble-backports/main amd64 Components [7052 B]
Get:20 http://azure.archive.ubuntu.com/ubuntu noble-backports/universe amd64 Components [28.4 kB]
Get:21 http://azure.archive.ubuntu.com/ubuntu noble-backports/restricted amd64 Components [212 B]
Get:22 http://azure.archive.ubuntu.com/ubuntu noble-backports/multiverse amd64 Components [212 B]
Get:23 http://azure.archive.ubuntu.com/ubuntu noble-security/main amd64 Packages [1023 kB]
Get:24 http://azure.archive.ubuntu.com/ubuntu noble-security/main Translation-en [180 kB]
Get:25 http://azure.archive.ubuntu.com/ubuntu noble-security/main amd64 Components [21.6 kB]
Get:26 http://azure.archive.ubuntu.com/ubuntu noble-security/universe amd64 Packages [876 kB]
Get:27 http://azure.archive.ubuntu.com/ubuntu noble-security/universe Translation-en [193 kB]
Get:28 http://azure.archive.ubuntu.com/ubuntu noble-security/universe amd64 Components [52.3 kB]
Get:29 http://azure.archive.ubuntu.com/ubuntu noble-security/restricted amd64 Packages [1484 kB]
Get:30 http://azure.archive.ubuntu.com/ubuntu noble-security/restricted Translation-en [323 kB]
Get:31 http://azure.archive.ubuntu.com/ubuntu noble-security/restricted amd64 Components [208 B]
Get:32 http://azure.archive.ubuntu.com/ubuntu noble-security/multiverse amd64 Components [208 B]
Get:33 https://packages.microsoft.com/ubuntu/24.04/prod noble/main all Packages [643 B]
Get:34 https://packages.microsoft.com/ubuntu/24.04/prod noble/main armhf Packages [9550 B]
Get:35 https://packages.microsoft.com/ubuntu/24.04/prod noble/main arm64 Packages [30.5 kB]
Get:36 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64  Packages [743 kB]
Err:36 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64  Packages
  File has unexpected size (745228 != 742938). Mirror sync in progress? [IP: 23.219.160.11 443]
  Hashes of expected file:
   - Filesize:742938 [weak]
   - SHA256:56e20427cd4c3c1727d8dfc7aa509f7654000a34b83d0671fb4ab86026fe66da
   - SHA1:a1cb4afef99aaca4f117c02bd5e0b7d06719b74a [weak]
   - MD5Sum:9a5a0589461fff134d6478c998fc6710 [weak]
  Release file created at: Tue, 22 Jul 2025 19:02:51 +0000
Get:37 https://packages.microsoft.com/ubuntu/24.04/prod noble/main amd64 Packages [44.9 kB]
Fetched 10.1 MB in 1s (6942 kB/s)
Reading package lists...
E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/Packages.gz  File has unexpected size (745228 != 742938). Mirror sync in progress? [IP: 23.219.160.11 443]
   Hashes of expected file:
    - Filesize:742938 [weak]
    - SHA256:56e20427cd4c3c1727d8dfc7aa509f7654000a34b83d0671fb4ab86026fe66da
    - SHA1:a1cb4afef99aaca4f117c02bd5e0b7d06719b74a [weak]
    - MD5Sum:9a5a0589461fff134d6478c998fc6710 [weak]
   Release file created at: Tue, 22 Jul 2025 19:02:51 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

Ubuntu2404 also:
Err:5 Index of /compute/cuda/repos/ubuntu2404/x86_64 Packages
File has unexpected size (745228 != 742938). Mirror sync in progress? [IP: 23.205.151.163 443]
Hashes of expected file:

  • Filesize:742938 [weak]
  • SHA256:56e20427cd4c3c1727d8dfc7aa509f7654000a34b83d0671fb4ab86026fe66da
  • SHA1:a1cb4afef99aaca4f117c02bd5e0b7d06719b74a [weak]
  • MD5Sum:9a5a0589461fff134d6478c998fc6710 [weak]
    Release file created at: Tue, 22 Jul 2025 19:02:51 +0000
    Hit:6 Index of /ubuntu noble-backports InRelease
    Fetched 1581 B in 2s (874 B/s)
    Reading package lists… Done
    E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/Packages.gz File has unexpected size (745228 != 742938). Mirror sync in progress? [IP: 23.205.151.163 443]
    Hashes of expected file:
    - Filesize:742938 [weak]
    - SHA256:56e20427cd4c3c1727d8dfc7aa509f7654000a34b83d0671fb4ab86026fe66da
    - SHA1:a1cb4afef99aaca4f117c02bd5e0b7d06719b74a [weak]
    - MD5Sum:9a5a0589461fff134d6478c998fc6710 [weak]
    Release file created at: Tue, 22 Jul 2025 19:02:51 +0000
    E: Some index files failed to download. They have been ignored, or old ones used instead.

Same issue:
ubuntu:noble-20250404

pc@user:~/work/rixjob$ curl -fsSLO https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
pc@user:~/work/rixjob$ sudo dpkg -i cuda-keyring_1.0-1_all.deb
Selecting previously unselected package cuda-keyring.
(Reading database … 73478 files and directories currently installed.)
Preparing to unpack cuda-keyring_1.0-1_all.deb …
Unpacking cuda-keyring (1.0-1) …
Setting up cuda-keyring (1.0-1) …

A deprecated public CUDA GPG key appear to be installed.
To remove the key, run this command:
sudo apt-key del 7fa2af80

pc@user:~/work/rixjob$ sudo apt-get update
Hit:1 Index of /ubuntu noble InRelease
Get:2 Index of /ubuntu noble-security InRelease [126 kB]
Get:3 Index of /ubuntu noble-updates InRelease [126 kB]
Hit:4 Index of /openjdk-r/ppa/ubuntu noble InRelease
Get:5 Index of /compute/cuda/repos/ubuntu2204/x86_64 InRelease [1,581 B]
Get:6 Index of /compute/cuda/repos/ubuntu2204/x86_64 Packages [1,851 kB]
Err:6 Index of /compute/cuda/repos/ubuntu2204/x86_64 Packages
File has unexpected size (1852870 != 1850507). Mirror sync in progress? [IP: 23.73.0.155 443]
Hashes of expected file:

  • Filesize:1850507 [weak]
  • SHA256:97960e9159be087be063fd3fcd63217a13dcd40b7ae329f38ddcee2541e2406b
  • SHA1:7baa6fed520300c1fdb5eb10ca02367f0806deb9 [weak]
  • MD5Sum:4f4f50310c11c9a0ba217aacc75d8f47 [weak]
    Release file created at: Tue, 22 Jul 2025 19:05:10 +0000
    Get:7 Index of /ubuntu noble-backports InRelease [126 kB]
    Get:8 Index of /ubuntu noble-updates/universe amd64 Packages [1,444 kB]
    Get:9 Index of /ubuntu noble-security/main amd64 Packages [1,288 kB]
    Get:10 Index of /ubuntu noble-updates/multiverse amd64 Packages [45.2 kB]
    Get:11 Index of /ubuntu noble-updates/main amd64 Packages [1,625 kB]
    Get:12 Index of /ubuntu noble-updates/restricted amd64 Packages [1,992 kB]
    Get:13 Index of /ubuntu noble-backports/universe amd64 Packages [33.0 kB]
    Get:14 Index of /ubuntu noble-backports/main amd64 Packages [48.8 kB]
    Get:15 Index of /ubuntu noble-security/multiverse amd64 Packages [23.0 kB]
    Get:16 Index of /ubuntu noble-security/universe amd64 Packages [1,132 kB]
    Get:17 Index of /ubuntu noble-security/restricted amd64 Packages [1,879 kB]
    Fetched 9,890 kB in 2s (3,967 kB/s)
    Reading package lists… Done
    E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/Packages.gz File has unexpected size (1852870 != 1850507). Mirror sync in progress? [IP: 23.73.0.155 443]
    Hashes of expected file:
    - Filesize:1850507 [weak]
    - SHA256:97960e9159be087be063fd3fcd63217a13dcd40b7ae329f38ddcee2541e2406b
    - SHA1:7baa6fed520300c1fdb5eb10ca02367f0806deb9 [weak]
    - MD5Sum:4f4f50310c11c9a0ba217aacc75d8f47 [weak]
    Release file created at: Tue, 22 Jul 2025 19:05:10 +0000
    E: Some index files failed to download. They have been ignored, or old ones used instead.
    pc@user:~/work/rixjob$ sudo apt-get update
    Hit:1 Index of /ubuntu noble-security InRelease
    Hit:2 Index of /ubuntu noble InRelease
    Hit:3 Index of /ubuntu noble-updates InRelease
    Hit:4 Index of /ubuntu noble-backports InRelease
    Hit:5 Index of /openjdk-r/ppa/ubuntu noble InRelease
    Get:6 Index of /compute/cuda/repos/ubuntu2204/x86_64 InRelease [1,581 B]
    Get:7 Index of /compute/cuda/repos/ubuntu2204/x86_64 Packages [1,851 kB]
    Err:7 Index of /compute/cuda/repos/ubuntu2204/x86_64 Packages
    File has unexpected size (1852870 != 1850507). Mirror sync in progress? [IP: 2.16.106.152 443]
    Hashes of expected file:
  • Filesize:1850507 [weak]
  • SHA256:97960e9159be087be063fd3fcd63217a13dcd40b7ae329f38ddcee2541e2406b
  • SHA1:7baa6fed520300c1fdb5eb10ca02367f0806deb9 [weak]
  • MD5Sum:4f4f50310c11c9a0ba217aacc75d8f47 [weak]
    Release file created at: Tue, 22 Jul 2025 19:05:10 +0000
    Fetched 1,581 B in 1s (2,865 B/s)
    Reading package lists… Done
    E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/Packages.gz File has unexpected size (1852870 != 1850507). Mirror sync in progress? [IP: 2.16.106.152 443]
    Hashes of expected file:
    - Filesize:1850507 [weak]
    - SHA256:97960e9159be087be063fd3fcd63217a13dcd40b7ae329f38ddcee2541e2406b
    - SHA1:7baa6fed520300c1fdb5eb10ca02367f0806deb9 [weak]
    - MD5Sum:4f4f50310c11c9a0ba217aacc75d8f47 [weak]
    Release file created at: Tue, 22 Jul 2025 19:05:10 +0000
    E: Some index files failed to download. They have been ignored, or old ones used instead.
1 Like

Same here. Every time NVIDIA’s CDN goes out of sync and breaks apt-get update. Feels like their infra team is either asleep or just doesn’t care. For a company selling ā€œaccelerated computingā€, their repo sync speed is a joke.

1 Like

Heya.

We have just started using the NVidia apt in our CI/CD pipelines (literally an hour ago!) and we’re now running into this issue. I’ve search this forum a bit and noticed this issue is not an outlier, it happens every now and then.

Could you tell me how often (in your experience) does this happen? Otherwise I need to take counter-measures right away, e.g. by mirroring the developer.download.nvidia.com apt remote internally (we do anyway) but only manually sync it.
We rely on working apt mirrors, otherwise this will disrupt our workflows.

Seems like it is fixed already.

same here.