JetPack L4T 3.2 Release fails to install CUDA Toolkit on host machine.

Hey,

I’m running Ubuntu 16.04 and running into issues when I’m trying to install the CUDAToolkit to host. It keeps trying to add an arm64 repository that Xenial doesn’t seem to have.

Version:  9.0
DashedVersion:  9-0
root     18658  0.1  0.0 265364  3372 ?        Ssl  08:28   0:02 /home/nvidia/jetpack32final/_installer/sudo_daemon -installer=18533 -d=/home/nvidia/jetpack32final/_installer/tmp
0
/home/nvidia/jetpack32final//_installer/run_command  -c="dpkg -i --force-overwrite /home/nvidia/jetpack32final/jetpack_download/cuda-repo-ubuntu1604-9-0-local_9.0.252-1_amd64.deb; apt-key add /var/cuda-repo-9-0-local/*.pub; apt-get update; apt-get install -y --force-yes cuda-toolkit-9-0; dpkg --add-architecture arm64;apt-get update; apt-get -y --force-yes install  cuda-toolkit-9-0 cuda-cross-aarch64-9-0" -d=/home/nvidia/jetpack32final//_installer/tmp -l=/home/nvidia/jetpack32final/_installer/logs//cuda_host_9.0.log
(Reading database ... 301779 files and directories currently installed.)
Preparing to unpack .../cuda-repo-ubuntu1604-9-0-local_9.0.252-1_amd64.deb ...
Unpacking cuda-repo-ubuntu1604-9-0-local (9.0.252-1) over (9.0.252-1) ...
Setting up cuda-repo-ubuntu1604-9-0-local (9.0.252-1) ...
OK
Ign:1 http://dl.google.com/linux/chrome/deb stable InRelease
Hit:2 http://dl.google.com/linux/chrome/deb stable Release
Hit:3 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu xenial InRelease
Hit:4 http://archive.ubuntu.com/ubuntu xenial InRelease
Hit:6 https://download.docker.com/linux/ubuntu xenial InRelease
Hit:7 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial InRelease
Hit:8 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:9 http://ppa.launchpad.net/mystic-mirage/pycharm/ubuntu xenial InRelease
Ign:10 http://security.ubuntu.com/ubuntu xenial-security/main arm64 Packages
Ign:10 http://security.ubuntu.com/ubuntu xenial-security/main arm64 Packages
Ign:10 http://security.ubuntu.com/ubuntu xenial-security/main arm64 Packages
Err:10 http://security.ubuntu.com/ubuntu xenial-security/main arm64 Packages
  404  Not Found
Ign:11 http://archive.ubuntu.com/ubuntu xenial/main arm64 Packages
Ign:11 http://archive.ubuntu.com/ubuntu xenial/main arm64 Packages
Ign:11 http://archive.ubuntu.com/ubuntu xenial/main arm64 Packages
Err:11 http://archive.ubuntu.com/ubuntu xenial/main arm64 Packages
  404  Not Found
Reading package lists...
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial/main/binary-arm64/Packages  404  Not Found
E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/xenial-security/main/binary-arm64/Packages  404  Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
cuda-toolkit-9-0 is already the newest version (9.0.252-1).
The following packages were automatically installed and are no longer required:
  gir1.2-gtk-2.0 libatk1.0-dev libcairo-script-interpreter2 libcairo2-dev
  libfontconfig1-dev libfreetype6-dev libgdk-pixbuf2.0-dev libglib2.0-dev
  libgtk2.0-dev libgtkglext1 libharfbuzz-dev libharfbuzz-gobject0
  libpango1.0-dev libpcre3-dev libpcre32-3 libpcrecpp0v5 libpixman-1-dev
  libxcb-shm0-dev libxcomposite-dev libxcursor-dev libxft-dev libxinerama-dev
  libxml2-utils libxrandr-dev libxrender-dev linux-headers-4.10.0-28
  linux-headers-4.10.0-28-generic linux-image-4.10.0-28-generic
  linux-image-extra-4.10.0-28-generic x11proto-composite-dev
  x11proto-randr-dev x11proto-render-dev x11proto-xinerama-dev
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Ign:1 http://dl.google.com/linux/chrome/deb stable InRelease
Hit:2 http://dl.google.com/linux/chrome/deb stable Release
Hit:4 https://download.docker.com/linux/ubuntu xenial InRelease
Hit:5 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu xenial InRelease
Hit:6 http://archive.ubuntu.com/ubuntu xenial InRelease
Hit:7 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:8 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial InRelease
Hit:9 http://ppa.launchpad.net/mystic-mirage/pycharm/ubuntu xenial InRelease
Ign:10 http://archive.ubuntu.com/ubuntu xenial/main arm64 Packages
Ign:11 http://security.ubuntu.com/ubuntu xenial-security/main arm64 Packages
Ign:10 http://archive.ubuntu.com/ubuntu xenial/main arm64 Packages
Ign:11 http://security.ubuntu.com/ubuntu xenial-security/main arm64 Packages
Ign:10 http://archive.ubuntu.com/ubuntu xenial/main arm64 Packages
Err:10 http://archive.ubuntu.com/ubuntu xenial/main arm64 Packages
  404  Not Found
Ign:11 http://security.ubuntu.com/ubuntu xenial-security/main arm64 Packages
Err:11 http://security.ubuntu.com/ubuntu xenial-security/main arm64 Packages
  404  Not Found
Reading package lists...
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial/main/binary-arm64/Packages  404  Not Found
E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/xenial-security/main/binary-arm64/Packages  404  Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package cuda-cross-aarch64-9-0

What’s odd is it seems to ignore the repository and then error after it. I am running this out of my home directory. I know there’s some funkiness with file permissions on extraction. I’ve tried removing all arm64 packages and repos with:

sudo apt-get purge ".*:arm64"
sudo dpkg --remove-architecture arm64

But again, it’s just going to get readded up above every time.

Any suggestions?

Hi mlorch,

We tried install CUDA toolkit on ubuntu 16.04, but no issue.
Could you install again?

Thanks!

I’m having the same problem… below is the last part of the file cuda_host_9.0.log
Any ideas on how to install cuda_cross-aarch64-9-0?

Preparing to unpack .../cuda-toolkit-9-0_9.0.176-1_amd64.deb ...
Unpacking cuda-toolkit-9-0 (9.0.176-1) ...
Setting up cuda-license-9-0 (9.0.176-1) ...
*** LICENSE AGREEMENT ***
By using this software you agree to fully comply with the terms and 
conditions of the EULA (End User License Agreement). The EULA is located
at /usr/local/cuda-9.0/doc/EULA.txt. The EULA can also be found at
http://docs.nvidia.com/cuda/eula/index.html. If you do not agree to the
terms and conditions of the EULA, do not use the software.

Setting up cuda-misc-headers-9-0 (9.0.176-1) ...
Setting up cuda-core-9-0 (9.0.176-1) ...
Setting up cuda-cudart-9-0 (9.0.176-1) ...
Setting up cuda-driver-dev-9-0 (9.0.176-1) ...
Setting up cuda-cudart-dev-9-0 (9.0.176-1) ...
Setting up cuda-command-line-tools-9-0 (9.0.176-1) ...
Setting up cuda-cublas-9-0 (9.0.176.2-1) ...
Setting up cuda-cublas-dev-9-0 (9.0.176.2-1) ...
Setting up cuda-cufft-9-0 (9.0.176-1) ...
Setting up cuda-cufft-dev-9-0 (9.0.176-1) ...
Setting up cuda-curand-9-0 (9.0.176-1) ...
Setting up cuda-curand-dev-9-0 (9.0.176-1) ...
Setting up cuda-cusolver-9-0 (9.0.176-1) ...
Setting up cuda-cusolver-dev-9-0 (9.0.176-1) ...
Setting up cuda-cusparse-9-0 (9.0.176-1) ...
Setting up cuda-cusparse-dev-9-0 (9.0.176-1) ...
Setting up cuda-nvrtc-9-0 (9.0.176-1) ...
Setting up cuda-nvrtc-dev-9-0 (9.0.176-1) ...
Setting up cuda-npp-9-0 (9.0.176-1) ...
Setting up cuda-npp-dev-9-0 (9.0.176-1) ...
Setting up cuda-nvgraph-9-0 (9.0.176-1) ...
Setting up cuda-nvgraph-dev-9-0 (9.0.176-1) ...
Setting up cuda-samples-9-0 (9.0.176-1) ...
Setting up cuda-documentation-9-0 (9.0.176-1) ...
Setting up cuda-libraries-dev-9-0 (9.0.176-1) ...
Setting up cuda-nvml-dev-9-0 (9.0.176-1) ...
Setting up cuda-visual-tools-9-0 (9.0.176-1) ...
Setting up cuda-toolkit-9-0 (9.0.176-1) ...
Processing triggers for libc-bin (2.23-0ubuntu10) ...
W: --force-yes is deprecated, use one of the options starting with --allow instead.
Hit:1 http://archive.canonical.com/ubuntu xenial InRelease
Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease
Ign:3 http://dl.google.com/linux/chrome/deb stable InRelease
Hit:4 http://ppa.launchpad.net/rael-gc/scudcloud/ubuntu xenial InRelease
Hit:5 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:6 http://dl.google.com/linux/chrome/deb stable Release
Hit:7 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
Hit:8 http://deb.torproject.org/torproject.org xenial InRelease
Hit:9 http://archive.ubuntu.com/ubuntu xenial-security InRelease
Ign:10 http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64  InRelease
Hit:11 http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64  Release
Hit:12 https://dl.winehq.org/wine-builds/ubuntu xenial InRelease
Hit:13 https://nvidia.github.io/libnvidia-container/ubuntu16.04/amd64  InRelease
Hit:14 https://nvidia.github.io/nvidia-container-runtime/ubuntu16.04/amd64  InRelease
Hit:15 https://download.docker.com/linux/ubuntu xenial InRelease
Get:16 https://nvidia.github.io/nvidia-docker/ubuntu16.04/amd64  InRelease [1 096 B]
Fetched 1 096 B in 1s (621 B/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package cuda-cross-aarch64-9-0

borishiacb,

How is your environment? Is it also a 16.04 host? Did you ever install any cuda package before jetpack do?

Hi WayneWWW,
yes I’m also using Ubuntu 16.04 and I was using Cuda8 until I decided to retake my Jetson project and saw the new Jetpack3.2 in an email. I did a sudo apt-get update + sudo apt-get install cuda and got the last version installed (9.1.85-1) without any problems and it seems my card drivers were also updated without problems. Restarted and saw that /usr/local/cuda is now pointing to cuda-9.1 but the old cuda-8.0 folder is still there. Then I executed the JetPack-L4T-3.2-linux-x64_b196.run and it seems it installed cuda-9.0.176-1 and now /usr/local/cuda points to cuda-9.0. The problem is that the Jetpack3.2 installer stopped when trying to find cuda-cross-aarch64-9-0 which I can’t find wither on the web.

Below there is some info about my systems as of today.

boris:~$ nvcc -V
  nvcc: NVIDIA (R) Cuda compiler driver
  Copyright (c) 2005-2017 NVIDIA Corporation
  Built on Fri_Sep__1_21:08:03_CDT_2017
  Cuda compilation tools, release 9.0, V9.0.176
boris:~$ uname -r
  4.13.0-37-generic
boris:~$ lspci | grep -i nvidia
  01:00.0 3D controller: NVIDIA Corporation GM108M [GeForce 940M] (rev a2)
boris:~$ uname -m && cat /etc/*release
  x86_64
  DISTRIB_ID=Ubuntu
  DISTRIB_RELEASE=16.04
  DISTRIB_CODENAME=xenial
  DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
  NAME="Ubuntu"
  VERSION="16.04.3 LTS (Xenial Xerus)"
  ID=ubuntu
  ID_LIKE=debian
  PRETTY_NAME="Ubuntu 16.04.3 LTS"
  VERSION_ID="16.04"
  HOME_URL="http://www.ubuntu.com/"
  SUPPORT_URL="http://help.ubuntu.com/"
  BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
  VERSION_CODENAME=xenial
  UBUNTU_CODENAME=xenial

I think the cause of this issue is from jetpack, which does not detect your cuda 9.1. Jetapck expects a clean environment but it turns out not. We got some issues on 16.04 host when there is pre-installed package. The quick way to resolve this issue is uninstall the cuda 9.1 manually.

Really sorry for inconvenience. I will report this issue to enhance jetpack.

Hi again,
I tried your suggestion, uninstall Cuda 9.1 manually and Ii also erased Cuda 8.0 and Cuda 9.0 and the symbolic links and all references to Cuda from my PATH and LD_LIBRARY_PATH. Then I run the Jetpack installer again and I get the same error.

E: Unable to locate package cuda-cross-aarch64-9-0

I guess that if there are no other suggestions I’ll have to try and use JetPack 3.1

Are you able to see it in apt-cache? (I guess not)

apt-cache show cuda-cross-*

The package manager seems to only see the 8.0 version. This is what I get:

boris:~/Downloads$ apt-cache show cuda-cross-* | grep cuda-cross-aarch64
  Package: cuda-cross-aarch64-8-0
  Package: cuda-cross-aarch64-8-0
  Filename: ./cuda-cross-aarch64-8-0_8.0.61-1_amd64.deb
  Package: cuda-cross-aarch64-8-0
  Filename: ./cuda-cross-aarch64-8-0_8.0.44-1_amd64.deb
  Package: cuda-cross-aarch64
  Depends: cuda-cross-aarch64-8-0
  Filename: ./cuda-cross-aarch64_8.0.61-1_amd64.deb
  Package: cuda-cross-aarch64
  Depends: cuda-cross-aarch64-8-0
  Filename: ./cuda-cross-aarch64_8.0.44-1_amd64.deb

and the ‘/etc/apt/sources.list.d/cuda.list’ file contains the following single line:

deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 /

borishiacb,

Does jetpack download “./var/cuda-repo-9-0-local/cuda-cross-aarch64_9.0.252-1_amd64.deb” to your host?

Yes, I see it inside /var/cuda-repo-9-0-local but I don’t see that file inside the download folder of Jetpack in my home directory. There are two files inside /var/cuda-repo-9-0-local:

cuda-cross-aarch64_9.0.252-1_amd64.deb
cuda-cross-aarch64-9-0_9.0.252-1_amd64.deb

Should I install it before running the Jetpack installer? If so, what command should I use?

sudo dpkg -i /path/to/deb/file

and then use apt-get to install it again.

Hi!
I had exactly the same problem …

When I tried to install cuda-cross-aarch64-9-0_9.0.252-1_amd64.deb

dpkg: dependency problems prevent configuration of cuda-cross-aarch64-9-0:
 cuda-cross-aarch64-9-0 depends on cuda-toolkit-9-0; however:
  Package cuda-toolkit-9-0 is not installed.
 cuda-cross-aarch64-9-0 depends on g++-aarch64-linux-gnu; however:
  Package g++-aarch64-linux-gnu is not installed.

and I had to add additional Ubuntu source archive:

https://launchpad.net/ubuntu xenial main

because there is no arm64 directory on the main us.ubuntu.com

So, after that I run:

sudo dpkg -i /var/cuda-repo-9-0-local/*.deb
sudo apt -f install

and that’s all. Then I’ve pressed ‘Next’ in JetPack installer and it finished well.

Hope it will help others.

This didn’t work for me, it stops after a warning saying that there is no Release file in that repository and that the repository can’t be authenticated. It fails to fetch packages and stops.

However the following lines worked:

Thanks for the suggestion!
Boris

PS: I have now a freshly flashed Jetson TX2 but I can’t find any samples or “/home/ubuntu/NVIDIA_CUDA-_Samples/bin/aarch64/linux/release/” folder as it’s described in the instructions manual. Any ideas why or where to find the samples?

I have a laptop I ran 3.2 from…I specifically checked to not delete temporary files after running…I don’t know if this applies to that directory or not. On this laptop the install location was “/home/ubuntu/Jetson_3.2/”. As a result the laptop has:

/home/ubuntu/Jetson_3.2/<b>64_TX2/NVIDIA_CUDA-9.0_Samples/bin/aarch64/linux/release/</b>

This of course requires sample code install to be checked and after done not deleted. The Jetson itself is the one which has “/home/ubuntu/NVIDIA_CUDA-9.0_Samples/”.

NOTE: Just realized probably the host would have the “/home/ubuntu/NVIDIA_…” if this laptop had an NVIDIA video card/CUDA…but you did mention aarch64 which would never be installed directly to a bin directory on the host (those CUDA files are a side-effect of installing them to the Jetson).

I have a similar problem. I am installing JetPack 3.2. I get the message:
E: Unable to locate package cuda-toolkit-8-0-82
E: Unable to locate package cuda-cross_aarch64-8-0-82

I have the following in var directory:
cuda-repo-8-0-local
cuda-repo-9-0-local

Is your host Ubuntu (and if so, which version)? Do you have an NVIDIA GPU installed on the host with the NVIDIA driver?

Yes. Ubuntu 16.04. NVIDIA GEFORCE is installed on the host.

Also, in the 8-0-local directory, I have version 64 of cuda and not 82.

For this specific case are you attempting to install CUDA to host or to Jetson?

FYI, the CUDA repo package is the one which tells the operating system that the “/var” files are for those package requests. For example, on R28.2 on the Jetson itself there is this file (version changes depending on release…arm64 is for Jetson, amd64 is for host PC, armhf is for older 32-bit ARM):

cuda-repo-l4t-9-0-local_9.0.252-1_arm64.deb

(R28.2.1 is out, I haven’t looked yet to see what the exact repo file name is)

Installing the files in “/var” purely by copy would leave the operating system not knowing those are packages and thus even if in place those packages would not be visible. Installing via a repo “.deb” package means “apt” tools can find this. JetPack always installs the repo package first. Are you installing via JetPack or trying to install manually? If manually, then it means you didn’t update apt to know the local repo is there.

FYI, the more recent JetPack releases won’t contain the older version 8 CUDA. Are you looking for a specific CUDA release?