Upgrading CUDA on Jetson TK1

I recently upgraded my Jetson TK1 to Ubuntu 16.04, inadvertently breaking my CUDA 6.5 installation. It broke because the Ubuntu upgrade came with an upgrade of gcc, and the old nvcc doesn’t support gcc versions 4.9 and up. So I tried to upgrade CUDA to CUDA 7.0, but that failed because of a file signature problem:

E: Failed to fetch file:/var/cuda-repo-7-0-local/Release No Hash entry in Release file /var/lib/apt/lists/partial/_var_cuda-repo-7-0-local_Release which is considered strong enough for security purposes

So how do I get out of this? I saw one thread that suggested symlinking the old gcc-4.8 compiler to /usr/local/cuda/bin/gcc, but that failed trying to executed cc1plus.

Any help would be appreciated. My CUDA world is dysfunctional.

A TK1 is 32-bit, and the most recent version of CUDA supporting 32-bit is 6.5. There is no CUDA version more recent which runs on anything but 64-bit, so your only recourse is to revert to 6.5. Your best bet would be to flash to L4T R21.5 (sorry, it sounds like you wanted CUDA 7, it just isn’t possible with that hardware).

It’s a bit of a stretch to say I “wanted” CUDA 7. What I want is a working system. I naively upgraded to Ubuntu 16.04, which brought along with it gcc 5.x, which is greater than 4.8, which broke the nvcc compiler in CUDA 6.5. I then, also naively, guessed that an upgrade of CUDA was called for, which I now learn is not the case. I’m content to use 6.5, but only if it actually works. And since it refuses to run with a version of gcc greater than 4.8, I need to know how to work around that problem. I’m even willing to go back to Ubuntu 14.04, as much as I dislike reverting OSes, if necessary, if I could find guidance on how to do that. Maybe NVidia has a downloadable image for the original SD card?

In the following, hw.cu is plain-old hello-world, right out of K&R C, but with the .cu extension so nvcc will take it.

$ nvcc hw.cu -o hw
In file included from /usr/local/cuda-6.5/bin/…/targets/armv7-linux-gnueabihf/include/cuda_runtime.h:59:0,
from :0:
/usr/local/cuda-6.5/bin/…/targets/armv7-linux-gnueabihf/include/host_config.h:82:2: error: #error – unsupported GNU version! gcc 4.9 and up are not supported!
#error – unsupported GNU version! gcc 4.9 and up are not supported!

I found one forum entry that suggested

$ sudo ln -s /usr/bin/gcc-4.8 /usr/local/cuda/bin/gcc

that was alleged to solve the problem, but when I tried it, nvcc failed because it couldn’t launch cc1plus, whatever that is.

$ nvcc -V
nvcc: NVIDIA ® Cuda compiler driver
Copyright © 2005-2014 NVIDIA Corporation
Built on Fri_Dec_12_11:12:07_CST_2014
Cuda compilation tools, release 6.5, V6.5.35

$ nvcc hw.cu -o hw
gcc: error trying to exec ‘cc1plus’: execvp: No such file or directory

The upgrade of Ubuntu, when not via the flash mechanism, will leave the Jetson unusable (things will break, Ubuntu does not understand Jetson hardware requirements…you can upgrade packages with apt, but don’t ever upgrade the base version of Ubuntu with any mechanism other than flash). I suspect the only way to get things working as they should is to revert by flashing L4T R21.5. CUDA 6.5 has worked quite nicely on this release.

Issues with the compiler seem odd if the JTK1’s install is complete. To see if at minimum the NVIDIA-specific files are in place run this command:

sha1sum -c /etc/nv_tegra_release

…if anything in that fails, then the direct hardware access files are corrupted or missing (such as being overwritten).

I have found the “factory reset” instructions and will attempt that tonight. Thanks for the clarification.

I’d advise version R21.5:

Does NVIDIA have proper, current, correct instructions on how to re-flash to 21.5? All Google finds me is out-of-date older versions that refer to non-existent servers for downloading the files for flashing.

If you use JetPack from an Ubuntu 14 host, it’s just a case of selecting what you want to do (and being sure it is for TK1, not TX1). The minimal method (and which works from any Linux x86_64 desktop host) requires download of driver package plus sample rootfs…JetPack does this too, but acts as a front end. Flashing from command line is reliable, and if desired, you can then run JetPack without flashing to install other packages.

If you choose command line, download driver package, unpack sample rootfs in the rootfs subdirectory…it is very important to use sudo while doing the unpacking:

# unpack driver...
# you may want to be sure you have about 25GB free space..."df -H ."
cd Linux_for_Tegra
cd rootfs
sudo tar xvfj
cd ..
sudo ./apply_binaries.sh

This leaves you ready to flash. Those steps only need to be done once. To flash, put the Jetson in recovery mode (hold down the recover button and either power up or reset power). Check your host sees the Jetson via the micro-B USB cable…any response implies the Jetson is seen in recovery mode:

lsusb -d 0955:7140

Now flash with sudo:

sudo ./flash.sh -S 14580MiB jetson-tk1 mmcblk0p1

Note that the “-S 14580MiB” just uses the largest rootfs you can…unless you want some special reserved partition this is what you want. After the flash (takes a long time) it should reboot and “just work”. When using command line there is no attempt to configure host networking, so you’ll need to know the IP address your DHCP server provides…either via the host doing DHCP serving and logging it, or via a regular router listing what it assigned. Other than that it should just work and you can add packages as desired with JetPack or manually. You can even have JetPack do the flash itself, the steps above are for minimal command line use in case you can’t use JetPack.

Thank you! I’m back in business.