Where is the latest version of CUDA available that could be installed on the jetsons?

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

Hello to everyone.

I’ve just installed CUDA 11 on the jetson nano using this link :

wget http://developer.download.nvidia.com/compute/cuda/11.0.2/local_installers/cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_arm64.deb

but I suspect that this is not the latest version of CUDA available than I can try to install. Can someone point me to the latest version ? Don’t care if it for Orin or Xavier. It may work even on the jetson nano. I’ve found this version,that’s for ubuntu 22.04 :

What’s sbsa ? :D I want to understand if it is compatible with the jetson nano.

1 Like
2

The versions of CUDA you find that are not specifically for a Jetson require a discrete GPU (dGPU), where the GPU is on PCI (PCI query functions can be used to find and control such a GPU). The Jetson has an integrated GPU (iGPU), which is tied directly to the memory controller. Those other versions will not work, and attempting to install one might end up making you flash the unit again to get it working.

The only version of the Jetson CUDA you can use are the ones packaged with JetPack/SDK Manager (you can run that and tell it to install various optional items if you uncheck flash and uncheck installing things to host PC, and leave the Jetson fully running, not in recovery mode). This is the only version of such software you can use with a Jetson.

Unfortunately, the Nano reached “feature end of life” long ago. You won’t find newer releases available. You would need newer hardware, e.g., Xavier or Orin. Xavier works with all of the current releases, including CUDA 11.x. When JetPack/SDKM increments to a new major version though, Xavier will not have that newer release since it too has reached “feature EOL”. The Orin would support CUDA 11.x, and whenever 12.x comes out, this would be ported to Orin, but not Xavier.

3

I call discrete GPU the gpu that I have on my PC (I have the RTX 2080 ti) ; but my PC runs with Linux x64 bit,not on the ARM architecture. The CUDA versions that I’ve mentioned on my post run on arm64. Which kind of hardware is the one where I can install CUDA (till version 12.1.1) for arm64 ? Check below :

Screenshot from 2023-05-07 00-45-40
Screenshot from 2023-05-07 00-45-401021×501 36.3 KB

you can even see there is the aarch64-jetson version for ubuntu 20.04. I wonder if it can work on the nano running with ubuntu 22.04 (upgrading 20.04 to 22.04).

Screenshot from 2023-05-07 00-49-13
Screenshot from 2023-05-07 00-49-131321×611 69.5 KB

1 Like
4

It is arm64, but that isn’t the problem. Those drivers are for dGPUs, e.g., used in a data warehouse or other “non-Jetson” setups. There is no CUDA other than that which comes with JetPack/SDKM which will work, they have no method of finding or using the iGPU. There are no plans to further develop newer releases beyond the 10.x CUDA for a Nano (which is a TX1 in smaller form factor). The only possibility of doing this is with a newer Jetson hardware, e.g., Xavier or Orin. By default Xavier can use either 10.x CUDA or 11.x if you flash with the correct JetPack (Xavier cannot support both simultaneously; it is tied to the release flashed). Orin only runs newer releases, and thus must run CUDA 11.x (but in the future it will probably have a 12.x available).

Just to emphasize, it isn’t the CPU architecture in question: It is the requirement of using PCIe versus integration directly to the memory controller.

5

Not sure how much this can applied to Nano, but I think that with JP5 the CUDA version installed by SDKM may be related to host CUDA version such as CUDA 11.8. See Simplifying CUDA Upgrades for NVIDIA Jetson Users | NVIDIA Technical Blog

6

Please,can you suggest to me a tool to check if the gpu of my jetson nano is used correctly by the drivers and the cuda libraries ? Because in jtop,even with ubuntu 18.04 + cuda 10,I don’t see any little bars moving to the right.

7

What do you see from:
ldconfig -p | grep 'cuda'

This is what libraries the linker sees in its default path, filtered for any with the name “cuda” in it.

Note that “tegrastats” is the method which guarantees to show what the GPU is doing. I think jtop uses some of the same methods (I’ve never looked at what it really does, I’m guessing), but there is a chance that something might not show up with jtop under some circumstances.

Another test would be to go to (the “cuda/” of this could be a specific numeric number, but usually the “default” version has a symbolic link just named “cuda/”):
/usr/local/cuda/samples/0_Simple/vectorAdd

Then build that:
sudo make -j 4

Once done, monitor jtop on one terminal (I do have jtop and am trying this myself). Then, on another terminal, set up a load from the “vectoraAdd/” directory (which should now have the “vectorAdd” program in it):

while [ true ]; do ./vectorAdd; done

(just an infinite loop of vectorAdd)

With that running you should see some GPU load.

8

Is there a msg between the boot messages that indicates that the GPU has been initialized correctly ? thanks.

9

What gives ?

sudo dmesg | grep gpu

As mentioned by @linuxdev, the CUDA samples would be the first thing to try for checking.
You may try /usr/local/cuda/samples/1_Utilities/deviceQuery and share what it returns.

NVIDIA gpu driver would provide stats into sysfs used by tegrastats or jtop;

sudo cat /sys/devices/gpu.0/load
1 Like
10 
$ sudo dmesg | grep gpu

[    0.488859] iommu: Adding device 57000000.gpu to group 23
[    0.488924] platform 57000000.gpu: domain=ffffffc0f9504b58 allocates as[0]=ffffffc0f9438318
[    0.488980] platform 57000000.gpu: domain=ffffffc0f9504b58 allocates as[1]=ffffffc0f9438380
[    0.489037] platform 57000000.gpu: domain=ffffffc0f9504b58 allocates as[2]=ffffffc0f94383e8
[    0.489112] platform 57000000.gpu: domain=ffffffc0f9504b58 allocates as[3]=ffffffc0f9438450
[    1.153959] vdd-gpu: applied init 1000000uV constraint
[    1.153990] vdd-gpu: 708 <--> 1323 mV at 997 mV 
[    4.141732] tegra_soctherm 700e2000.soctherm: thermtrip: will shut down when gpu reaches 103000 mC
[    4.141738] tegra_soctherm 700e2000.soctherm: throttrip: will throttle when gpu reaches 101000 mC
[    4.144032] parse_throttle_dt_data: clk=gpu type=4
[    5.455958] tegra dvfs: vdd-gpu: nominal 1078mV, offset 708000uV, step 10000uV, scaling enabled
[    5.457436] tegra_dvfs: vdd-gpu-vts: registered
[   12.996242] gpu-throttle-alert cooling device registered.
[   16.065482] nvgpu: 57000000.gpu           gm20b_init_clk_setup_sw:1268 [INFO]  GPCPLL initial settings: NA mode, M=1, N=34, P=3 (id = 1)
[   39.804211] Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_filter ip6_tables iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter zram overlay binfmt_misc nvgpu userspace_alert fuse ip_tables x_tables
[   39.827308] Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_filter ip6_tables iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter zram overlay binfmt_misc nvgpu userspace_alert fuse ip_tables x_tables
[   39.874191] Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_filter ip6_tables iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter zram overlay binfmt_misc nvgpu userspace_alert fuse ip_tables x_tables
11 
$ sudo cat /sys/devices/gpu.0/load
0

marietto@marietto-nano:/usr/local/cuda/samples/1_Utilities/deviceQuery$ sudo make

/usr/local/cuda-11.0/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -o deviceQuery.o -c deviceQuery.cpp
/usr/local/cuda-11.0/bin/nvcc -ccbin g++ -m64 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -o deviceQuery deviceQuery.o 
mkdir -p ../../bin/sbsa/linux/release
cp deviceQuery ../../bin/sbsa/linux/release

./deviceQuery
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL
12

For Nano, the cuda arch is 5.3: -gencode arch=compute_53,code=sm_53

but that will probably not solve the incompatible versions issue.

13

—> For Nano, the cuda arch is 5.3: -gencode arch=compute_53,code=sm_53

let me understand : the cuda arch for version 10 is 5.3,is this what you mean ? but if I have installed cuda 11,it can’t be 5.3,so I find “natural / normal” that it is 6.1. So,what are you telling to me ? that I can declare somewhere something like arch=5.3 even if I make the installation of cuda 11 ?

14

The CUDA arch is related to GPU hardware. Nano has Maxwell GPU that has CUDA capability 5.3 (TX2 has 6.2, Xavier has 7.2, Orin has 8.7).
For driving that HW, there is the GPU driver running.
Different CUDA versions could work for a given GPU architecture but not all, it would depend on GPU driver version.

15

so,what you suggest me to try to have CUDA 11 working on the nano ?

16 
sudo apt update
sudo apt install nvidia-jetpack

would install CUDA version for your L4T release.

17

But this is not the solution to install cuda 11. This is the way to reinstall cuda 10. Im not interested. I want to try somethng different to install Cuda 11,because I suspect that it may work.

18

Sorry, but I think that CUDA-10 is the latest CUDA version for Nano that is no longer suppported in JP5.
You may share why you suspect that newer version may work.
As GPU driver and CUDA are closed sources, I think that it would be very hard to get this running.

19

How did you get CUDA 11 on a Nano? I see this:

/usr/local/cuda-11.0/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -o deviceQuery.o -c deviceQuery.cpp

(emphasizing “/usr/local/cuda-11.0”)

CUDA 11 is not compatible with a Nano. This guarantees failure, the driver for the integrated GPU must come from JetPack/SDKM 4.x, which is CUDA 10.x.

20

—> You may share why you suspect that newer version may work.

sure. My suspect does not come from some technical information that I’ve got somewhere,because I wouldn’t even be able to understand if such high specific kind of information would help me to get what I want,but it comes from a phrase that you said :

For Nano, the cuda arch is 5.3: -gencode arch=compute_53,code=sm_53
but that will probably not solve the incompatible versions issue.

probably :

when I read this,I think that,in some particular circumstances,that you may know,it could work. If you want to explain what I could try that may work,you will make me happy.