There seems to be a mismatch between the cuda version of jetpack 6.1 (12.6) and the container for r34.6 (12.2).
Hi,
JetPack 6.1 BSP is r36.4.
Please find the container with the corresponding tag.
For example: nvcr.io/nvidia/l4t-cuda:12.6.11-runtime
Thanks.
Hi,
Yes, I meant 36.4, sorry.
Anyway, the problem remains, since in nvcr.io/nvidia/l4t-jetpack:r36.4.0 is installed cuda 12.2 by default.
Hi,
Thanks for reporting this.
Confirmed we can reproduce this issue internally.
We are checking this with our internal team. Will provide more info to you later.
Thanks.
I realize that this is actually the same issue as on my host running the container. I just upgraded an Orin NX J401 Seeed Studio freshly flashed R36.3/JP6.0 to JP 6.1 using instructions provided here, but only for the Upgradable Compute Stack section, not the preceding Upgrade Jetpack section.
Reason I did not apply the preceding Upgrade Jetpack section was that it seemed to be very similar (adding r36.4 to apt sources.list.d/nvidia…) and I was on R36.3 since it says this section can be used for upgrading from R36.3.
In short, I do actually not understand the difference between Upgrade Jetpack and Upgrade Compute Stack. I do realize the first one does a dist-upgrade, which is a major difference (and failing for the Orin NX / J401 carrier board by the way if I actually (second time I tried) run both sections described here.
Leading me to another documentation point where documentation could be clarified: when I do run the Upgrade Jetpack section for the NX on a J401 carrier board I do get the ERROR. Procedure for A_kernel update FAILED. (note: no kernel-dtb but kernel) and also the suggested solution using parted does not help because there is no mmcblk0.
This all as an aside. I was just reacting to imagining the situation of what OP (and I can see this too) sees in the container is highly similar to only executing the Upgrade compute stack section, not the Upgrade Jetpack section.
Hi,
JetPack indicates upgrading to a newer JetPack version.
For compute stack, this is our new feature that support running newer Jetson AI Stack software on the older BSP (ex. r36.3).
Thanks.
Thanks. The Update Computer Stack section does install Jetpack 6.1 too:
Add the R36.4/JP 6.1 repo:
$ echo "deb https://repo.download.nvidia.com/jetson/common r36.4 main" | sudo tee -a /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
$ echo "deb https://repo.download.nvidia.com/jetson/t234 r36.4 main" | sudo tee -a /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
Update the apt repo:
$ sudo apt-get update
Install Jetpack compute components:
$ sudo apt-get install nvidia-jetpack
So it upgrades to a new Jetpack version too, just as the Update Jetpack section (hence the confusion). It just leaves the older BSP intact as you say (36.3).
The Update Jetpack section also updates Jetpack, but updates the BSP too.
Just to add: when only executing the Update Compute Stack section, which will install Jetpack 6.1 and CUDA 12.6 but leave the system at R36.3, Jetpack will self-report as 6.1 in apt show, however nvidia-smi will report as cuda12.2 (and also /usr/local/cuda will (visibly) soft-link to cuda-12.2. However all files in the cuda12.2/include and ./lib64 dirs are actually (correctly), as part of JP 6.1) cuda 12.6 now (the .so files indicate as such). So there’s a mismatch there.
Hi,
Thanks for your patience. We have fixed the version mismatch issue.
Please re-pull the nvcr.io/nvidia/l4t-jetpack:r36.4.0 again.
Thanks.
1 Like
Thanks, yes, can confirm it’s all 12.6 now, only nvidia-smi reports 12.2 still, which is odd?
Hi,
Could you check it again?
nvidia-smi report 12.6 in our environment.
$ sudo docker run -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-jetpack:r36.4.0
root@tegra-ubuntu:/# nvidia-smi
Mon Nov 18 06:42:28 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 540.4.0 Driver Version: 540.4.0 CUDA Version: 12.6 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Orin (nvgpu) N/A | N/A N/A | N/A |
| N/A N/A N/A N/A / N/A | Not Supported | N/A N/A |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
Thanks.
Ok, I know why it reports as 12.2, it’s because the host itself reports it as 12.2. I can only run Update Compute Stack as explained above (the Upgrade Jetpack section including the ‘dist-upgrade’ will fail hard on a J-401 carrier board). So I have a host setup with Jetpack 6.1, and with cuda12.6 include files and libs in /usr/local/cuda/include|lib but still with Nvidia driver 540.3.0 and nvidia-smi reporting cuda 12.2. The docker will report the same thing.
Thanks!
fyi: nvcc reports 12.6 though.
Can I hook on to this question with another one since it relates to this specific container or shall I open a new topic? There is no vulkan.h in /usr/include while in regular flashed Orins there is. What would be the best way to get this since the nvidia repos in the docker are not listed in /etc/apt/sources.list.d? (probably for good reason). When I do add these repos and attempt to install nvidia-l4t-vulkan-sc-dev, it also wants to install nvidia-l4t-core, which fails with the following error:
Building dependency tree...
Reading state information...
The following additional packages will be installed:
nvidia-l4t-core
The following NEW packages will be installed:
nvidia-l4t-core nvidia-l4t-vulkan-sc-dev
0 upgraded, 2 newly installed, 0 to remove and 41 not upgraded.
Need to get 14.9 MB of archives.
After this operation, 41.8 MB of additional disk space will be used.
Get:1 https://repo.download.nvidia.com/jetson/t234 r36.3/main arm64 nvidia-l4t-core arm64 36.3.0-20240719161631 [3679 kB]
Get:2 https://repo.download.nvidia.com/jetson/t234 r36.3/main arm64 nvidia-l4t-vulkan-sc-dev arm64 36.3.0-20240719161631 [11.2 MB]
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:
Fetched 14.9 MB in 6s (2293 kB/s)
(Reading database ... 56775 files and directories currently installed.)
Preparing to unpack .../nvidia-l4t-core_36.3.0-20240719161631_arm64.deb ...
/var/lib/dpkg/tmp.ci/preinst: line 43: /proc/device-tree/compatible: No such file or directory
dpkg: error processing archive /var/cache/apt/archives/nvidia-l4t-core_36.3.0-20240719161631_arm64.deb (--unpack):
new nvidia-l4t-core package pre-installation script subprocess returned error exit status 1
Errors were encountered while processing:
/var/cache/apt/archives/nvidia-l4t-core_36.3.0-20240719161631_arm64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)