Apt-upgrade uploads nvidia-l4t-kernel

Autonomous Machines Jetson & Embedded Systems Jetson Orin NX
1

Hi all,
I’ve recently built a custom L4T RT image starting from jetson_36.4 sources (./source_sync.sh -k -t jetson_36.4) and Jetpack Jetson_Linux_R36.4.0_aarch64.tbz2 and Tegra_Linux_Sample-Root-Filesystem_R36.4.0_aarch64.tbz2.
I followed Kernel Customization — NVIDIA Jetson Linux Developer Guide 1 documentation.

I flashed the image in my Seeedstudio A603 with:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_t234_nvme.xml -p “-c bootloader/generic/cfg/flash_t234_qspi.xml” --showlogs --network usb0 jetson-orin-nano-devkit internal

At the very first boot I can see the kernel installed correctly:

nvidia@nvidia-orin-rt:~$ uname -a
Linux nvidia-orin-rt 5.15.148-rt-tegra #2 SMP PREEMPT_RT Tue Jan 21 17:21:55 +04 2025 aarch64 aarch64 aarch64 GNU/Linux

nvidia@nvidia-orin-rt:~$ cat /etc/nv_tegra_release
#R36 (release), REVISION: 4.0, GCID: 37537400, BOARD: generic, EABI: aarch64, DATE: Fri Sep 13 04:36:44 UTC 2024
#KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

The apt source.list is the following:

nvidia@nvidia-orin-rt:~$ cat /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
#SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#SPDX-License-Identifier: LicenseRef-NvidiaProprietary

#NVIDIA CORPORATION, its affiliates and licensors retain all intellectual
#property and proprietary rights in and to this material, related
documentation and any modifications thereto. Any use, reproduction,
#disclosure or distribution of this material and related documentation
#without an express license agreement from NVIDIA CORPORATION or
#its affiliates is strictly prohibited.

deb https://repo.download.nvidia.com/jetson/common r36.4 main
deb https://repo.download.nvidia.com/jetson/t234 r36.4 main
deb https://repo.download.nvidia.com/jetson/ffmpeg r36.4 main

So why upgrading the packages actually gets 36.4.3 packages as candidates?

nvidia@nvidia-orin-rt:~$ sudo apt update
Hit:1 https://repo.download.nvidia.com/jetson/common r36.4 InRelease
Hit:2 https://repo.download.nvidia.com/jetson/t234 r36.4 InRelease
Hit:3 Index of /ubuntu-ports jammy InRelease
Hit:4 https://repo.download.nvidia.com/jetson/ffmpeg r36.4 InRelease
Hit:5 Index of /ubuntu-ports jammy-updates InRelease
Hit:6 Index of /ubuntu-ports jammy-backports InRelease
Hit:7 Index of /ubuntu-ports jammy-security InRelease
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
255 packages can be upgraded. Run ‘apt list --upgradable’ to see them.
nvidia@nvidia-orin-rt:~$ sudo apt upgrade
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
Calculating upgrade… Done
Get more security updates through Ubuntu Pro with ‘esm-apps’ enabled:
python2.7-minimal libzbar0 libheif1 libiperf0 libjs-jquery-ui libopenexr-dev
libopenexr25 python3-scipy libpostproc55 libswscale-dev libavcodec58 iperf3
libpython2.7 libavutil56 libswscale5 libavutil-dev libswresample3
libavformat58 python2.7 libavformat-dev libavcodec-dev libde265-0
libpython2.7-minimal libpmix2 libpython2.7-stdlib libswresample-dev
libavfilter7
Learn more about Ubuntu Pro at Ubuntu Pro | Ubuntu

#Patches available for rsync vulnerabilities, including a potential RCE,
#tracked by CVE-2024-12084 through CVE-2024-12088 and CVE-2024-12747.
#For more see: https://ubuntu.com/blog/rsync-remote-code-execution

The following NEW packages will be installed:
ubuntu-pro-client ubuntu-pro-client-l10n
The following packages have been kept back:
libopencv-dev
The following packages will be upgraded:
alsa-ucm-conf apparmor apt base-files bind9-host bind9-libs bubblewrap busybox-initramfs busybox-static ca-certificates distro-info-data e2fsprogs fonts-opensymbol ghostscript ghostscript-x gir1.2-gst-plugins-base-1.0 gir1.2-gstreamer-1.0 gir1.2-javascriptcoregtk-4.0
gir1.2-mutter-10 gir1.2-packagekitglib-1.0 gir1.2-soup-2.4 gir1.2-webkit2-4.0 git git-man gjs gnome-control-center gnome-control-center-data gnome-control-center-faces gnome-shell gnome-shell-common gnome-shell-extension-ubuntu-dock gstreamer1.0-alsa gstreamer1.0-gl
gstreamer1.0-gtk3 gstreamer1.0-packagekit gstreamer1.0-plugins-base gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-good gstreamer1.0-pulseaudio gstreamer1.0-tools gstreamer1.0-x ipp-usb language-pack-en language-pack-en-base libapparmor1 libapt-pkg6.0 libarchive13
libcom-err2 libcups2 libcupsfilters1 libcurl3-gnutls libcurl4 libegl-mesa0 libexpat1 libexpat1-dev libext2fs2 libfprint-2-2 libgbm-dev libgbm1 libgd3 libgjs0g libgl1-mesa-dev libgl1-mesa-dri libglapi-mesa libglib2.0-0 libglib2.0-bin libglib2.0-data libglib2.0-dev
libglib2.0-dev-bin libglib2.0-doc libglib2.0-tests libglx-mesa0 libgs9 libgs9-common libgsf-1-114 libgsf-1-common libgstreamer-gl1.0-0 libgstreamer-plugins-base1.0-0 libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1.0-0 libgstreamer-plugins-good1.0-dev
libgstreamer1.0-0 libgstreamer1.0-dev libipa-hbac0 libjavascriptcoregtk-4.0-18 libmm-glib0 libmpg123-0 libmutter-10-0 libmysqlclient21 libnss-sss libopenjp2-7 liborc-0.4-0 liborc-0.4-dev liborc-0.4-dev-bin libpackagekit-glib2-18 libpam-sss libpcap0.8 libpoppler-glib8
libpoppler118 libpq5 libpython2.7 libpython2.7-minimal libpython2.7-stdlib libpython3-dev libpython3-stdlib libpython3.10 libpython3.10-dev libpython3.10-minimal libpython3.10-stdlib librados2 librbd1 libreoffice-base-core libreoffice-calc libreoffice-common
libreoffice-core libreoffice-draw libreoffice-gnome libreoffice-gtk3 libreoffice-impress libreoffice-math libreoffice-pdfimport libreoffice-style-breeze libreoffice-style-colibre libreoffice-style-elementary libreoffice-style-yaru libreoffice-writer libsoup-gnome2.4-1
libsoup2.4-1 libsoup2.4-common libss2 libssl3 libsss-certmap0 libsss-idmap0 libsss-nss-idmap0 libsysmetrics1 libtiff-dev libtiff5 libtiffxx5 libuno-cppu3 libuno-cppuhelpergcc3-3 libuno-purpenvhelpergcc3-3 libuno-sal3 libuno-salhelpergcc3-3 libwebkit2gtk-4.0-37
linux-firmware linux-libc-dev logsave modemmanager mutter-common nvidia-l4t-3d-core nvidia-l4t-apt-source nvidia-l4t-bootloader nvidia-l4t-camera nvidia-l4t-configs nvidia-l4t-core nvidia-l4t-cuda nvidia-l4t-cuda-utils nvidia-l4t-display-kernel nvidia-l4t-firmware
nvidia-l4t-gbm nvidia-l4t-graphics-demos nvidia-l4t-init nvidia-l4t-initrd nvidia-l4t-jetson-io nvidia-l4t-jetsonpower-gui-tools nvidia-l4t-kernel nvidia-l4t-kernel-dtbs nvidia-l4t-kernel-headers nvidia-l4t-kernel-oot-headers nvidia-l4t-kernel-oot-modules
nvidia-l4t-libwayland-client0 nvidia-l4t-libwayland-cursor0 nvidia-l4t-libwayland-egl1 nvidia-l4t-libwayland-server0 nvidia-l4t-multimedia nvidia-l4t-multimedia-utils nvidia-l4t-nvfancontrol nvidia-l4t-nvml nvidia-l4t-nvpmodel nvidia-l4t-nvpmodel-gui-tools
nvidia-l4t-nvsci nvidia-l4t-oem-config nvidia-l4t-openwfd nvidia-l4t-optee nvidia-l4t-pva nvidia-l4t-tools nvidia-l4t-vulkan-sc nvidia-l4t-vulkan-sc-dev nvidia-l4t-vulkan-sc-samples nvidia-l4t-vulkan-sc-sdk nvidia-l4t-wayland nvidia-l4t-weston nvidia-l4t-x11
nvidia-l4t-xusb-firmware nvme-cli openssl openvpn packagekit packagekit-tools poppler-utils python-apt-common python2.7 python2.7-minimal python3 python3-apt python3-dev python3-distupgrade python3-minimal python3-paramiko python3-pkg-resources python3-sss python3-uno
python3-update-manager python3-urllib3 python3.10 python3.10-dev python3.10-minimal qemu-efi qemu-efi-aarch64 rsync snapd sssd sssd-ad sssd-ad-common sssd-common sssd-ipa sssd-krb5 sssd-krb5-common sssd-ldap sssd-proxy thunderbird thunderbird-gnome-support
ubuntu-advantage-desktop-daemon ubuntu-advantage-tools ubuntu-desktop ubuntu-desktop-minimal ubuntu-drivers-common ubuntu-release-upgrader-core ubuntu-release-upgrader-gtk ubuntu-report uno-libs-private update-manager update-manager-core ure vim vim-common vim-runtime
xdg-desktop-portal xserver-common xserver-xephyr xserver-xorg-core xserver-xorg-legacy xwayland xxd
254 upgraded, 2 newly installed, 0 to remove and 1 not upgraded.
131 standard LTS security updates
Need to get 1,040 MB of archives.
After this operation, 62.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] ^C

This is in conflict with what you write here: Software Packages and the Update Mechanism — NVIDIA Jetson Linux Developer Guide 1 documentation
I expect I could stay with 6.1 if I don’t change any release version in my apt sources.

Could you tell me why this is happening? Tnks

4

Hi,
The guidance shows steps for upgrading from 36.3 to r36.4.

And r36.4.3 is later version of r36.4. If you run $ sudo apt update, the later version will be shown. To stay on r36.4, please have a script like:
OpenGL issue after install nvidia-jetpack via apt-get - #17 by DaneLLL
Compiz high load (180%) - #12 by DaneLLL

To install specific vesion.

5

Hi @DaneLLL,
The working version IS already installed. 36.4.0-20240912212859

nvidia@nvidia-orin-rt:~$ sudo apt search nvidia-l4t-jetson-multimedia-api
[sudo] password for nvidia:
Sorting… Done
Full Text Search… Done
nvidia-l4t-jetson-multimedia-api/stable 36.4.3-20250107174145 arm64 [upgradable from: 36.4.0-20240912212859]
NVIDIA Jetson Multimedia API is a collection of lower-level APIs that support flexible application development.

nvidia@nvidia-orin-rt:~$ sudo apt install nvidia-l4t-jetson-multimedia-api=36.4.0-20240912212859
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
nvidia-l4t-jetson-multimedia-api is already the newest version (36.4.0-20240912212859).
0 upgraded, 0 newly installed, 0 to remove and 268 not upgraded.

The issue is with the upgrade of the system that wants to upgrade it.
Is removing the nvidia repositories the only solution to stay with 36.4.0 instead of 36.4.3?

6

I’ll tell you more.
If I leave the sudo apt upgrade to update the kernel, the following boot will select 5.15.148-tegra NOT realtime.
Even following Kernel Customization — NVIDIA Jetson Linux Developer Guide 1 documentation next, the RT packages are related the old version 5.15.148-rt-tegra-36.4.0-20241014173246.

nvidia@nvidia-orin-rt:~$ cat /etc/nv_tegra_release
#R36 (release), REVISION: 4.3, GCID: 38968081, BOARD: generic, EABI: aarch64, DATE: Wed Jan 8 01:49:37 UTC 2025
#KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

nvidia@nvidia-orin-rt:~$ uname -a
Linux nvidia-orin-rt 5.15.148-tegra #1 SMP PREEMPT Tue Jan 7 17:14:38 PST 2025 aarch64 aarch64 aarch64 GNU/Linux
nvidia@nvidia-orin-rt:~$

nvidia@nvidia-orin-rt:~$ sudo apt install nvidia-l4t-rt-kernel nvidia-l4t-rt-kernel-headers nvidia-l4t-rt-kernel-oot-modules nvidia-l4t-display-rt-kernel
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
The following NEW packages will be installed:
nvidia-l4t-display-rt-kernel nvidia-l4t-rt-kernel nvidia-l4t-rt-kernel-headers
nvidia-l4t-rt-kernel-oot-modules
0 upgraded, 4 newly installed, 0 to remove and 1 not upgraded.
Need to get 52.7 MB of archives.
After this operation, 197 MB of additional disk space will be used.
Get:1 https://repo.download.nvidia.com/jetson/rt-kernel r36.4/main arm64 nvidia-l4t-rt-kernel arm64 5.15.148-rt-tegra-36.4.0-20241014173246 [29.3 MB]
Get:2 https://repo.download.nvidia.com/jetson/rt-kernel r36.4/main arm64 nvidia-l4t-display-rt-kernel arm64 5.15.148-rt-tegra-36.4.0-20241014173246 [1,174 kB]
Get:3 https://repo.download.nvidia.com/jetson/rt-kernel r36.4/main arm64 nvidia-l4t-rt-kernel-headers arm64 5.15.148-rt-tegra-36.4.0-20241014173246 [17.8 MB]
Get:4 https://repo.download.nvidia.com/jetson/rt-kernel r36.4/main arm64 nvidia-l4t-rt-kernel-oot-modules arm64 5.15.148-rt-tegra-36.4.0-20241014173246 [4,382 kB]
…

Rebooting after installing RT by OTA, prevent the correct booting with the error bash: cannot set terminal process group (-1): Inappropriate ioctl for device:
boot_rt_36.4.3.log (70.0 KB)

7

Hi,
Not understand this part and would like to get more information. If you have flashed the device with your self-built RT kernel, why do you run the command:

nvidia@nvidia-orin-rt:~$ sudo apt install nvidia-l4t-rt-kernel nvidia-l4t-rt-kernel-headers nvidia-l4t-rt-kernel-oot-modules nvidia-l4t-display-rt-kernel

You should have built these and flashed the self-built packages to the system.

8

Sure @DaneLLL
I initially flashed a self-built RT kernel (generic_rt_build.sh "enable") with 6.1 sources and custom carrier board’s drivers and my customization.
Unfortunately, after upgrading the packages, the orin NX boots into a non-rt kernel. I’m not sure if my old kernel is overwritten or what. As you can see from the second uname -a, it’s not PREEMPT_RT anymore.

I thought, probably wrongly, that reinstalling the rt debs could restore the RT kernel, updated with the new 36.4.3 packages. Actually installing these breaks the boot, so I guess it’s not a doable way.

9

I recently found another topic on Orin AGX where they faced the same issue: JetPack6.1 auto upgrade
Unfortunately there’s no solution even there

10

Hi,
We would like to set up developer kit and replicate the issue. Please check if the steps are correct:

  1. Flash developer kit to RT kernel
  2. Boot the system and run $ sudo apt update
  3. Reboot the system. After booting up, the system becomes non-RT kernel
11

Almost correct.

  1. Flash devkit with a RT kernel syncing the sources from jetson_36.4 and using Jetson_Linux_R36.4.0 zip archives.
    (In my case I’m using a Seeedstudio carrier board but the issue shall be in the devkit as well).
  2. Boot the system and run sudo apt update && sudo apt upgrade
  3. Reboot the system. It boots into Linux nvidia-orin-rt 5.15.148-tegra which is not RT anymore