Applying a PREEMPT-RT patch to JetPack 4.5 on Jetson Nano

So a while ago I found this instruction to compile a JetPack 4.3 Linux kernel with application of the PREEMPT-RT patches, but encountered an issue which I posted about in the thread, but got no response whatsoever.

This instruction is literally the only “step 1, step 2, profit” kind of guide on the topic that I was able to find, so what I’m asking is how should it be altered so that it applies to JetPack 4.5? Or, alternatively, what might have I done wrong and how could I make the 4.3 guide work, assuming I followed the instruction word for word on a clean Ubuntu 18.04 LTS install on the host machine?

Hi @trueshadow97, I’m the author of the 4.3 guide. During the day I will post here an updated version of the guide to work with version 4.5. I hope I can help you to make it work this time. Best regards.

Yeah, I remember your nickname. Glad to see you around, and thank you in advance!

Hi @trueshadow97, this is the guide updated for JetPack 4.5 (L4T 32.5):

Real-time Linux for Jetson Nano (L4T 32.5)

Install required packages

sudo apt-get update
sudo apt-get install libncurses5-dev
sudo apt-get install build-essential
sudo apt-get install bc
sudo apt-get install lbzip2
sudo apt-get install qemu-user-static
sudo apt-get install python

Create folder

mkdir $HOME/jetson_nano
cd $HOME/jetson_nano

Download required files

L4T Jetson Driver Package:

wget https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/T210/Tegra210_Linux_R32.5.0_aarch64.tbz2

L4T Sample Root File System:

wget https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/T210/Tegra_Linux_Sample-Root-Filesystem_R32.5.0_aarch64.tbz2

L4T Sources:

wget https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/sources/T210/public_sources.tbz2

GCC Tool Chain for 64-bit BSP:

wget http://releases.linaro.org/components/toolchain/binaries/7.3-2018.05/aarch64-linux-gnu/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz

Extract files

sudo tar xpf Tegra210_Linux_R32.5.0_aarch64.tbz2
cd Linux_for_Tegra/rootfs/
sudo tar xpf ../../Tegra_Linux_Sample-Root-Filesystem_R32.5.0_aarch64.tbz2
cd ../../
tar -xvf gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz
sudo tar -xjf public_sources.tbz2
tar -xjf Linux_for_Tegra/source/public/kernel_src.tbz2

Apply real-time patches

cd kernel/kernel-4.9/
./scripts/rt-patch.sh apply-patches

Configure and compile the kernel

TEGRA_KERNEL_OUT=jetson_nano_kernel
mkdir $TEGRA_KERNEL_OUT
export CROSS_COMPILE=$HOME/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_defconfig
make ARCH=arm64 O=$TEGRA_KERNEL_OUT menuconfig

General setup → Timer subsystem → Timer tick handling → Full dynticks system (tickless)
Kernel Features → Preemption Model: Fully Preemptible Kernel (RT)
Kernel Features → Timer frequency: 1000 HZ

make ARCH=arm64 O=$TEGRA_KERNEL_OUT -j4

sudo cp jetson_nano_kernel/arch/arm64/boot/Image $HOME/jetson_nano/Linux_for_Tegra/kernel/Image
sudo cp -r jetson_nano_kernel/arch/arm64/boot/dts/* $HOME/jetson_nano/Linux_for_Tegra/kernel/dtb/
sudo make ARCH=arm64 O=$TEGRA_KERNEL_OUT modules_install INSTALL_MOD_PATH=$HOME/jetson_nano/Linux_for_Tegra/rootfs/

cd $HOME/jetson_nano/Linux_for_Tegra/rootfs/
sudo tar --owner root --group root -cjf kernel_supplements.tbz2 lib/modules
sudo mv kernel_supplements.tbz2  ../kernel/

cd ..
sudo ./apply_binaries.sh

Generate Jetson Nano image

cd tools
sudo ./jetson-disk-image-creator.sh -o jetson_nano.img -b jetson-nano -r 300

For Jetson Nano 2GB version use -b jetson-nano-2gb-devkit

The -r 300 switch is the revision number of the Jetson Nano module to be used:
100 for revision A01
200 for revision A02
300 for revision B00 or B01
Nothing for Jetson Nano 2GB or Jetson Xavier NX (do not use the -r switch)

Reference: https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/flashing.html#wwpID0E0TJ0HA

The resulting image is: $HOME/jetson_nano/Linux_for_Tegra/tools/jetson_nano.img

Flash the image to the micro SD card using Etcher
Use the micro SD to boot the Nano with monitor, mouse and keyboard
Follow the steps for first boot setup
To finish, install CUDA and the rest of Jetson SDK components using SDK manager

I successfully generated a working image for my Jetson Nano following these instructions. Please let me know if you have any issues.

Best regards

Thanks a lot! I’ll try this out and get back to you if any issues arise.

On step make ARCH=arm64 O=$TEGRA_KERNEL_OUT -j4, I get the following error:

make[1]: Entering directory '/home/shadow97/kernel/kernel-4.9/jetson_nano_kernel'
../scripts/gcc-version.sh: line 25: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: No such file or directory
../scripts/gcc-version.sh: line 26: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: No such file or directory
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  GEN     ./Makefile
scripts/kconfig/conf  --silentoldconfig Kconfig
#
# configuration written to .config
#
arch/arm64/Makefile:49: LSE atomics not supported by binutils
../scripts/gcc-version.sh: line 25: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: No such file or directory
../scripts/gcc-version.sh: line 26: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: No such file or directory
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  CHK     include/config/kernel.release
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  GEN     ./Makefile
  WRAP    arch/arm64/include/generated/asm/bugs.h
  WRAP    arch/arm64/include/generated/asm/clkdev.h
  WRAP    arch/arm64/include/generated/asm/cputime.h
  WRAP    arch/arm64/include/generated/asm/delay.h
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  WRAP    arch/arm64/include/generated/asm/div64.h
  WRAP    arch/arm64/include/generated/asm/dma.h
  WRAP    arch/arm64/include/generated/asm/dma-contiguous.h
  WRAP    arch/arm64/include/generated/asm/early_ioremap.h
  WRAP    arch/arm64/include/generated/asm/errno.h
  WRAP    arch/arm64/include/generated/asm/emergency-restart.h
  CHK     include/generated/uapi/linux/version.h
  WRAP    arch/arm64/include/generated/asm/hw_irq.h
  UPD     include/generated/uapi/linux/version.h
  WRAP    arch/arm64/include/generated/asm/ioctl.h
  UPD     include/config/kernel.release
  WRAP    arch/arm64/include/generated/asm/ioctls.h
  WRAP    arch/arm64/include/generated/asm/ipcbuf.h
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  WRAP    arch/arm64/include/generated/asm/irq_regs.h
  WRAP    arch/arm64/include/generated/asm/kdebug.h
  WRAP    arch/arm64/include/generated/asm/kmap_types.h
  WRAP    arch/arm64/include/generated/asm/kvm_para.h
  WRAP    arch/arm64/include/generated/asm/local.h
  WRAP    arch/arm64/include/generated/asm/local64.h
  WRAP    arch/arm64/include/generated/asm/mcs_spinlock.h
  WRAP    arch/arm64/include/generated/asm/mm-arch-hooks.h
  WRAP    arch/arm64/include/generated/asm/mman.h
  WRAP    arch/arm64/include/generated/asm/msgbuf.h
  WRAP    arch/arm64/include/generated/asm/msi.h
  WRAP    arch/arm64/include/generated/asm/mutex.h
  WRAP    arch/arm64/include/generated/asm/poll.h
  WRAP    arch/arm64/include/generated/asm/preempt.h
  WRAP    arch/arm64/include/generated/asm/resource.h
  WRAP    arch/arm64/include/generated/asm/rwsem.h
  WRAP    arch/arm64/include/generated/asm/segment.h
  WRAP    arch/arm64/include/generated/asm/sembuf.h
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  WRAP    arch/arm64/include/generated/asm/shmbuf.h
  WRAP    arch/arm64/include/generated/asm/serial.h
  WRAP    arch/arm64/include/generated/asm/sizes.h
  WRAP    arch/arm64/include/generated/asm/simd.h
  WRAP    arch/arm64/include/generated/asm/socket.h
  WRAP    arch/arm64/include/generated/asm/sockios.h
  WRAP    arch/arm64/include/generated/asm/swab.h
  HOSTCC  scripts/basic/bin2c
  WRAP    arch/arm64/include/generated/asm/switch_to.h
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  WRAP    arch/arm64/include/generated/asm/termbits.h
  WRAP    arch/arm64/include/generated/asm/termios.h
  WRAP    arch/arm64/include/generated/asm/trace_clock.h
  WRAP    arch/arm64/include/generated/asm/types.h
  WRAP    arch/arm64/include/generated/asm/unaligned.h
  WRAP    arch/arm64/include/generated/asm/user.h
  WRAP    arch/arm64/include/generated/asm/vga.h
  WRAP    arch/arm64/include/generated/asm/xor.h
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  WRAP    arch/arm64/include/generated/uapi/asm/kvm_para.h
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  Using .. as source for kernel
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  CHK     include/generated/utsrelease.h
  UPD     include/generated/utsrelease.h
make[1]: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: Command not found
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/genksyms/genksyms.o
  SHIPPED scripts/genksyms/parse.tab.c
  SHIPPED scripts/genksyms/lex.lex.c
  SHIPPED scripts/genksyms/keywords.hash.c
  SHIPPED scripts/genksyms/parse.tab.h
  HOSTCC  scripts/genksyms/parse.tab.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/genksyms/lex.lex.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  SHIPPED scripts/dtc/dtc-lexer.lex.c
  SHIPPED scripts/dtc/dtc-parser.tab.h
  SHIPPED scripts/dtc/dtc-parser.tab.c
  HOSTLD  scripts/genksyms/genksyms
  CC      scripts/mod/empty.o
/bin/sh: 1: /home/shadow97/jetson_nano/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc: not found
../scripts/Makefile.build:335: recipe for target 'scripts/mod/empty.o' failed
make[3]: *** [scripts/mod/empty.o] Error 127
../scripts/Makefile.build:649: recipe for target 'scripts/mod' failed
make[2]: *** [scripts/mod] Error 2
make[2]: *** Waiting for unfinished jobs....
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTLD  scripts/dtc/dtc
/home/shadow97/kernel/kernel-4.9/Makefile:593: recipe for target 'scripts' failed
make[1]: *** [scripts] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/home/shadow97/kernel/kernel-4.9/jetson_nano_kernel'
Makefile:171: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2

All the steps leading up to this went on without errors. After performing the necessary configurations in make ARCH=arm64 O=$TEGRA_KERNEL_OUT menuconfig, I didn’t save the kernel configuration as a separate file, but instead exitted the script with saving the changes. Maybe that was the reason for the issue, but what do I save the config file as if that’s the case?

Hi @trueshadow97, after changing options in menuconfig you just need to select save and then exit. It should work if you do that.

The separate <Save> option prompted me to enter the config file name. Instead, I chose <Exit>, and, upon being prompted whether I would like to save the changes to configuration, I chose Yes. Was that wrong? I might not have noticed another options.

Oh, I understand. I always just let the default config file name and save. Since that is something I always do without thinking, I am not familiar with what happens when the exit option is selected. It’s weird, but maybe the file was not correctly saved. Please, try to repeat the process saving the configuration to see if that solves the problem.

Best regards

The default config file name is literally .config, as far as my memory serves me. Should I try saving it, even if under that name?

Yes, the .config file name is correct. Save it using that name.

Thanks for the advice, I’ll try that out when I can. For whatever reason, I kept thinking that “config” is the file extension rather than the name of a file that’s supposed to be hidden (hence the dot in the beginning) to begin with.

Ok, I was able to compile the RT image and successfully boot from it. Thank you kindly for the help! I don’t know, though, if 66us is within correct max latency range when stress-tested with hdparm -t /dev/mmcblk0.

sudo chrt -f 99 ./cyclictest -t1 -p 80 -i 10000 -n -l 100000
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 2.84 1.96 1.49 2/862 10515           

T: 0 (10461) P:80 I:10000 C:   4138 Min:      8 Act:   28 Avg:   21 Max:      66

Hi @trueshadow97, I’m glad it’s working now. 66us seems correct to me, taking into account you are running a stress load. You can also try running this latency script without stress:
https://www.osadl.org/Create-a-latency-plot-from-cyclictest-hi.bash-script-for-latency-plot.0.html

The script runs for 5 hours and 33 minutes, so it’s a good way to know the latencies you can expect in your applications. You need to install gnuplot to be able to generate the latency plot. According to my experience with the Nano, you should get maximum latencies around 50us, that’s the best real-time behavior I have seen with rt patches in the Nano.

Best regards

Thanks again, I’ll try that out.