Failure to boot with stock kernel when build from source

[robert.a.hilton.jr]

I am having trouble with our Jetson Orin NX 16GB booting after installing the kernel when built from source.

Here are the steps I am taking to build and install the kernel and flash the board:

Steps to build kernel and flash Jetson Orin NX 16GB

#Setup L4T Folder and exports
cd /home/${USER}/JetsonFlashing
sudo tar --same-owner -xvf Jetson_Linux_R36.2.0_aarch64.tbz2
sudo tar --same-owner -xvf Tegra_Linux_Sample-Root-Filesystem_R36.2.0_aarch64.tbz2 --directory Linux_for_Tegra/rootfs/
export TOOLCHAIN_PATH=/home/${USER}/Documents/JetsonFlashing/aarch64--glibc--stable-2022.08-1
export L4T_DIR=/home/${USER}/JetsonFlashing/Linux_for_Tegra

sudo ./apply_binaries.sh

cd ${L4T_DIR}/source




#Without this block in the middle, the board will flash and boot fine (after patching usb-gadget driver on r36.2 release)
#########################################################
#Build Kernel, modules, graphics driver, and dtbs
./source_sync.sh -k -t rel-36_eng_2024-02-05
./generic_rt_build.sh "enable"
export CROSS_COMPILE=${TOOLCHAIN_PATH}/bin/aarch64-buildroot-linux-gnu-
make -C kernel
export INSTALL_MOD_PATH=${L4T_DIR}/rootfs/
sudo -E make install -C kernel
sudo cp -f kernel/kernel-jammy-src/arch/arm64/boot/Image ${L4T_DIR}/kernel/Image
export IGNORE_PREEMPT_RT_PRESENCE=1
export KERNEL_HEADERS=$PWD/kernel/kernel-jammy-src
make modules
sudo -E make modules_install
make dtbs
sudo cp -f nvidia-oot/device-tree/platform/generic-dts/dtbs/* ${L4T_DIR}/kernel/dtb/
#########################################################


sudo FAB=000 BOARDID=3767 BOARDSKU=0000 BOARDREV= FUSELEVEL=1 CHIPREV=  ./build_l4t_bup.sh --single-image-bup A_xusb-fw p3768-0000-p3767-0000-a0 mmcblk0p1
sudo ./l4t_generate_soc_bup.sh -b p3768-0000-p3767-0000-a0 t23x

sudo systemctl stop udisks2.service
sudo service nfs-kernel-server start

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p “-c bootloader/generic/cfg/flash_t234_qspi.xml” --showlogs --network usb0 p3768-0000-p3767-0000-a0-nvme external

sudo service nfs-kernel-server stop
sudo systemctl start udisks2.service

I have attached the boot log from serial console as well as the flash log.
flash_3-6_0_20240210-171406.log (51.5 KB)
bootlog.txt (196.1 KB)

I’m not sure what we’re doing wrong. Our intent is to build the RT kernel, however, it fails to boot with or without patching RT

[robert.a.hilton.jr]

This line is a typo:

sudo ./l4t_generate_soc_bup.sh -b p3768-0000-p3767-0000-a0 t23x

It is supposed to read:

sudo ./l4t_generate_soc_bup.sh -b jetson-orin-nano-devkit t23x

[kayccc]

Sorry for the late response.
Is this still an issue to support? Any result can be shared?

[DaveYYY]

Sorry that I missed this.
Please follow this post to update the initrd for RT kernel, or the driver for NVMe will fail to load because the version of the driver does not match with the kernel image:

JP 6 Flash Issue NVME

YES. We have identified the root cause of this issue. The issue is that for JetPack 5, NVMe and USB drivers are integrated into NVIDIA’s downstream kernel, and built directly into the kernel image; for JetPack 6, in order to be compatible with upstream kernel, a lot of drivers are pulled out of kernel source, and built as kernel modules. Therefore, initrd must contain these drivers for NVMe/USB to be mounted as rootfs during booting, but initrd does not get updated during kernel building/flas…

[robert.a.hilton.jr]

I followed this and unpacked initrd, replaced the drivers in nvme and usb. Then repacked, copied to rootfs/boot and then flashed with initrd flash. I am receiving the same error. Do I need to replace the entire NVME and USB folders in initrd? I only replaced the .ko files that were present in the initrd when I unpacked it.

Please let me know if there is something else I should do

[DaveYYY]

You need to make sure the folder is named 5.15.122-rt-tegra.
It won’t work a folder named 5.15.122-tegra even if the files are updated.

If you don’t know what I’m talking about, or you just cannot get it done, please wait for JetPack 6 GA, which comes with an official solution to this issue.

[robert.a.hilton.jr]

Hi Dave,

Yes, I changed the folder name to include rt and made sure the strings matched in the rootfs/var/modules/… folder. I followed your instructions step by step word for word but still no success.

Thank you,
Rob

[DaveYYY]

Actually you can simply direct copy & paste the entire /lib/modules/5.15.122-rt-tegra from rootfs into initrd, and delete the old one named 5.15.122-tegra.
It is supposed to work this way.