Jetpack-5.0.2 - Failed to flash NVMe with custom dtb and kernel's Image

[khang.l4es]

Dear Community,

Below are the steps that I was using for flashing to NVMe drive mounted to the carrier board of Jetson Xavier NX :

1. Apply patches and cross-build the customized kernel :
   I followed the instructions in the Developer Guide to apply my modification to official kernel and successfully built the Image, tegra194-p3668-0001-p3509-0000.dtb and nvgpu.ko for my custom carrier board.

Note : I haven’t known yet how to update the loadable kernel modules with Jetpack-5.0.2 so I configured them as built-in modules so that they were included in the new Image.

2. Update the target BSP :

$ cp kernel_out/arch/arm64/boot/Image /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/kernel/ 

$ cp kernel_out/arch/arm64/boot/dts/nvidia/tegra194-p3668-0001-p3509-0000.dtb /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/kernel/dtb/

$ sudo cp kernel_out/drivers/gpu/nvgpu/nvgpu.ko /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/rootfs/usr/lib/modules/5.10.104-tegra/kernel/drivers/gpu/nvgpu/

3. Create new system binaries :

$ cd  /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra
$ sudo  run apply_binaries.sh

4. Generate new system image and flash to target board :
   a. eMMC :

$  sudo ./flash.sh --no-flash jetson-xavier-nx-devkit-emmc mmcblk0p1
$  sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1

The flashing to eMMC was successful, the board rebooted and I selected to boot from eMMC → I could see that custom drivers loaded.

b. NVMe :

$  sudo ./flash.sh --no-flash jetson-xavier-nx-devkit-emmc nvme0n1p1
$  sudo ./flash.sh jetson-xavier-nx-devkit-emmc nvme0n1p1

The flashing said successful, the board rebooted and I selected to boot from NVMe but the older custom system in NVMe still persisted (with older username and password). Of course no custom driver loaded.

I then used the following convenient script : $ sudo ./nvsdkmanager_flash.sh --storage nvme0n1p1, it also finished successfully, the board rebooted and I selected to boot from NVMe (there’s no eMMC anymore, which is the difference compared to using flash.sh command) → The system was updated but with default BSP and required intial setup for Ubuntu as well.

(No eMMC option in the Boot Manager Menu)

Could you help to point out what was the missing steps, please ?

Best Regards,
Khang

[WayneWWW]

Hi,

please read point 4 in below thread. Though it is for jetson nano, it is basically same.

Boot from external drive

Hello, I understand that there are lots of people asking about how to boot from external drive (e.g. usb/NVMe drive). And also lots of methods all around this forum. Thanks for everyone to share their steps to achieve that. This post is just to give some basic concepts about how jetson platform boots from external drive. And answer some frequently-asked questions. I believe every method shared on this forum uses the same idea but just in different forms. 1. Jetson is not able to “fully” b…

You need to use initrd_flash tool instead.

[khang.l4es]

Hi @WayneWWW ,

Thanks for your prompt reply. By initrd_flash, do you mean l4t_initrd_flash.sh script ? If YES, could you narrow down which workflow for my case if possible? Also, according to the following instruction Flashing Support — Jetson Linux<br/>Developer Guide 34.1 documentation, the NVMe SSD needs to be connected to host PC, NOT target device → Is that correct?

Regards,
K

[WayneWWW]

NVMe could be connected to your target board to do the flash too.
Connecting to host PC is just one of the workflow.

[khang.l4es]

Hi @WayneWWW ,

From your comment Initrd flash boot order - #15 by WayneWWW, it looks like workflow 11 requires that NVMe connected to host PC, isnt it ?

[khang.l4es]

I also found that maybe the example 2 of workflow 10 close to my case as I also want to boot from the attached NVMe SSD:

Example 2: In this example, you want to boot Jetson Xavier NX SD from an
attached NVMe SSD. The SD card does not need to be plugged in. You can also
apply this if you don’t want to use the emmc on the Jetson Xavier NX emmc.

First step: Put the device into recovery mode, then generate qspi only images
for the internal device:
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash jetson-xavier-nx-devkit-qspi internal

Note: The board name given here is not jetson-xavier-nx-devkit or
jetson-xavier-nx-devkit-emmc so that no SD card or eMMC images are generated.

Second step: Put the device into recovery mode, then generate a normal
filesystem for the external device:
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash
–external-device nvme0n1p1
-S 8GiB -c ./tools/kernel_flash/flash_l4t_nvme.xml
–external-only --append jetson-xavier-nx-devkit external

Third step: Put the device into recovery mode, then flash both images:
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only

But I am not sure how to replace jetson-xavier-nx-devkit-qspi in the first step in case of Xavier NX eMMC, maybe jetson-xavier-nx-devkit-emmc-qspi ?

[khang.l4es]

Hi @WayneWWW,

It seems that with example 2 of workflow 10 above, I could boot from NVMe SSD (only storage option to boot from in UEFI) and my custom kernel drivers were also loaded. However, I wonder how to extend to the correct size of the NVMe SSD instead of 8GiB in order to have place for installing other SDK compopents :

I tried to modify the option -S 200GiB in the step 2 but got following error :

[   2.0700 ] tegrahost_v2 --chip 0x19 0 --partitionlayout flash.xml.bin --updatesig images_list_signed.xml
[   2.0746 ] tegraparser_v2 --generategpt --pt flash.xml.bin
[   2.0760 ] End sector for APP, expected at: 121208798, actual: 0
Error: Return value 4
Command tegraparser_v2 --generategpt --pt flash.xml.bin
Error: /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/bootloader/signed/flash.idx is not found
Error: failed to relocate images to /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/tools/kernel_flash/images
Cleaning up...

Looks like that I need to update the ./tools/kernel_flash/flash_l4t_nvme.xml, especially for the num_sectors :

<partition_layout version="01.00.0000">
    <device type="nvme" instance="0" sector_size="512" num_sectors="121208832">

Do you agree or anything else that I need to consider as well ?

Best Regards,
K

[khang.l4es]

Hi @WayneWWW,

By updating the num_sectors, I was able to flash the NVMe with desired size. Thanks for your support!

K,