# Jetpack-5.0.2 - Failed to flash NVMe with custom dtb and kernel's Image

**URL:** <https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855>\
**Category:** Jetson Xavier NX\
**Tags:** nvme, reflash, kernel\
**Created:** [January 6, 2023, 12:53pm UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855 "2023-01-06T12:53:13Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![khang.l4es](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/khang.l4es/32/318565_2.png) [@khang.l4es](https://forums.developer.nvidia.com/u/khang.l4es)\
**Post date:** [January 6, 2023, 12:53pm UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/1 "2023-01-06T12:53:13Z")

</div>

Dear Community,

Below are the steps that I was using for flashing to NVMe drive mounted to the carrier board of Jetson Xavier NX :

1. Apply patches and cross-build the customized kernel :  
I followed [the instructions in the Developer Guide](https://docs.nvidia.com/jetson/archives/r35.1/DeveloperGuide/text/SD/Kernel/KernelCustomization.html) to apply my modification to official kernel and successfully built the _Image_, _tegra194-p3668-0001-p3509-0000.dtb_ and _nvgpu.ko_ for my custom carrier board.

Note : I haven’t known yet [how to update the loadable kernel modules with Jetpack-5.0.2](https://forums.developer.nvidia.com/t/jetpack-5-0-2-kernel-customization-how-to-update-loadable-kernel-modules/238818) so I configured them as built-in modules so that they were included in the new Image.

1. Update the target BSP :

```auto
$ cp kernel_out/arch/arm64/boot/Image /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/kernel/ 

$ cp kernel_out/arch/arm64/boot/dts/nvidia/tegra194-p3668-0001-p3509-0000.dtb /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/kernel/dtb/

$ sudo cp kernel_out/drivers/gpu/nvgpu/nvgpu.ko /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/rootfs/usr/lib/modules/5.10.104-tegra/kernel/drivers/gpu/nvgpu/

```

1. Create new system binaries :

```auto
$ cd /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra
$ sudo run apply_binaries.sh

```

1. Generate new system image and flash to target board :  
a. eMMC :

```auto
$ sudo ./flash.sh --no-flash jetson-xavier-nx-devkit-emmc mmcblk0p1
$ sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1

```

The flashing to eMMC was successful, the board rebooted and I selected to boot from eMMC → I could see that custom drivers loaded.

b. NVMe :

```auto
$ sudo ./flash.sh --no-flash jetson-xavier-nx-devkit-emmc nvme0n1p1
$ sudo ./flash.sh jetson-xavier-nx-devkit-emmc nvme0n1p1

```

The flashing said successful, the board rebooted and I selected to boot from NVMe but the older custom system in NVMe still persisted (with older username and password). Of course no custom driver loaded.

I then used the following convenient script : _$ sudo ./nvsdkmanager\_flash.sh --storage nvme0n1p1_, it also finished successfully, the board rebooted and I selected to boot from NVMe (there’s no eMMC anymore, which is the difference compared to using flash.sh command) → The system was updated but with default BSP and required intial setup for Ubuntu as well.

 ![JP502_UEFI_BootManager_Without_eMMC](https://global.discourse-cdn.com/nvidia/original/3X/8/1/814a781d5d20547a440ee914a13f579cc2d821fd.jpeg)  
(No eMMC option in the Boot Manager Menu)

Could you help to point out what was the missing steps, please ?

Best Regards,  
Khang

---

<div class="post-metadata">

**Author:** ![WayneWWW](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/waynewww/32/14043_2.png) [@WayneWWW](https://forums.developer.nvidia.com/u/WayneWWW)\
**Post date:** [January 6, 2023, 1:19pm UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/2 "2023-01-06T13:19:05Z")

</div>

Hi,

please read point 4 in below thread. Though it is for jetson nano, it is basically same.

> [@Boot from external drive](https://forums.developer.nvidia.com/t/boot-from-external-drive/182883):
>
> Hello, I understand that there are lots of people asking about how to boot from external drive (e.g. usb/NVMe drive). And also lots of methods all around this forum. Thanks for everyone to share their steps to achieve that. This post is just to give some basic concepts about how jetson platform boots from external drive. And answer some frequently-asked questions. I believe every method shared on this forum uses the same idea but just in different forms. 1. Jetson is not able to “fully” b…

You need to use initrd\_flash tool instead.

---

<div class="post-metadata">

**Author:** ![khang.l4es](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/khang.l4es/32/318565_2.png) [@khang.l4es](https://forums.developer.nvidia.com/u/khang.l4es)\
**Post date:** [January 6, 2023, 2:07pm UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/3 "2023-01-06T14:07:25Z")

</div>

Hi @WayneWWW ,

Thanks for your prompt reply. By initrd\_flash, do you mean l4t\_initrd\_flash.sh script ? If YES, could you narrow down which workflow for my case if possible? Also, according to the following instruction [Flashing Support — Jetson Linux\<br/\>Developer Guide 34.1 documentation](https://docs.nvidia.com/jetson/archives/r35.1/DeveloperGuide/text/SD/FlashingSupport.html#to-set-up-an-nvme-drive-manually-for-booting), the NVMe SSD needs to be connected to host PC, NOT target device → Is that correct?

Regards,  
K

---

<div class="post-metadata">

**Author:** ![WayneWWW](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/waynewww/32/14043_2.png) [@WayneWWW](https://forums.developer.nvidia.com/u/WayneWWW)\
**Post date:** [January 6, 2023, 2:08pm UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/4 "2023-01-06T14:08:05Z")

</div>

NVMe could be connected to your target board to do the flash too.  
Connecting to host PC is just one of the workflow.

---

<div class="post-metadata">

**Author:** ![khang.l4es](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/khang.l4es/32/318565_2.png) [@khang.l4es](https://forums.developer.nvidia.com/u/khang.l4es)\
**Post date:** [January 6, 2023, 4:00pm UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/5 "2023-01-06T16:00:29Z")

</div>

Hi @WayneWWW ,

From your comment [Initrd flash boot order - #15 by WayneWWW](https://forums.developer.nvidia.com/t/initrd-flash-boot-order/231415/15), it looks like workflow 11 requires that NVMe connected to host PC, isnt it ?

---

<div class="post-metadata">

**Author:** ![khang.l4es](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/khang.l4es/32/318565_2.png) [@khang.l4es](https://forums.developer.nvidia.com/u/khang.l4es)\
**Post date:** [January 6, 2023, 4:08pm UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/6 "2023-01-06T16:08:01Z")

</div>

I also found that maybe the example 2 of workflow 10 close to my case as I also want to boot from the attached NVMe SSD:

Example 2: In this example, you want to boot Jetson Xavier NX SD from an  
attached NVMe SSD. The SD card does not need to be plugged in. You can also  
apply this if you don’t want to use the emmc on the Jetson Xavier NX emmc.

First step: Put the device into recovery mode, then generate qspi only images  
for the internal device:  
$ sudo ./tools/kernel\_flash/l4t\_initrd\_flash.sh --no-flash jetson-xavier-nx-devkit-qspi internal

Note: The board name given here is not jetson-xavier-nx-devkit or  
jetson-xavier-nx-devkit-emmc so that no SD card or eMMC images are generated.

Second step: Put the device into recovery mode, then generate a normal  
filesystem for the external device:  
$ sudo ./tools/kernel\_flash/l4t\_initrd\_flash.sh --no-flash   
–external-device nvme0n1p1   
-S 8GiB -c ./tools/kernel\_flash/flash\_l4t\_nvme.xml   
–external-only --append jetson-xavier-nx-devkit external

Third step: Put the device into recovery mode, then flash both images:  
$ sudo ./tools/kernel\_flash/l4t\_initrd\_flash.sh --flash-only

But I am not sure how to replace jetson-xavier-nx-devkit-qspi in the first step in case of Xavier NX eMMC, maybe jetson-xavier-nx-devkit-emmc-qspi ?

---

<div class="post-metadata">

**Author:** ![khang.l4es](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/khang.l4es/32/318565_2.png) [@khang.l4es](https://forums.developer.nvidia.com/u/khang.l4es)\
**Post date:** [January 7, 2023, 1:07am UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/7 "2023-01-07T01:07:47Z")

</div>

Hi @WayneWWW,

It seems that with example 2 of workflow 10 above, I could boot from NVMe SSD (only storage option to boot from in UEFI) and my custom kernel drivers were also loaded. However, I wonder how to extend to the correct size of the NVMe SSD instead of 8GiB in order to have place for installing other SDK compopents :

 ![NVMe_Mem](https://global.discourse-cdn.com/nvidia/original/3X/7/6/76d33fdd41133639472fe1ccb3a63ab302863b2f.jpeg)

I tried to modify the option -S 200GiB in the step 2 but got following error :

```auto
[2.0700] tegrahost_v2 --chip 0x19 0 --partitionlayout flash.xml.bin --updatesig images_list_signed.xml
[2.0746] tegraparser_v2 --generategpt --pt flash.xml.bin
[2.0760] End sector for APP, expected at: 121208798, actual: 0
Error: Return value 4
Command tegraparser_v2 --generategpt --pt flash.xml.bin
Error: /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/bootloader/signed/flash.idx is not found
Error: failed to relocate images to /workspace/Platforms/Nvidia-Jetson/Software/SDK_downloads/JetPack_5.0.2_Linux_JETSON_XAVIER_NX_TARGETS/Linux_for_Tegra/tools/kernel_flash/images
Cleaning up...

```

Looks like that I need to update the _./tools/kernel\_flash/flash\_l4t\_nvme.xml_, especially for the _num\_sectors_ :

```auto
<partition_layout version="01.00.0000">
    <device type="nvme" instance="0" sector_size="512" num_sectors="121208832">

```

Do you agree or anything else that I need to consider as well ?

Best Regards,  
K

---

<div class="post-metadata">

**Author:** ![khang.l4es](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/khang.l4es/32/318565_2.png) [@khang.l4es](https://forums.developer.nvidia.com/u/khang.l4es)\
**Post date:** [January 8, 2023, 1:19am UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/8 "2023-01-08T01:19:22Z")

</div>

Hi @WayneWWW,

By updating the num\_sectors, I was able to flash the NVMe with desired size. Thanks for your support!

K,

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [January 22, 2023, 1:20am UTC](https://forums.developer.nvidia.com/t/jetpack-5-0-2-failed-to-flash-nvme-with-custom-dtb-and-kernels-image/238855/9 "2023-01-22T01:20:03Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.