Create custom nvme image for Jetson

Dear all,

I am currently using the Jetpack 5.0 to flash a product grade Jetson Xavier (jetson-xavier-devkit-emmc). Therefore, I am putting the Jetson into recovery mode and run following command from the host after I ran the apply_binaries script:

sudo ./nvsdkmanager_flash.sh --storage nvme0n1p1

The Jetson is booting up as expected and I can setup my required environment on the Jetson. However, I want to deploy the same setup to multiple emmc-Jetsons and struggle with creating an .img file from the nvme memory. I know that the flash.sh scripts provide a function to create an image from the onboard emmc which is unfortunately not working for the nvme (tried it already multiple times).

My questions are:

  1. How can I create an img file from a custom nvme setup?
  2. How can I flash this image on the another emmc-Jetson?

I appreciate any helpful comments or workarounds to make this work out. If anyone else is setting up multiple emmc-Jetsons with nvme memory, please reach out to me too, I would love to hear your experience on the setup process.

Best,
Julia

Please refer to the initrd flash. You can all use initrd flash to flash nvme and emmc. Copy the image from nvme to replace the one when flashing emmc.

https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3261/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/flashing.html#wwpID0E0PI0HA

Sorry for the misunderstanding. I do not want to flash the emmc. I only want to flash the nvme of another Jetson with the custom image.

I also managed to flash the nvme with the original image with this command, following the nvsdkmanager_flash script:

sudo NO_ROOTFS=0 ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml --showlogs jetson-xavier-nx-devkit-qspi internal

Could you please send me the command to download the image from the nvme? I could not figure it out until now. The flash.sh script can only download the image from the emmc but not from the nvme. I have not yet found the command within the initrd scripts to download the image from nvme.

You can use img2simg /dev/nvme0n1p1 system.img.ext to clone out a backup image.

Thank you for your response. Just to be clear, you recommend me to run:

img2simg /dev/nvme01n1p1 system.img.ext

on the Jetson board and store the file on an external memory like an USB key. Now I have the image file.

What is the next step when I want to flash this system.img.ext to a new Jetson. Which command do I need to run then?

You can replace that one under your initrd flash created image. Run --flash-only in initrd flash so that it will just use this image to flash.

Thus you are suggesting to replace the file system.img (and its hash file?) in the folder Linux_for_Tegra/tools/kernel_flash/images/external/ with the system.img.ext which I just copied from the board and then run which command?

 sudo ./nvsdkmanager_flash.sh --storage nvme0n1p1 --flash-only

or do I need to run the tools/kernel_flash/l4t_initrd_flash.sh script with specific flags only to flash the APP partition?

Could you read the initrd_flash readme file first? I am not sure why you insist using nvsdkmanager_flash.sh.

What I am talking about are all using initrd flash. Not nvsdkmanager_flash.sh.

I am really sorry, if I have upset you. But believe me I would not have written a post here, if I would have found the information I was looking for in the ReadMe. And as customer, who is about to setup a complete robot fleet with Jetsons, I am asking for help finding the right command, since I have tried out quite a few steps already that simply have not worked out.

Thus when using the initrd script, how is the correct command to flash it to the nvme memory? Could you please provide me one example that you have run and can verify that it works?

Would it look like something similar to this?

 sudo ./tools/kernel_flash/l4t_initrd_flash.sh \
  --external-device nvme0n1p1 \
  -c ./tools/kernel_flash/flash_l4t_nvme.xml \
   --external-only --flash-only jetson-xavier-devkit-emmc nvme0n1p1

or do I need to specify the APP partition with the -k flag?

The readme has the workflow for the external driver case. Try with the basic one first. See what is generated from the command, then you shall find the image file which you could replace.

After replace, --flash-only would be in use so that initrd flash won’t generate new image, but will directly flash the file replaced by you.

Thus, I tried the falshing process once more as described in Workflow 10: Generate images for internal device and external device seperately then flash- Example 2: In this example, you want to boot Jetson Xavier NX SD from an attached NVMe SSD. The SD card does not need to be plugged in. You can also apply this if you don't want to use the emmc on the Jetson Xavier NX emmc.

I ran following commands to initially flash the the Jetson. First, downloading the Tegra_Linux_Sample-Root-Filesystem_R34.1.0_aarch64.tbz2 and Jetson_Linux_R34.1.0_aarch64.tbz2 package and extracting both of them:

sudo tar xpf Jetson_Linux_R34.1.0_aarch64.tbz2
cd Linux_for_Tegra/rootfs
sudo tar xpf ../../Tegra_Linux_Sample-Root-Filesystem_R34.1.0_aarch64.tbz2

Then for the first time applying the binaries in the Linux_for_Tegra folder:

sudo ./apply_binaries.sh

Next, following the Workflow 10 - Example 2. (Please note that I had to use the flash_l4t_external.xml because the one suggested in the ReadMe was not working for me. Then the Jetson did not boot up)

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash jetson-xavier-nx-devkit-qspi internal
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash \
            --external-device nvme0n1p1 \
            -c ./tools/kernel_flash/flash_l4t_external.xml \
            --external-only --append jetson-xavier-nx-devkit external

Then I put the device into the recovery mode and flashed the image:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --showlogs

So far, so good. Now the Jetson is booting up and I can configure the Ubuntu 20.04 system and install custom packages. Once I configured everything I took a look at the partitions with gparted and the look well set:

After that I mounted a usb key and stored the system image with the img2simg command as you suggested:

sudo img2simg /dev/nvme0n1p1 system.img.ext

The original system.img in Linux_for_Tegra/tools/kernel_flash/images/external has 5.04GB and the one I created 5.5GB. Thus everything looks good. For the test now, I flashed once more the original image and setup the Ubuntu tocheck, if the new image is really flashed.

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash \
            --external-device nvme0n1p1 \
            -c ./tools/kernel_flash/flash_l4t_external.xml \
            --external-only --append jetson-xavier-nx-devkit external

Now I remove the original system.img in Linux_for_Tegra/tools/kernel_flash/images/external and the system.img.sha1sum file:

sudo rm system.img*

Rename system.img.ext to system.img and create a new sha1sum file:

mv system.img.ext system.img
sha1sum system.img > system.img.sha1sum

Then both files are copied to the external image folder:

sudo cp system.img* Linux_for_Tegra/tools/kernel_flash/images/external/

And now I tried flash the Jetson:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash \
            --external-device nvme0n1p1 \
            -c ./tools/kernel_flash/flash_l4t_external.xml \
            --external-only --append jetson-xavier-nx-devkit external

And receive an out of memory error at the end of the flashing process.

[ 210]: l4t_flash_from_kernel: Successfully flash the qspi
[ 210]: l4t_flash_from_kernel: Flashing success
error: file_write: write: No space left on device
Cannot write output file
[ 459]: l4t_flash_from_kernel: The device size indicated in the partition layout xml is smaller than the actual size. This utility will try to fix the GPT.
[ 459]: l4t_flash_from_kernel: nvsimg2img /home/jnitsch/nvidia_jetson/Linux_for_Tegra/tools/kernel_flash/images/external/system.img /dev/sdf1 failed
[ 459]: l4t_flash_from_kernel: Error flashing external device
Error flashing non-qspi storage

Cleaning up...

Do you have any idea what I missed here?

1 Like

@WayneWWW do you have any updates on what I could have done wrong here?

hi, if you want to flash only externel nvme use existed systemimg, may you can use --flash-only option.

 ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only \
   jetson-xavier-nx-devkit-emmc external

the final 'external’is show nvme uuid,and if you use nvme0n1p1,it will show ‘bootdev=nvme0n1p1’ in /boot/xxx/xx.conf file.

Thank you, I will just wait for the latest release and then try your suggestion. I will let you know how the test goes.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.