Not enough space when flashing customized rootfs

jameskuo · January 21, 2025, 3:38am

Hi Nvidia:
We’re trying to flash customized rootfs to NVMe on Jetpack 6.0, AGX Orin.
The command we used to flash is:

# Extract customized rootfs.tar.zst to rootfs folder
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "--no-systemimg -c bootloader/generic/cfg/flash_t234_qspi.xml" --showlogs --network usb0 jetson-agx-orin-devkit external

This one works before. In such way, we are able to flash rootfs to different NVMe with different size, like 128G, 256G, etc.
However, now the customized rootfs seems to be too large for eMMC, leading to an error while generating system.img.
Full logs after running flash cmd:
flashNvmeLog.log (134.3 KB)

At the end of the logs, the tar command report errors while making system.img:

Making system.img... 
	populating rootfs from /home/wulun/flashingTest/js1100_cstRootfs_v1/rootfs ... Failed. Your APPSIZE might be too small.
Error: Failed to generate images for external device
Cleaning up...

which is happened in flash.sh line 590.

We checked both NVMe on Host PC and NVMe on AGX Orin, both of them have enough space and inode.
How can we avoid this error? If the system.img is for eMMC, can we ignore it via cmd parameters?
Or, can we use another rootfs folder to create this dummy system.img, another folder for the customized rootfs?

Please help.

Many Thanks.

carolyuu · January 21, 2025, 4:00am

Hi,
If the device cannot be flashed/booted, please refer to the page to get uart log from the device:
Jetson/General debug - eLinux.org
And get logs of host PC and Jetson device for reference. If you are using custom board, you can compare uart log of developer kit and custom board to get more information.
Also please check FAQs:
Jetson AGX Orin FAQ
If possible, we would suggest follow quick start in developer guide to re-flash the system:
Quick Start — NVIDIA Jetson Linux Developer Guide 1 documentation
And see if the issue still persists on a clean-flashed system.
Thanks!

jameskuo · January 21, 2025, 5:32am

To reproduce the issue, you can:

Prepare a clean Linux_for_Tegra for download center
De-compress rootfs in rootfs folder
sudo dd if=/dev/zero of=Linux_for_Tegra/rootfs/bigfile bs=1M count=61440
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "--no-systemimg -c bootloader/generic/cfg/flash_t234_qspi.xml" --showlogs --network usb0 jetson-agx-orin-devkit external

KevinFFF · January 21, 2025, 6:20am

Hi jameskuo,

Are you using the devkit or custom board for AGX Orin?

jameskuo:

# Extract customized rootfs.tar.zst to rootfs folder
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "--no-systemimg -c bootloader/generic/cfg/flash_t234_qspi.xml" --showlogs --network usb0 jetson-agx-orin-devkit external

Do you use this command to flash the board and you hit the error in flashNvmeLog.log?

Please try if the following suggestions can help.

use flash_l4t_t234_nvme.xml instead of flash_l4t_external.xml
add --external-only to flash NVMe SSD only to prevent it create internal image for eMMC

jameskuo · January 21, 2025, 7:01am

Hi Kevin:
Thanks for the reply.
We tried three ways to flash:

$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_t234_nvme.xml -p "--no-systemimg -c bootloader/generic/cfg/flash_t234_qspi.xml" --showlogs --network usb0 jetson-agx-orin-devkit external

Changed NVMe xml only, and the issue remained, logs:
flashLog-nvmeXml.log (59.3 KB)

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-only --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "--no-systemimg -c bootloader/generic/cfg/flash_t234_qspi.xml" --showlogs --network usb0 jetson-agx-orin-devkit external

Add external only flag, logs: flashNvme-externalOnly.log (188.1 KB)

However, no rootfs flashed.

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-only --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_t234_nvme.xml -p "--no-systemimg -c bootloader/generic/cfg/flash_t234_qspi.xml" --showlogs --network usb0 jetson-agx-orin-devkit external

Add external only flag and changed the XML file, Logs:
flashNvme-externalOnly_diffXml.log (188.1 KB)

Still, no rootfs flashed.

Yes, we’re using custom board for AGX Orin.

Precisely, sudo ./tools/kernel_flash/l4t_initrd_flash.sh ..... | tee flashNvmeLog.log

KevinFFF · January 22, 2025, 8:35am

Have you confirmed that rootfs not flashed in this case?
I’ve seen following logs there:

[ 2]: l4t_flash_from_kernel: Starting to flash to external device
Active index file is /mnt/external/flash.idx
Number of lines is 61
max_index=60
[ 2]: l4t_flash_from_kernel: Successfully flash the external device
[ 2]: l4t_flash_from_kernel: Flashing success

Please try to create any tmp file on your board and perform the flash process again to check if it has not been flashed.

Or you can try using the following commands to flash the board with 3 steps.

Step1. create internal(QSPI)
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs -p "-c bootloader/generic/cfg/flash_t234_qspi.xml" --no-flash --network usb0 jetson-agx-orin-devkit internal

Step2. create external(NVMe SSD)
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --no-flash --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_t234_nvme.xml --external-only --append --network usb0 jetson-agx-orin-devkit external

Step3. flash both
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --network usb0 --flash-only

jameskuo · January 22, 2025, 8:46am

Hi Kevin:
Thanks for the reply.

Yes, I confirmed. All the history I typed were remained.

About the steps, we’ll give it a try.

Many thanks.

jameskuo · January 23, 2025, 7:06am

Hi Kevin:
3 Steps failed at step 2, same error message.
Logs are attached:
appendError.log (25.3 KB)

Many Thanks.

KevinFFF · January 23, 2025, 7:22am

Making system.img... 
	populating rootfs from /home/wulun/flashingTest/js1100_cstRootfs_v1/rootfs ... Failed. Your APPSIZE might be too small.

This error is from flash.sh.

Do you know what’s the size of your custom system.ima.raw? Or the data size under /home/wulun/flashingTest/js1100_cstRootfs_v1/rootfs?
If it is 100GiB, please use the following command with -S option specified in Step2 instead.

Step2. create external(NVMe SSD)
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --no-flash --external-device nvme0n1p1 -S 100GiB -c ./tools/kernel_flash/flash_l4t_t234_nvme.xml --external-only --append --network usb0 jetson-agx-orin-devkit external

jameskuo · January 23, 2025, 7:49am

Hi Kevin:
Thanks for the reply.
After adding -S 100GiB, the flashing process can be finished.
However, the device being flashed is not able to boot. We are verifying the root cause and will update once we have further results.

Happy New Year.

KevinFFF · January 24, 2025, 7:07am

Okay, it seems you’ve resolved the flashing issue.
For the boot issue, please just check the full serial console log for errors.
Thanks, Happy CNY~

jameskuo · February 3, 2025, 7:58am

Hi KevinFFF:
Not sure what happened during CNY, but we can now boot to OS normally with the 3 steps you provided, and add -S 100GiB flag in step 2.

Thank you for your support.

system · February 24, 2025, 7:37am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Custom board use l4t_initrd_flash.sh fail with Waiting for target to boot-up Jetson AGX Orin reflash , board-design	14	95	November 19, 2024
Jetson Orin NX 16GB with custom carrier board boot failed Jetson Orin NX boot , board-design , chinese	15	73	January 14, 2025
Flash a configuration that emulates a Jetson Orin NX to NVMeSSD Jetson AGX Orin reflash , nvme	10	1107	February 6, 2023
ERROR after flashing J401 Orin NX Board boot\extlinux\extlinux.conf: Not Found Jetson Orin NX reflash	28	977	March 3, 2024
Use Custom Rootfs for Nvme SSD for mass flashing Jetson AGX Orin reflash	3	326	June 3, 2024
Flashing a cloned system.img Jetson Orin Nanon Developer Kit Jetson Orin Nano clone	26	209	November 27, 2024
Timeout when try to flash NVMe on AGX Orin Jetson AGX Orin reflash , chinese	11	590	May 7, 2024
Flashing external NVMe SSD directly on host does not work Jetson Orin NX reflash , security	11	156	November 13, 2024
Using Orin AGX to emulate Orin NX with filesystem on NVMe SSD Jetson AGX Orin	7	575	November 8, 2023
Can't boot orin nx after flashing custom Kernel Jetson Orin NX boot , reflash , kernel	14	886	January 30, 2024

Not enough space when flashing customized rootfs

Related topics