Massflash image with ADDITIONAL_DTB_OVERLAY_OPT option

So what comment did you use to flash the device?
Did you enable secure boot?

Then that may be the case.

Yes, we have secure boot enabled.

To generate the Massflash Image:

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ROOTFS_ENC=1 BOARDID=3668 FAB=301 BOARDSKU=0001 BOARDREV=E.0 ./tools/kernel_flash/l4t_initrd_flash.sh -u $ROOT_DIR/keys/key.pem -v $ROOT_DIR/keys/sbk_key.txt -p "-i ./ekb.key" --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml --external-only -S 40GiB --no-flash --massflash 8 --showlogs jetson-xavier-nx-devkit-emmc nvme0n1p1

To flash the device from Linux_for_Tegra/mfi_ folder :

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --massflash 8 --flash-only

How can I make sure, that .sb file is updated with latest fdtoverlay changes with custom.dtbo and BootOrderNvme.dtbo as applied below ? I see kernel_tegra194-p3668-0001-p3509-0000.dtb has these overlay changes but kernel_tegra194-p3668-0001-p3509-0000.dtb.sb doesn’t.

#Apply custom.dtbo

dtc -O dtb -o Linux_for_Tegra/kernel/dtb/custom.dtbo -@ Linux_for_Tegra/dts/custom.dts
fdtoverlay -i Linux_for_Tegra/kernel/dtb/tegra194-p3668-0001-p3509-0000.dtb -o Linux_for_Tegra/kernel/dtb/tegra194-p3668-0001-p3509-0000-extended.dtb Linux_for_Tegra/kernel/dtb/custom.dtbo

mv Linux_for_Tegra/kernel/dtb/tegra194-p3668-0001-p3509-0000-extended.dtb Linux_for_Tegra/kernel/dtb/tegra194-p3668-0001-p3509-0000.dtb

#Apply BootOrderNvme.dtbo
fdtoverlay -i Linux_for_Tegra/kernel/dtb/tegra194-p3668-0001-p3509-0000.dtb -o Linux_for_Tegra/kernel/dtb/tegra194-p3668-0001-p3509-0000-extended.dtb Linux_for_Tegra/kernel/dtb/BootOrderNvme.dtbo

mv Linux_for_Tegra/kernel/dtb/tegra194-p3668-0001-p3509-0000-extended.dtb Linux_for_Tegra/kernel/dtb/tegra194-p3668-0001-p3509-0000.dtb

What changes are taking effect now?
Only custom.dtbo or both custom.dtbo and BootOrderNvme.dtbo are gone?

I didn’t flash the jetson again yet but currently what I observe is in Linux_for_Tegra/bootloader on host pc:

kernel_tegra194-p3668-0001-p3509-0000.dtb.sb  -> has nvme change but custom.dtbo change is the previous one
kernel_tegra194-p3668-0001-p3509-0000.dtb  -> has nvme change and also new custom.dtbo change

I guess, kernel_tegra194-p3668-0001-p3509-0000.dtb is regenerated while creating massflash image again with new custom.dtbo ( as we change within custom.dtbo “name” value ) but kernel_tegra194-p3668-0001-p3509-0000.dtb.sb remains the old one. Not sure, why .sb file is not regenerated while creating massflash image.

But in Linux_for_Tegra/mfi_jetson-xavier-nx-devkit-emmc/bootloader : there is only one file and I guess that has to be the final dtb: [Since mfi_ folder, there is no .sb file ]

kernel_tegra194-p3668-0001-p3509-0000.dtb -> it has both BootorderNvme.dtbo and custom.dtbo changes

I will flash the jetson again and post again regarding what changes takes effect…

After flashing, the custom.dtbo change is working correctly.
However, boot order set as “nvme” from BootOrderNvme.dtbo is not reflected in uefi menu and the jetson continues to boot from emmc by default.

However, the boot-order change can be seen in the final dtb file: kernel_tegra194-p3668-0001-p3509-0000.dtb
Is this change not good enough to boot from NVME by default ? or some extra things needed to be done ?

 firmware {

                uefi {

                        variables {

                                gNVIDIATokenSpaceGuid {

                                        DefaultBootPriority {
                                                locked;
                                                data = "nvme";
                                        };
                                };
                        };
                };

                optee {
                        compatible = "linaro,optee-tz";
                        method = "smc";
                        status = "disabled";
                };

                android {
                        hardware = "galen-k414";
                };
        };

Then if you manually change the boot order in UEFI, would it work?
It’s just that settings in dtb do not take effect?

Yes, manually selecting boot device as NVME from UEFI Menu works !
But, the dtb changes doesn’t take effect. That means, it didn’t boot from NVME by default. Still the EMMC is reflected as the default boot device.

Hi,

Can you please show me what do you get with df -h after booting into eMMC and NVMe?
Also, would it make any difference if you apply it with L4TConfiguration.dtbo instead of BootOrderNvme.dtbo?
https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/SD/Bootloader/UEFI.html#customizing-the-default-boot-order-in-the-configuration-file

I don’t have anything on emmc as I use --external-only flag while creating Mass flash image.
However, “df -h” on NVME returns :

df -h
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/crypt_root   39G  8.6G   29G  24% /
/dev/mapper/crypt_UDA   279M   24K  257M   1% /mnt/crypt_UDA
none                    3.3G     0  3.3G   0% /dev
tmpfs                   3.4G     0  3.4G   0% /dev/shm
tmpfs                   684M  2.2M  682M   1% /run
tmpfs                   5.0M  4.0K  5.0M   1% /run/lock
tmpfs                   3.4G     0  3.4G   0% /sys/fs/cgroup
/dev/nvme0n1p1          371M   96M  248M  28% /boot
/dev/sda1               3.7G  224K  3.7G   1% /mnt/hxdongle
tmpfs                   684M     0  684M   0% /run/user/1000```

I checked again that using “ADDITIONAL_DTB_OVERLAY_OPT=“BootOrderNvme.dtbo”” indeed works when I don’t use the --external-only flag in the below command without creating massflash image, rather just directly flashing jetson:

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ROOTFS_ENC=1 ./tools/kernel_flash/l4t_initrd_flash.sh -u $ROOT_DIR/keys/key.pem -v $ROOT_DIR/keys/sbk_key.txt -p "-i ./ekb.key" nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml -S 40GiB --showlogs jetson-xavier-nx-devkit-emmc nvme0n1p1

But, I don’t want to flash EMMC . Workflow 11, example 2 from README_initrd_flash.txt mentions:

**"Workflow 11: Generate images for internal device and external device seperately"** 
**Example 2: In this example, you want to boot Jetson Xavier NX SD from an**
**attached NVMe SSD. The SD card does not need to be plugged in. You can also**
**apply this if you don't want to use the emmc on the Jetson Xavier NX emmc.**

Do I need to generate qspi-only images for BootorderNvme.dtbo to take effect ?

Regards,

What if you do

-p "-c bootloader/t186ref/cfg/flash_l4t_t194_qspi_p3668.xml"

instead of --external-only?

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ROOTFS_ENC=1 BOARDID=3668 FAB=301 BOARDSKU=0001 BOARDREV=E.0 ./tools/kernel_flash/l4t_initrd_flash.sh -u $ROOT_DIR/keys/key.pem -v $ROOT_DIR/keys/sbk_key.txt -p "-i ./ekb.key -c bootloader/t186ref/cfg/flash_l4t_t194_qspi_p3668.xml" --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml  -S 40GiB --no-flash --massflash 8 --showlogs jetson-xavier-nx-devkit-emmc nvme0n1p1

We already have -c option with “./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml” conf file, will it be able to accept another "-c bootloader/t186ref/cfg/flash_l4t_t194_qspi_p3668.xml " within -p ?

Does the above command looks good ?

YES.
I don’t think they contradict each other.

1 Like

This works now.
The Boot order seems to be updated now with both L4TConfiguration.dtbo or BootOrderNvme.dtbo .

Could you please give a background on excluding --external-only flag and including bootloader/t186ref/cfg/flash_l4t_t194_qspi_p3668.xml ?

What is the reasoning behind this works ? Is it related to boot-flow ? Please refer me to Nvidia documentation

I’m just doing some guessing here.

The boot order is determined by UEFI, which resides in the QSPI memory.
Then you can take a look at the following snippet taken from Linux_for_Tegra/tools/kernel_flash/l4t_flash_from_kernel.sh

1142 function flash_qspi
1143 {
1144 	if [ "${host_mode}" = "0" ] && [ "${external_only}" = "0" ] && is_spi_flash; then
1145 		print_log "Starting to flash to qspi"
1146 		if [ -z "${target_partname}" ]; then
1147 			erase_spi /dev/mtd0
1148 		fi
1149 		write_to_spi
1150 
1151 		print_log "Successfully flash the qspi"
1152 	fi
1153 }

As you can see, if you explicitly specify –external-only, the QSPI memory will not be updated at all.
Also, UEFI reads device tree from the dedicated bootloader-dtb partition, which also resides in the QSPI memory, so checking if it’s taking effect in the rootfs is meaningless.

Therefore, if you don’t want to flash the internal eMMC, and need to update the QSPI memory to update the boot order, instead of –external-only, you need to specify -p "-c bootloader/t186ref/cfg/flash_l4t_t194_qspi_p3668.xml" to tell the script to generate images for only QSPI but not eMMC.

1 Like

Thank you @DaveYYY for the clear explanation.

I also encountered one error when disk encryption is enabled after removing “–external-only”:

###############################################################################
# L4T BSP Information:
# R35 , REVISION: 4.1
# User release: 0.0
###############################################################################
Error: When disk encryption is enabled, root device cannot be nvme0n1p1

It comes from code below in flash.sh:

if [[ “${rootdev_type}” == “external” && “${disk_enc_enable}” -eq 1 ]]; then
echo “Error: When disk encryption is enabled, root device cannot be ${target_rootdev}”
exit 1;
fi;

If I just avoid the " exit 1" in this check, it works !
Can I somehow avoid this error or this check is needed ?

Seems like you need to generate images for QSPI and external storage device separately.
Like the case for Orin NX/Nano as shown in the guide:
https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/SD/FlashingSupport.html#using-initrd-flash-with-orin-nx-and-nano

# Generate images for QSPI
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs -p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" --no-flash --network usb0 jetson-orin-nano-devkit internal

# Generate images for external storage device
$ sudo ROOTFS_ENC=1 ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --no-flash --external-device nvme0n1p1 -i ./ekb.key -c ./tools/kernel_flash/flash_l4t_t234_nvme_rootfs_enc.xml --external-only --append --network usb0 jetson-orin-nano-devkit external

# Flash images into the both storage devices
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --network usb0 --flash-only
1 Like

Do we need to set the jetson board to recovery before creating image for QSPI ?
Ideally, we want to create the final image in an offline mode without the board to be in recovery mode.

Are these commands equivalent to create QSPI images ?

sudo ./tools/kernel_flash/l4t_initrd_flash.sh -p "-c bootloader/t186ref/cfg/flash_l4t_t194_qspi_p3668.xml" --showlogs --no-flash jetson-xavier-nx-devkit-emmc internal
        
 & 

sudo ./tools/kernel_flash/l4t_initrd_flash.sh  --showlogs --no-flash jetson-xavier-nx-devkit-qspi internal

We tried both of these versions to create QSPI images and then ran the below command to create the final image with Disk encryption:

 sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ROOTFS_ENC=1 BOARDID=3668 FAB=301 BOARDSKU=0001 BOARDREV=E.0 ./tools/kernel_flash/l4t_initrd_flash.sh -u $ROOT_DIR/keys/key.pem -v $ROOT_DIR/keys/sd/sbk_key.txt  -p "-i ./ekb.key" --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml -S 40GiB --no-flash --massflash 8 --external-only --append --showlogs jetson-xavier-nx-devkit-emmc nvme0n1p1

This created a MASSFLASH image successfully, when we flashed this image to the Jetson board, the flashing is successfull and it gave no errors.

However, after flashing the board continues to be in RECOVERY Mode. 0955:7e19 and the board doesn’t boot up.

Do you know why the board doesn’t boot up after flashing ? Are we doing something wrong ?

Thanks in advance !

Regards

If the original issue for this topic has been solved, please file a new topic for other issues you encounter

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.