Xavier NX disk encryption with NVME SSD

Hello,

I am using r35.1 BSP version to flash NVME SSD(128GB) attached to Xavier NX jetson (unfused board) with disk encryption.
I am following below link for trying to flash NVME SSD.

https://docs.nvidia.com/jetson/archives/r35.3.1/DeveloperGuide/text/SD/Security/DiskEncryption.html#how-to-flash-an-encrypted-rootfs-to-an-external-storage-device

Here are the steps which I followed :

echo "00000000000000000000000000000000" > ekb.key 
#[T194 example]# This is default KEK2 root key for unfused boardecho "00000000000000000000000000000000" > kek2.key
# This is the default initial vector for EKB.
echo "bad66eb4484983684b992fe54a648bb8" > fv_ekb_t194     
# Generate user-defined symmetric key files# openssl rand -rand /dev/urandom -hex 16 > sym_t194.key
# openssl rand -rand /dev/urandom -hex 16 > sym2_t194.key
echo "00000000000000000000000000000000" > sym_t194.key
echo "00000000000000000000000000000000" > sym2_t194.key
python3 gen_ekb.py -chip t194 -kek2_key kek2.key -fv fv_ekb_t194 -in_sym_key sym_t194.key -in_sym_key2 sym2_t194.key -out eks_t194.img

#Rename eks_t194.img image to eks.img and copy it to Linux_for_Tegra/bootloader folder
mv eks_t194.img eks.img 
cp eks.img Linux_for_Tegra/bootloader

#Run flash command with l4t_initrd_flash.sh:

sudo ROOTFS_ENC=1 ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml -p "-i ekb.key" -S 40GiB --showlogs --external-only jetson-xavier-nx-devkit-emmc nvme0n1p1 

The external ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml is unchanged as in r35.1 BSP.

The flash command runs successfully but jetson cannot boot up leading to kernel panic.

Sometimes, it just shows black screen with cursor blinking on bootup too. What could be the reason it is not booting when disk encryption is enabled ?

Note: Flash command without disk encryption works successfully on NVME SSD and the board can boot up.

Logs snapshot from flash command:

***************************************
*                                     *
*  Step 3: Start the flashing process *
*                                     *
***************************************
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for device to expose ssh ......RTNETLINK answers: File exists
RTNETLINK answers: File exists
4194304
[ 0]: l4t_flash_from_kernel: Starting to create gpt for external device
Active index file is Linux_for_Tegra/tools/kernel_flash/images/external/flash.idx
Number of lines is 14
max_index=13
writing item=1, 9:0:primary_gpt, 512, 19968, gpt_primary_9_0.bin, 16896, fixed-<reserved>-0, ceab6688b18a8d21a4531a29cbbff3a3e4f8aea6
Writing primary_gpt partition with gpt_primary_9_0.bin
Offset is not aligned to K Bytes, no optimization is applied
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/gpt_primary_9_0.bin of=/dev/sde bs=1 skip=0  seek=512 count=16896
16896+0 records in
16896+0 records out
16896 bytes (17 kB, 16 KiB) copied, 0,0330837 s, 511 kB/s
Writing primary_gpt partition done
Error: The backup GPT table is corrupt, but the primary appears OK, so that will be used.
Warning: Not all of the space available to /dev/sde appears to be used, you can fix the GPT to use all of the space (an extra 127910575 blocks) or continue with the current setting? 
Writing secondary_gpt partition with gpt_secondary_9_0.bin
Offset is not aligned to K Bytes, no optimization is applied
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/gpt_secondary_9_0.bin of=/dev/sde bs=1 skip=0  seek=62545444352 count=16896
16896+0 records in
16896+0 records out
16896 bytes (17 kB, 16 KiB) copied, 0,045289 s, 373 kB/s
Writing secondary_gpt partition done
Error: The backup GPT table is not at the end of the disk, as it should be.  Fix, by moving the backup to the end (and removing the old backup)?

                                                                          

                                                                          
Fix/Ignore? Fix
Warning: Not all of the space available to /dev/sde appears to be used, you can fix the GPT to use all of the space (an extra 127910575 blocks) or continue with the current setting? 

                                                                          

                                                                          
Fix/Ignore? Fix
Model: ext0 0 (scsi)
Disk /dev/sde: 128GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name          Flags
 1      20,5kB  419MB   419MB   ext4         APP           msftdata
 2      419MB   42,9GB  42,5GB               APP_ENC       msftdata
 3      42,9GB  43,0GB  83,9MB               recovery      msftdata
 4      43,0GB  43,0GB  524kB                recovery-dtb  msftdata
 5      43,0GB  43,1GB  83,9MB               kernel        msftdata
 6      43,1GB  43,2GB  83,9MB               kernel_b      msftdata
 7      43,2GB  43,2GB  524kB                kernel-dtb    msftdata
 8      43,2GB  43,2GB  524kB                kernel-dtb_b  msftdata
 9      43,2GB  43,3GB  105MB                RECROOTFS     msftdata
10      43,3GB  43,4GB  67,1MB  fat32        esp           boot, esp
11      43,4GB  62,5GB  19,2GB               UDA           msftdata

[ 2]: l4t_flash_from_kernel: Expanding last partition to fill the storage device
[ 3]: l4t_flash_from_kernel: Successfully create gpt for external device
Run command: partprobe on root@fe80::1%usb0
[ 4]: l4t_flash_from_kernel: Starting to flash to external device
Active index file is /Linux_for_Tegra/tools/kernel_flash/images/external/flash.idx
Number of lines is 14
max_index=13
writing item=0, 9:0:master_boot_record, 0, 512, mbr_9_0.bin, 512, fixed-<reserved>-0, 694898d1c345bdb31b377790ed7fc0b0db184bf7
writing item=1, 9:0:primary_gpt, 512, 19968, gpt_primary_9_0.bin, 16896, fixed-<reserved>-0, ceab6688b18a8d21a4531a29cbbff3a3e4f8aea6
writing item=2, 9:0:APP, 20480, 419430400, , , fixed-<reserved>-1, 
Formatting APP partition /dev/sde1 ...
mke2fs 1.45.5 (07-Jan-2020)
Creating filesystem with 102400 4k blocks and 102400 inodes
Filesystem UUID: 01c8dc7d-d09d-4bf2-a6c5-b893308dc9eb
Superblock backups stored on blocks: 
	32768, 98304

Allocating group tables: 0/4   done                            
Writing inode tables: 0/4   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/4   done

Formatting APP parition done
Formatting APP partition /dev/sde1 ...
tar --xattrs -xpf Linux_for_Tegra/tools/kernel_flash/images/external/system_boot.img  --checkpoint=10000 --warning=no-timestamp --numeric-owner  -C  /tmp/ci-KaUubGgJ0B
writing item=3, 9:0:APP_ENC, 419450880, 42530242560, system_root_encrypted.img_ext, 10660964008, fixed-<reserved>-2, 
Writing APP_ENC partition with system_root_encrypted.img_ext
Get size of partition through connection.
Run command: if ! blkdiscard /dev/nvme0n1p2; then
			echo Cannot erase before writing sparse image. Write zero to partition /dev/nvme0n1p2;
			dd if=/dev/zero of=/dev/nvme0n1p2 status=progress oflag=direct; fi on root@fe80::1%usb0
nvsimg2img /Linux_for_Tegra/tools/kernel_flash/images/external/system_root_encrypted.img_ext /dev/sde2
Writing APP_ENC partition done
writing item=4, 9:0:recovery, 42949693440, 83886080, recovery.img, 45840384, fixed-<reserved>-3, ba71b122e5fedf1b2681f2e2b83d3e166dd7ae8e
Writing recovery partition with recovery.img
Get size of partition through connection.
45840384 bytes from /Linux_for_Tegra/tools/kernel_flash/images/external/recovery.img to /dev/sde: 1KB block=44766 remainder=0
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/recovery.img of=/dev/sde bs=1K skip=0  seek=41943060 count=44766
44766+0 records in
44766+0 records out
45840384 bytes (46 MB, 44 MiB) copied, 4,2452 s, 10,8 MB/s
Writing recovery partition done
writing item=5, 9:0:recovery-dtb, 43033579520, 524288, tegra194-p3668-0001-p3509-0000.dtb.rec, 323850, fixed-<reserved>-4, 17be71eba78134f329187e7148e5cfb9060f760d
Writing recovery-dtb partition with tegra194-p3668-0001-p3509-0000.dtb.rec
Get size of partition through connection.
323850 bytes from //Linux_for_Tegra/tools/kernel_flash/images/external/tegra194-p3668-0001-p3509-0000.dtb.rec to /dev/sde: 1KB block=316 remainder=266
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/tegra194-p3668-0001-p3509-0000.dtb.rec of=/dev/sde bs=1K skip=0  seek=42024980 count=316
316+0 records in
316+0 records out
323584 bytes (324 kB, 316 KiB) copied, 0,0297858 s, 10,9 MB/s
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/tegra194-p3668-0001-p3509-0000.dtb.rec of=/dev/sde bs=1 skip=323584  seek=43033903104 count=266
266+0 records in
266+0 records out
266 bytes copied, 0,00115238 s, 231 kB/s
Writing recovery-dtb partition done
writing item=6, 9:0:kernel, 43034103808, 83886080, boot.img, 45692928, fixed-<reserved>-5, 782e914ae938f9e65888b92b219b8c3e3f7677a7
Writing kernel partition with boot.img
Get size of partition through connection.
45692928 bytes from /Linux_for_Tegra/tools/kernel_flash/images/external/boot.img to /dev/sde: 1KB block=44622 remainder=0
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/boot.img of=/dev/sde bs=1K skip=0  seek=42025492 count=44622
44622+0 records in
44622+0 records out
45692928 bytes (46 MB, 44 MiB) copied, 4,32023 s, 10,6 MB/s
Writing kernel partition done
writing item=7, 9:0:kernel_b, 43117989888, 83886080, boot.img, 45692928, fixed-<reserved>-6, 782e914ae938f9e65888b92b219b8c3e3f7677a7
Writing kernel_b partition with boot.img
Get size of partition through connection.
45692928 bytes from /Linux_for_Tegra/tools/kernel_flash/images/external/boot.img to /dev/sde: 1KB block=44622 remainder=0
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/boot.img of=/dev/sde bs=1K skip=0  seek=42107412 count=44622
44622+0 records in
44622+0 records out
45692928 bytes (46 MB, 44 MiB) copied, 4,29429 s, 10,6 MB/s
Writing kernel_b partition done
writing item=8, 9:0:kernel-dtb, 43201875968, 524288, kernel_tegra194-p3668-0001-p3509-0000.dtb, 323850, fixed-<reserved>-7, 17be71eba78134f329187e7148e5cfb9060f760d
Writing kernel-dtb partition with kernel_tegra194-p3668-0001-p3509-0000.dtb
Get size of partition through connection.
323850 bytes from /Linux_for_Tegra/tools/kernel_flash/images/external/kernel_tegra194-p3668-0001-p3509-0000.dtb to /dev/sde: 1KB block=316 remainder=266
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/kernel_tegra194-p3668-0001-p3509-0000.dtb of=/dev/sde bs=1K skip=0  seek=42189332 count=316
316+0 records in
316+0 records out
323584 bytes (324 kB, 316 KiB) copied, 0,0550766 s, 5,9 MB/s
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/kernel_tegra194-p3668-0001-p3509-0000.dtb of=/dev/sde bs=1 skip=323584  seek=43202199552 count=266
266+0 records in
266+0 records out
266 bytes copied, 0,0143005 s, 18,6 kB/s
Writing kernel-dtb partition done
writing item=9, 9:0:kernel-dtb_b, 43202400256, 524288, kernel_tegra194-p3668-0001-p3509-0000.dtb, 323850, fixed-<reserved>-8, 17be71eba78134f329187e7148e5cfb9060f760d
Writing kernel-dtb_b partition with kernel_tegra194-p3668-0001-p3509-0000.dtb
Get size of partition through connection.
323850 bytes from /Linux_for_Tegra/tools/kernel_flash/images/external/kernel_tegra194-p3668-0001-p3509-0000.dtb to /dev/sde: 1KB block=316 remainder=266
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/kernel_tegra194-p3668-0001-p3509-0000.dtb of=/dev/sde bs=1K skip=0  seek=42189844 count=316
316+0 records in
316+0 records out
323584 bytes (324 kB, 316 KiB) copied, 0,106321 s, 3,0 MB/s
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/kernel_tegra194-p3668-0001-p3509-0000.dtb of=/dev/sde bs=1 skip=323584  seek=43202723840 count=266
266+0 records in
266+0 records out
266 bytes copied, 0,00111431 s, 239 kB/s
Writing kernel-dtb_b partition done
writing item=10, 9:0:RECROOTFS, 43202924544, 104857600, , , fixed-<reserved>-9, 
[ 365]: l4t_flash_from_kernel: Warning: skip writing RECROOTFS partition as no image is specified
writing item=11, 9:0:esp, 43307782144, 67108864, esp.img, 67108864, fixed-<reserved>-10, e3c638dea5918840af6cd251b4d3ee9f94a67332
Writing esp partition with esp.img
Get size of partition through connection.
67108864 bytes from /Linux_for_Tegra/tools/kernel_flash/images/external/esp.img to /dev/sde: 1KB block=65536 remainder=0
dd if=/Linux_for_Tegra/tools/kernel_flash/images/external/esp.img of=/dev/sde bs=1K skip=0  seek=42292756 count=65536
65536+0 records in
65536+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 6,98295 s, 9,6 MB/s
Writing esp partition done
writing item=12, 9:0:UDA, 43374891008, 19170553344, system_uda_encrypted.img_ext, 43778232, expand-<reserved>-11, 45cf8b5b0d4f9077082a90b5a5ce70cd9f09b54f
Writing UDA partition with system_uda_encrypted.img_ext
Get size of partition through connection.
Run command: if ! blkdiscard /dev/nvme0n1p11; then
			echo Cannot erase before writing sparse image. Write zero to partition /dev/nvme0n1p11;
			dd if=/dev/zero of=/dev/nvme0n1p11 status=progress oflag=direct; fi on root@fe80::1%usb0
nvsimg2img /Linux_for_Tegra/tools/kernel_flash/images/external/system_uda_encrypted.img_ext /dev/sde11
Writing UDA partition done
writing item=13, 9:0:secondary_gpt, 62545444352, 16896, gpt_secondary_9_0.bin, 16896, fixed-<reserved>-0, cb22f956dd7a8811c9801edb76300a4cfc173ef7
[ 375]: l4t_flash_from_kernel: Successfully flash the external device
[ 375]: l4t_flash_from_kernel: Flashing success
[ 375]: l4t_flash_from_kernel: The device size indicated in the partition layout xml is smaller than the actual size. This utility will try to fix the GPT.

Reboot target
Run command: sync; nohup reboot &>/dev/null & exit on root@fe80::1%usb0
SSH ready
Success
Cleaning up...

hello adit_bhrgv,

could you please moving to the latest release version, i.e. l4t-r35.3.1.
we’ve test locally with l4t-r35.3.1, disk encryption works normally, please see-also Topic 248137, comment #29 for reference,
thanks

I tried Disk encryption with r35.3.1 with NVME SSD flash, it also doesn’t work for me.
Please note disk encryption works for me in r35.1 bsp with emmc.

I also tried disk encryption in r35.3.1 with emmc, however it gets stuck at :

[   5.6398 ] adding BCH for blob_tegra194-p3668-0001-p3509-0000_aligned.dtb
[   5.6949 ] tegrasign_v3.py --key None --list blob_tegra194-p3668-0001-p3509-0000_aligned_sigheader.dtb_list.xml --pubkeyhash pub_key.key
[   5.6955 ] Assuming zero filled SBK key
[   5.7007 ] Warning: pub_key.key is not found
[   5.6668 ] tegrahost_v2 --chip 0x19 0 --updatesigheader blob_tegra194-p3668-0001-p3509-0000_aligned_sigheader.dtb.encrypt blob_tegra194-p3668-0001-p3509-0000_aligned_sigheader.dtb.hash zerosbk
[   5.6838 ] tegrahost_v2 --chip 0x19 --generateblob blob.xml blob.bin
[   5.6862 ] number of images in blob are 10
[   5.6868 ] blobsize is 6432664
[   5.6870 ] Added binary blob_nvtboot_recovery_cpu_t194_sigheader.bin.encrypt of size 233040
[   5.6921 ] Added binary blob_nvtboot_recovery_t194_sigheader.bin.encrypt of size 206016
[   5.6929 ] Added binary blob_preboot_c10_prod_cr_sigheader.bin.encrypt of size 24016
[   5.6936 ] Added binary blob_mce_c10_prod_cr_sigheader.bin.encrypt of size 145184
[   5.6943 ] Added binary blob_mts_c10_prod_cr_sigheader.bin.encrypt of size 3430416
[   5.6961 ] Added binary blob_bpmp-2_t194_sigheader.bin.encrypt of size 1007392
[   5.6977 ] Added binary blob_tegra194-a02-bpmp-p3668-a00_lz4_sigheader.dtb.encrypt of size 36176
[   5.6984 ] Added binary blob_spe_t194_sigheader.bin.encrypt of size 95232
[   5.6989 ] Added binary blob_tos-optee_t194_sigheader.img.encrypt of size 914992
[   5.6994 ] Added binary blob_tegra194-p3668-0001-p3509-0000_sigheader.dtb.encrypt of size 340032
[   5.7039 ] Sending bootloader and pre-requisite binaries
[   5.7070 ] tegrarcm_v2 --download blob blob.bin
[   5.7087 ] Applet version 01.00.0000
[   5.7843 ] Sending blob
[   5.7846 ] [................................................] 100%
[   6.6748 ] tegrarcm_v2 --boot recovery
[   6.6764 ] Applet version 01.00.0000
[   7.7573 ] tegrarcm_v2 --isapplet

My question was regarding disk encryption on NVME SSD.

Have you also tested it with external NVME SSD on r35.3.1 or just emmc ? the reference link you shared is only using emmc

could you please check tools/kernel_flash/README_initrd_flash.txt, and you may check Workflow 10 for the steps of disk encryption support on external device.

sudo ROOTFS_ENC=1 ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml -p "-i ./ekb.key" -S 20GiB --showlogs --external-only jetson-xavier-nx-devkit-emmc external

I tried above command from Workflow 10:

sudo ROOTFS_ENC=1 ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml -p "-i ./ekb.key" -S 40GiB --showlogs --external-only jetson-xavier-nx-devkit-emmc external

With this command, it flashes Jetson but not NVME, rather EMMC, not sure why ?

lsblk 
NAME           MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
loop0            7:0    0    16M  1 loop  
mmcblk0        179:0    0  14.7G  0 disk  
├─mmcblk0p1    179:1    0   400M  0 part  /boot
├─mmcblk0p2    179:2    0  13.6G  0 part  
│ └─crypt_root 252:0    0  13.6G  0 crypt /
├─mmcblk0p3    179:3    0    64M  0 part  
├─mmcblk0p4    179:4    0   448K  0 part  
├─mmcblk0p5    179:5    0  32.6M  0 part  
├─mmcblk0p6    179:6    0   2.5M  0 part  
├─mmcblk0p7    179:7    0    64K  0 part  
├─mmcblk0p8    179:8    0     1M  0 part  
├─mmcblk0p9    179:9    0     1M  0 part  
├─mmcblk0p10   179:10   0     1M  0 part  
├─mmcblk0p11   179:11   0   1.5M  0 part  
├─mmcblk0p12   179:12   0     1M  0 part  
├─mmcblk0p13   179:13   0    64M  0 part  
├─mmcblk0p14   179:14   0   448K  0 part  
├─mmcblk0p15   179:15   0  32.5M  0 part  
├─mmcblk0p16   179:16   0    80M  0 part  
├─mmcblk0p17   179:17   0   512K  0 part  
├─mmcblk0p18   179:18   0   100M  0 part  
├─mmcblk0p19   179:19   0    64M  0 part  
└─mmcblk0p20   179:20   0 245.5M  0 part  
  └─crypt_UDA  252:1    0 229.5M  0 crypt /mnt/crypt_UDA
zram0          251:0    0 856.6M  0 disk  [SWAP]
zram1          251:1    0 856.6M  0 disk  [SWAP]
zram2          251:2    0 856.6M  0 disk  [SWAP]
zram3          251:3    0 856.6M  0 disk  [SWAP]
nvme0n1        259:0    0 119.2G  0 disk  
├─nvme0n1p1    259:1    0   400M  0 part  
├─nvme0n1p2    259:2    0  39.6G  0 part  
├─nvme0n1p3    259:3    0    64M  0 part  
├─nvme0n1p4    259:4    0   512K  0 part  
├─nvme0n1p5    259:5    0    32M  0 part  
├─nvme0n1p6    259:6    0    64M  0 part  
├─nvme0n1p7    259:7    0   512K  0 part  
├─nvme0n1p8    259:8    0    32M  0 part  
├─nvme0n1p9    259:9    0    80M  0 part  
├─nvme0n1p10   259:10   0   512K  0 part  
├─nvme0n1p11   259:11   0   100M  0 part  
├─nvme0n1p12   259:12   0    64M  0 part  
├─nvme0n1p13   259:13   0    80M  0 part  
├─nvme0n1p14   259:14   0   512K  0 part  
├─nvme0n1p15   259:15   0    64M  0 part  
└─nvme0n1p16   259:16   0  78.7G  0 part  

there’re APP and APP_ENC partitions, it’s the commands to flash an encrypted rootfs to an NVMe SSD.
or… did you meant you would like to boot from NVME?

Yes

see-also developer guide, Flashing to an NVMe Drive.
and… you may modify the boot order via UEFI if the target still boots from its internal storage.

Thanks. I think the issue is resolved. It was booting with emmc by default and had to select NVME boot option manually.
How can I change the boot order in r35.1 BSP release to set nvme to be default ? Do I need to change dtb ?

Thanks

hello adit_bhrgv,

please refer to developer guide, you may try several ways to Customizing the Default Boot Order in the Configuration File.

Thanks.

I changed the L4TConfiguration.dts as below in variable gNVIDIATokenSpaceGuid to boot by default from nvme and compiled dts to dtbo and placed in LInux_for_Tegra/kernel/dtb folder :

firmware {

                                uefi {

                                        variables {

                                                gNVIDIAPublicVariableGuid {

                                                        QuickBootEnabled {
                                                                data = [00];
                                                                non-volatile;
                                                        };

                                                        NewDeviceHierarchy {
                                                                data = [01];
                                                                non-volatile;
                                                        };
                                                };
                                                gNVIDIATokenSpaceGuid {

                                                        DefaultBootPriority {
                                                                data = "nvme,emmc";
                                                                locked;
                                                        };
                                               };
                                        };
                                };
                        };
                };

But still it is booting from emmc by default. Only way right now to boot from nvme for me is to select Boot options from Monitor connected to jetson.

Can you please let me know what else I need to do to change boot order via .dtbo?

hello adit_bhrgv,

dtbo it’s a device tree binary overlay.
had you done running Jetson-IO for system configuration to load the overlays?

Yes I ran it now, it saved a kernel_tegra194-p3668-0001-p3509-0000-user-custom.dtb file in /boot folder.
Basically, this dtb has default boot-order priority as “emmc,nvme”. (Just for Info, I changed the default order to be emmc, nvme because I wanted the jetson to try booting from emmc first)

After rebooting the jetson, the jetson still boots from nvme. I think this might be because while running the JetsonIO tool, the documentation says it will also update the extlinux.conf. But FDT entry in my /boot/extlinux/extlinux.conf shows “FDT /boot/dtb/kernel_tegra194-p3668-0001-p3509-0000.dtb” . Shouldn’t it show the new custom dtb generated here ? Or do I have to manually update this ?

it should create a new LABEL with your customize entry, and you may check the 2nd line for the DEFAULT config.

This is the extlinux.conf which got saved in /boot/extlinux:

TIMEOUT 30
DEFAULT JetsonIO

MENU TITLE L4T boot options

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      FDT /boot/dtb/kernel_tegra194-p3668-0001-p3509-0000.dtb
      INITRD /boot/initrd
      APPEND ${cbootargs} root=/dev/nvme0n1p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

# When testing a custom kernel, it is recommended that you create a backup of
# the original kernel and add a new entry to this file so that the device can
# fallback to the original kernel. To do this:
#
# 1, Make a backup of the original kernel
#      sudo cp /boot/Image /boot/Image.backup
#
# 2, Copy your custom kernel into /boot/Image
#
# 3, Uncomment below menu setting lines for the original kernel
#
# 4, Reboot

# LABEL backup
#    MENU LABEL backup kernel
#    LINUX /boot/Image.backup
#    FDT /boot/dtb/kernel_tegra194-p3668-0001-p3509-0000.dtb
#    INITRD /boot/initrd
#    APPEND ${cbootargs}

this doesn’t look correct, you don’t the field with LABEL JetsonIO.
here’s an alternative way, you may keep using primary LABEL, updating the FDT entry to point-to your customize dtb.

I tired that workaround to use my custom dtb in the FDT entry. but it also doesn’t work.,

FYI, I am using r35.1 BSP.

I am confused if it is using CBoot or UEFI ?

since the original issue “Xavier NX disk encryption with NVME SSD” has resolved.
could you please try moving to latest release to confirm the boot order modification? and… let’s have a new discussion thread to follow-up.

sure