Flashing Disk Encrypted 4TB SSD

  • Hardware Platform (Jetson): Jetson Orin Nano 8GB dev kit

  • JetPack Version: JetPack 5.1.3 (r35.5.0)

  • Issue Type (questions, new requirements, bugs): bug

Hi,I’m struggling to flash R35.5.0 to a 4TB NVME, getting error: file_write: write: No space left on device.The SSD I’m using has the following stats:

Disk /dev/nvme0n1: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: CT4000P3PSSD8
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt

If I flash using the following settings

ROOTFS_GIB=1344GiB         # size of APP_ENC partition (≈ 1 344 GiB) Works
EXT_NUM_SECTORS=7814037168     # exact sector count from `fdisk -l`
USER=root BOARDID=3767 FAB=300 BOARDSKU=0005 CHIP_SKU=D5 ./flash.sh --no-flash -k A_eks jetson-orin-nano-devkit internal

XML_FILE="./tools/kernel_flash/flash_l4t_t234_nvme_rootfs_enc.xml"
sed -i "s/NUM_SECTORS/${EXT_NUM_SECTORS}/g" "$XML_FILE"
 USER=root BOARDID=3767 FAB=300 BOARDSKU=0005 CHIP_SKU=D5 \
    ./tools/kernel_flash/l4t_initrd_flash.sh \
      -p '-c bootloader/t186ref/cfg/flash_t234_qspi.xml --no-systemimg' \
      --showlogs \
      --no-flash \
      --network usb0 \
      jetson-orin-nano-devkit internal
USER=root BOARDID=3767 FAB=300 BOARDSKU=0005 CHIP_SKU=D5ROOTFS_ENC=1 \
    ./tools/kernel_flash/l4t_initrd_flash.sh \
      --external-device nvme0n1p1 \
      -c ./tools/kernel_flash/flash_l4t_t234_nvme_rootfs_enc.xml \
      -p "--generic-passphrase" \
      -S $ROOTFS_GIB \
      -i "$L4T_DIR/source/public/optee/samples/hwkey-agent/host/tool/gen_ekb/keys/sym2_t234.key" \
      --external-only \
      --append \
      --showlogs \
      --no-flash \
      --network usb0 \
      --massflash 10 \
      jetson-orin-nano-devkit external
./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --network usb0 --flash-only

The system flashes and partitions the device as it should, with the partition table shown in the flash only step:

Number Start  End   Size  File system Name        Flags
 3   20.5kB 134MB  134MB        A_kernel      msftdata
 4   134MB  135MB  786kB        A_kernel-dtb    msftdata
 5   135MB  168MB  33.2MB        A_reserved_on_user msftdata
 6   168MB  302MB  134MB        B_kernel      msftdata
 7   302MB  303MB  786kB        B_kernel-dtb    msftdata
 8   303MB  336MB  33.2MB        B_reserved_on_user msftdata
 9   336MB  420MB  83.9MB        recovery      msftdata
10   420MB  421MB  524kB        recovery-dtb    msftdata
11   421MB  488MB  67.1MB fat32    esp         boot, esp
12   488MB  572MB  83.9MB        recovery_alt    msftdata
13   572MB  572MB  524kB        recovery-dtb_alt  msftdata
14   572MB  639MB  67.1MB        esp_alt       msftdata
15   639MB  1059MB 419MB        UDA         msftdata
16   1059MB 1562MB 503MB        reserved      msftdata
 1   1562MB 1981MB 419MB  ext4     APP         msftdata
 2   1981MB 1445GB 1443GB        APP_ENC       msftdata

However if I up the ROOTFS size to:

ROOTFS_GIB=2688GiB         # size of APP_ENC partition (≈ 1 344 GiB) Works
EXT_NUM_SECTORS=7814037168     # exact sector count from `fdisk -l`

The APP_ENC partition is smaller than defined

Number Start  End   Size  File system Name        Flags
 3   20.5kB 134MB  134MB        A_kernel      msftdata
 4   134MB  135MB  786kB        A_kernel-dtb    msftdata
 5   135MB  168MB  33.2MB        A_reserved_on_user msftdata
 6   168MB  302MB  134MB        B_kernel      msftdata
 7   302MB  303MB  786kB        B_kernel-dtb    msftdata
 8   303MB  336MB  33.2MB        B_reserved_on_user msftdata
 9   336MB  420MB  83.9MB        recovery      msftdata
10   420MB  421MB  524kB        recovery-dtb    msftdata
11   421MB  488MB  67.1MB fat32    esp         boot, esp
12   488MB  572MB  83.9MB        recovery_alt    msftdata
13   572MB  572MB  524kB        recovery-dtb_alt  msftdata
14   572MB  639MB  67.1MB        esp_alt       msftdata
15   639MB  1059MB 419MB        UDA         msftdata
16   1059MB 1562MB 503MB        reserved      msftdata
 1   1562MB 1981MB 419MB  ext4     APP         msftdata
 2   1981MB 689GB  687GB        APP_ENC       msftdata

With the error:

error: file_skip: lseek64: Invalid argument
error: file_skip: lseek64: Invalid argument
error: file_skip: lseek64: Invalid argument
error: file_write: write: No space left on device
Cannot write output file

All the code in the above remains the same, just a change in ROOT FS size. I’d expect the End and Size would match 2688GiB.

Could you please advise, and let me know if you need any further information.

Thanks

*** Please note that this reply is generated by LLM automatically ***

Based on the provided information, it appears that you are experiencing issues with flashing the R3550 image to a 4TB NVME device on a Jetson Orin Nano. The error message “No space left on device” suggests that the device is running out of space during the flashing process.

To troubleshoot this issue, let’s analyze the provided information:

  1. The SSD stats show that the device has a total capacity of 4TB, with 3.64TB available for use.
  2. The initial flashing attempt with a ROOTFS size of 2688GiB (approximately 2.7TB) is successful, and the partition table is created as expected.
  3. However, when you increase the ROOTFS size to 2688GiB, the APP_ENC partition becomes smaller than defined, and the error “No space left on device” occurs.

Comparing the two scenarios, it’s clear that the issue arises when you increase the ROOTFS size. This suggests that the problem might be related to the partition layout or the available space on the device.

Here are a few potential explanations:

  • The increased ROOTFS size might be exceeding the available space on the device, causing the APP_ENC partition to become smaller than defined.
  • The partition layout might not be optimized for the increased ROOTFS size, leading to conflicts with other partitions.

To resolve this issue, you could try the following:

  • Verify that the ROOTFS size is not exceeding the available space on the device. You can check the available space using the df command.
  • Review the partition layout and adjust it as needed to accommodate the increased ROOTFS size. You can use tools like fdisk or parted to modify the partition layout.
  • Consider reducing the ROOTFS size to a value that is within the available space on the device.

Additionally, you may want to check the following:

  • Ensure that the disk_enc_enable and EMMC_CFG settings are correctly configured in the board configuration file.
  • Verify that the flash.sh script is using the correct partition layout file.

If you’re still experiencing issues, please provide more information about your setup, including the exact commands you’re using to flash the device and the output of the df command. This will help me better understand the issue and provide more specific guidance.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

The LLM response does not help, commands ran are in the original post

hello user_rw23,

please update your flash command line for adding -S and EXT_NUM_SECTORS.
here’s an example of 128GiB NVMe SSD.
– Set EXT_NUM_SECTORS=240000000 (about 114GiB) → it should smaller than NVMe actual size.
– Set -S = 100GiB → bigger than APP size.
$ sudo ROOTFS_ENC=1 EXT_NUM_SECTORS=240000000 ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --no-flash --external-device nvme0n1p1 -i ./ekb.key -S 100GiB -c ./tools/kernel_flash/flash_l4t_t234_nvme_rootfs_enc.xml --external-only --append --network usb0 jetson-orin-nano-devkit external

Hi Jerry,

Using
ROOTFS_GIB=2688GiB # size of APP_ENC partition
EXT_NUM_SECTORS=77578130000 # Around 3972 to give some margin

and the command

USER=root BOARDID=3767 FAB=300 BOARDSKU=0005 CHIP_SKU=D5 ROOTFS_ENC=1 EXT_NUM_SECTORS=77578130000./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --no-flash --external-device nvme0n1p1 -i /image-builder/work/Linux_for_Tegra/source/public/optee/samples/hwkey-agent/host/tool/gen_ekb/keys/sym2_t234.key -S 2688GiB -c ./tools/kernel_flash/flash_l4t_t234_nvme_rootfs_enc.xml -p --generic-passphrase --external-only --append --network usb0 --massflash 10 jetson-orin-nano-devkit external

Still gives an incorrect partition table


Fix/Ignore? Fix

Warning: Not all of the space available to /dev/nvme0n1 appears to be used, you can fix the GPT to use all of the space (an extra 3519069873 blocks) or continue with the current setting? 

Model: CT4000P3PSSD8 (nvme)

Disk /dev/nvme0n1: 4001GB

Sector size (logical/physical): 512B/512B

Partition Table: gpt

Disk Flags: 




Number  Start   End     Size    File system  Name                Flags

3      20.5kB  134MB   134MB                A_kernel            msftdata

4      134MB   135MB   786kB                A_kernel-dtb        msftdata

5      135MB   168MB   33.2MB               A_reserved_on_user  msftdata

6      168MB   302MB   134MB                B_kernel            msftdata

7      302MB   303MB   786kB                B_kernel-dtb        msftdata

8      303MB   336MB   33.2MB               B_reserved_on_user  msftdata

9      336MB   420MB   83.9MB               recovery            msftdata

10      420MB   421MB   524kB                recovery-dtb        msftdata

11      421MB   488MB   67.1MB  fat32        esp                 boot, esp

12      488MB   572MB   83.9MB               recovery_alt        msftdata

13      572MB   572MB   524kB                recovery-dtb_alt    msftdata

14      572MB   639MB   67.1MB               esp_alt             msftdata

15      639MB   1059MB  419MB                UDA                 msftdata

16      1059MB  1562MB  503MB                reserved            msftdata

1      1562MB  1981MB  419MB   ext4         APP                 msftdata

2      1981MB  689GB   687GB                APP_ENC             msftdata




[ 10]: l4t_flash_from_kernel: Successfully create gpt for external device

[ 10]: l4t_flash_from_kernel: Starting to flash to emmc

hello user_rw23,

BTW, had you tried to format this 4TB SSD?

Hi jerry,

Yes the SSD goes to the correct size if I flash JP5.1.3 with SDK manager

hello user_rw23,

this looks incorrect.

it’s 512 bytes for each sector, so.. you should configure EXT_NUM_SECTORS = 5662310500 which is around 2700GiB