Disk encryption on SD card without secure boot

Hi, Jerry.

I’ve run sudo ROOTFS_ENC=1 ./flash.sh -i ./sym2_t234.key jetson-orin-nano-devkit internal command on clean sources version r36.4.3.
The result is the same as previously - UEFI exiting with error.
Please find attached boot log.
boot-log.txt (37.6 KB)

Please note that I’m not enabling secure boot and using zero filled keys for disk encryption.
Could it be source of my problems?
Have you tried to run it on your board with similar configurations?

hello ValeriyI,

we’ve tested disk encryption without secure boot, but we did not tested with zero keys.
please double check you’ve using the script file (i.e. gen_ekb.py) which match your L4T release version to re-generate EKS image.

Hi Jerry,

I’ve generated fresh eks image and encryption key.
The result is still the same. I’m running this command:
sudo ROOTFS_ENC=1 ./flash.sh -i ./sym2_t234.key jetson-orin-nano-devkit internal
And have this result in boot log:

....L4TLauncher: Attempting Direct Boot
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services...
��ERROR:   **************************************
ERROR:   RAS Uncorrectable Error in ACI, base=0xe01a000:
ERROR:          Status = 0xec000504
ERROR:   SERR = Assertion failure: 0x4
ERROR:          IERR = AR decode Error: 0x5
ERROR:          Overflow (there may be more errors) - Uncorrectable
ERROR:          MISC0 = 0x1
ERROR:          MISC1 = 0x0
ERROR:          MISC2 = 0x0
ERROR:          MISC3 = 0x0
ERROR:          ADDR = 0x8000fffffffff000
ERROR:   **************************************
ERROR:   sdei_dispatch_event returned -1
ERROR:   Powering off core

hello ValeriyI,

FYI,
we’ve test again with fresh JP-6.2 environment to verify it works on Orin-Nano SD.

Hi, @JerryChang,

Thank you for reply.
The command doesn’t work for me.
I’m still having a crash after UEFI boot as in my previous messages.
Here is my top level script that does the build.
Maybe you can spot something unusual.

#!/bin/bash

set -e

trap "echo 'Error: $BASH_SOURCE:$LINENO $BASH_COMMAND' >> error.log" ERR

DATETIME_DIR=$(date +"%Y%m%d_%H%M%S")

function prepare_build() {
    sudo rm -rd Linux_for_Tegra || true

    VERSION=R36.4.3 
    DOWNLOADS_PATH="../downloads/$VERSION"

    tar xvf $DOWNLOADS_PATH/Jetson_Linux_${VERSION}_aarch64.tbz2
    sudo tar xvf $DOWNLOADS_PATH/Tegra_Linux_Sample-Root-Filesystem_${VERSION}_aarch64.tbz2 -C Linux_for_Tegra/rootfs
    tar xfv $DOWNLOADS_PATH/public_sources.tbz2  

    pushd Linux_for_Tegra/source
    tar xvf nvidia-jetson-optee-source.tbz2

    pushd optee/samples/hwkey-agent/host/tool/gen_ekb

    sed -i '8s/^# //' example.sh
    sed -i '13s/^# //' example.sh
    sed -i '15s/^# //' example.sh
    sed -i '17s/^# //' example.sh

    ./example.sh

    popd
    popd 

    pushd Linux_for_Tegra

    cp source/optee/samples/hwkey-agent/host/tool/gen_ekb/eks_t234.img ./bootloader/eks_t234.img
    cp source/optee/samples/hwkey-agent/host/tool/gen_ekb/sym2_t234.key ./sym2_t234.key

    sudo ./apply_binaries.sh
    popd
}

function create_log_dir() {
    # Create directory with current date and time
    mkdir -p logs/"$DATETIME_DIR"
}

function flash_rootfs_encryption_one_stage() {
    # UEFI boot error
    sudo ROOTFS_ENC=1 ./flash.sh -i ./sym2_t234.key jetson-orin-nano-devkit internal  \
    | tee -a "../logs/$DATETIME_DIR/combined-build-flash.txt"
}

create_log_dir
prepare_build
pushd Linux_for_Tegra
flash_rootfs_encryption_one_stage
popd

Could you please provide your build log?
Maybe I can find the problem by comparing my build log with yours.

hello ValeriyI,

please try running apply_binaries.sh first.
it’s to ensure you’re using eks_t234.img from gen_ekb.py.
for instance,

BTW,
please double check below, we’re only using sym2_t234 to create EKS image, eks_t234.img.
for instance,

 echo "0000000000000000000000000000000000000000000000000000000000000000" > oem_k1.key
 echo "0000000000000000000000000000000000000000000000000000000000000000" > sym_t234.key
 echo "00000000000000000000000000000000" > auth_t234.key
 openssl rand -rand /dev/urandom -hex 16 > sym2_t234.key   # disk encryption key

 python3 gen_ekb.py -chip t234 -oem_k1_key oem_k1.key \
        -in_sym_key sym_t234.key \
        -in_sym_key2 sym2_t234.key \
        -in_auth_key auth_t234.key \
        -in_ftpm_sn 00000000000000000000 \
        -in_ftpm_eps_seed ftpm_eps_seed_file \
        -in_ftpm_rsa_ek_cert ftpm_rsa_ek_cert.der \
        -in_ftpm_ec_ek_cert ftpm_ec_ek_cert.der \
        -in_sid_cert sid_cert.der \
        -in_ftpm_rsa_ek_csr ftpm_rsa_ek_csr.der \
        -in_ftpm_ec_ek_csr ftpm_ec_ek_csr.der \
        -out eks_t234.img

@JerryChang I’m having this error with and without ROOTFS_ENC=1. Seems it’s not related to disk encryption.

hello ValeriyI,

did you meant you still reproduce the same failure by flashing JP-6.2 onto Orin Nano SD directly?
ERROR: RAS Uncorrectable Error in ACI, base=0xe01a000:

@JerryChang yes, when I use command suggested by you ./flash.sh, but without ROOTFS_ENC=1 I’m still having this error ERROR: RAS Uncorrectable Error in ACI, base=0xe01a000:

But when I use l4t_initrd_flash.sh command without ROOTFS_ENC=1 it works.

It seems that ./flash.sh isn’t working for me either way.

please see-also Jetson Orin Nano Developer Kit Getting Started Guide | NVIDIA Developer, did you update the firmware accordingly?

@JerryChang I did firmware upgrade from version 5 to 6 according to the guide.
Device boots without errors if I flash it using l4t_initrd_flash.sh without rootfs encryption.
When I flash using method that you provided flash.sh with or without rootfs encryption - I have the same boot error:
ERROR: RAS Uncorrectable Error in ACI, base=0xe01a000:

hello ValeriyI,

l4t_initrd_flash.sh should works with ROOTFS_ENC=1 after you upgrade device firmware already update to the latest version.
for instance,
Jetson System firmware version 36.4.3-gcid-38968081 date 2025-01-08T01:18:20+00:00

we’ve tested again to flash on Orin-Nano with disk encryption on SD card, confirmed it’s working.
here’re steps for your reference,
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --network usb0 --no-flash --showlogs -p "-c bootloader/generic/cfg/flash_t234_qspi.xml" jetson-orin-nano-devkit internal
$ sudo ROOTFS_ENC=1 ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --no-flash --external-device mmcblk0p1 -i ./sym2_t234.key -c ./tools/kernel_flash/flash_l4t_t234_nvme_rootfs_enc.xml --external-only --append --network usb0 jetson-orin-nano-devkit external
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --showlogs --network usb0 --flash-only

note…
that 2nd steps is using external-device mmcblk0p1.
you may see-also To Flash the Jetson Developer Kit Operating Software of [Jetson Orin Nano Developer Kit (SD card)] for reference.

HI, @JerryChang

Here is my firmware version:
Jetson UEFI firmware (version 36.4.3-gcid-38968081 built on 2025-01-08T01:18:20+00:00)

I’ve tried the steps you advised and got the same result as before - kernel panic after pcie initialization.

Could you please advise how I can debug the issue?
Do you want me to give you more info about my setup or any additional logs?

Looking forward for your advise.
Thanks

hello ValeriyI,

so.. such failure RAS Uncorrectable Error in ACI, base=0xe01a000: has gone?
you’ve boot into kernel? and.. we’re now looping back to 1st comment, right?

[    8.524256] tegra194-pcie 14160000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[    8.716865] Kernel panic - not syncing:
[    8.716870] Attempted to kill init! exitcode=0x00007f00
[    8.716874] CPU: 3 PID: 1 Comm: bash Not tainted 5.15.148-tegra #1
[    8.716879] Hardware name: NVIDIA NVIDIA Jetson Orin Nano Developer Kit/Jetson, BIOS 36.4.0-gcid-37537400 09/13/2024
[    8.716881] Call trace:
[    8.716881]  dump_backtrace+0x0/0x1d0
[    8.716895]  show_stack+0x34/0x50
[    8.716899]  dump_stack_lvl+0x68/0x8c
[    8.716905]  dump_stack+0x18/0x3c
[    8.716906]  panic+0xc4/0x398
[    8.716912]  do_exit+0xa04/0xa30
[    8.716916]  do_group_exit+0x44/0xb0
[    8.716919]  __arm64_sys_exit_group+0x2c/0x30
[    8.716922]  invoke_syscall+0x5c/0x150
[    8.716927]  el0_svc_common.constprop.0+0x64/0x120
[    8.716930]  do_el0_svc+0x74/0xb0
[    8.716933]  el0_svc+0x28/0x90
[    8.716936]  el0t_64_sync_handler+0xac/0x130

Hi @JerryChang,
I have different issues for different flash methods.
When I’m using ./flash.sh I have error in UEFI RAS Uncorrectable Error in ACI, base=0xe01a000:
When I’m using ./tools/kernel_flash/l4t_initrd_flash.sh I can boot kernel and have pcie error like in first comment.

hello ValeriyI,

please use l4t_initrd_flash.sh to flash your target,
regrading to PCIE errors, it may due to PCIE controller setups. is the error always happened after " tegra194-pcie 14160000.pcie:"?

@JerryChang yes, it looks like error is always in the same place

Hi,

Is is possible to disable pcie 14160000 in the device tree and try again?

Just an experiment to check if pcie is related to this crash.

Hi, @WayneWWW!

I’ve tried your suggestion.
I’ve followed the this guide.
Here are my steps, just for reference, to make sure I did all correctly:

  1. Download public sources and unpack them
  2. Build kernel, modules, update initrd, and copy to flash dir
  3. Modify file tegra234-p3768-0000.dtsi. Find pcie@14160000 declaration, change status to ‘disabled’
  4. Build dtbs, copy to dir folder
  5. Unpack tegra234-p3768-0000+p3767-0005.dtb and verify that pcie@14160000 is disabled
  6. Flash board with ROOTFS_ENC=1 using l4t_initrd_flash.sh script

Here is the boot log

[    9.688016] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    9.688019] pci 0001:00:00.0:   bridge window [io  0x1000-0x1fff]
[    9.688024] pci 0001:00:00.0:   bridge window [mem 0x20a8000000-0x20a80fffff]
[    9.688164] pcieport 0001:00:00.0: Adding to iommu group 4
[    9.688285] pcieport 0001:00:00.0: PME: Signaling with IRQ 191
[    9.688414] pcieport 0001:00:00.0: AER: enabled with IRQ 191
[    9.689357] tegra194-pcie 141e0000.pcie: Adding to iommu group 6
[    9.692099] tegra194-pcie 141e0000.pcie: host bridge /bus@0/pcie@141e0000 ranges:
[    9.692126] tegra194-pcie 141e0000.pcie:      MEM 0x3000000000..0x3227ffffff -> 0x3000000000
[    9.692135] tegra194-pcie 141e0000.pcie:      MEM 0x3228000000..0x322fffffff -> 0x0040000000
[    9.692140] tegra194-pcie 141e0000.pcie:       IO 0x003e100000..0x003e1fffff -> 0x003e100000
[    9.692701] tegra194-pcie 141e0000.pcie: iATU unroll: enabled
[    9.692705] tegra194-pcie 141e0000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[    9.863948] Kernel panic - not syncing:
[    9.863954] Attempted to kill init! exitcode=0x00007f00
[    9.863960] CPU: 2 PID: 1 Comm: bash Tainted: G            E     5.15.148-tegra #1
[    9.863964] Hardware name: NVIDIA NVIDIA Jetson Orin Nano Developer Kit/Jetson, BIOS 36.4.3-gcid-38968081 01/08/2025
[    9.863966] Call trace:
[    9.863967]  dump_backtrace+0x0/0x1c0
[    9.863981]  show_stack+0x34/0x50
[    9.863985]  dump_stack_lvl+0x68/0x84
[    9.863991]  dump_stack+0x18/0x34
[    9.863993]  panic+0xc4/0x398
[    9.863999]  do_exit+0xa04/0xa30
[    9.864003]  do_group_exit+0x44/0xb0
[    9.864006]  __arm64_sys_exit_group+0x2c/0x30
[    9.864009]  invoke_syscall+0x5c/0x130
[    9.864013]  el0_svc_common.constprop.0+0x64/0x110
[    9.864017]  do_el0_svc+0x74/0xa0
[    9.864020]  el0_svc+0x28/0x80
[    9.864023]  el0t_64_sync_handler+0xa4/0x130
[    9.864026]  el0t_64_sync+0x1a4/0x1a8

As you can see now kernel panic appears after another device.
So I think it’s not related to pcie. It seems to me the problem is inside init script. Somewhere around rootfs decryption.