Will not boot after enabling Security Boot (Jetson AGX Xavier)

[JulienMoinard]

Dear JerryChang,

Thank you so much I understand now why it doesn’t work.

For information the path is wrong the right one seems to be :
$L4T_Sources/r32.5/Linux_for_Tegra/source/public/trusty_src.tbz2/trusty/app/sample/nvidia-sample/hwkey-agent/CA_sample/tool/gen_ekb/example.sh

But I am facing to an issue, I need two keys (one for kernel user_key and one for disk encryption provided with -i in flash.sh)

In documentation I can read usage 2 times of -in sym_key but in your example only one -in_sym_key can be use ?

In your documentation

python3 gen_ekb.py -kek2_key <kek2_fuse_key_file>
-fv <fv_for_ekb_ek>
-in_sym_key <sym_key_file>
-in sym_key2 <sym2_key_file>
-out <eks_image_file>

In your example script, only the first -in_sym is used ?

I need to duplicate load symmetric key code ?

with open(args.in_sym_key[0], ‘rb’) as infd:
tmp = infd.read().strip()
in_content += codecs.decode(tmp, ‘hex’)

I am verry close to have a secure boot with full encryption enabled.
I need your help to finish my project.

Thank you so much for your help.
Best regards,

[JerryChang]

@JulienMoinard

In your example script, only the first -in_sym is used ?

hello JulienMoinard,

did you download the latest r32.5 sources release package?
I can see two symmetric key files were used in the example,
for example,
gen_ekb/example.sh

python3 gen_ekb.py -kek2_key kek2_key \
        -fv fv_ekb \
        -in_sym_key sym.key \
        -in_sym_key2 sym2.key \
        -out eks.img

there’s also handing in the python scripts. ./gen_ekb/gen_ekb.py

def main():
    global verbose
    ...
    if not all(map(os.path.exists, [args.kek2_key[0], args.fv[0], args.in_sym_key[0], args.in_sym_key2[0]])):

    ...
    # load sym key file
    with open(args.in_sym_key[0], 'rb') as infd:
        tmp = infd.read().strip()
        in_content += codecs.decode(tmp, 'hex')

    with open(args.in_sym_key2[0], 'rb') as infd:
        tmp = infd.read().strip()
        in_content += codecs.decode(tmp, 'hex')

[JulienMoinard]

Ok thank so much, I probably use the wrong sources files but I click on your link in your last post so I don’t understand because the path is not right and it was an old file october 2020 with only one key…
Anyway that fine I add by myself modifications to use two keys.

So, I generate my custom eks.img with two custom key and put eks.img to bootloader folder.

But I am always stuck on nvidia logo at boot with
[0011.085] I> Copying kernel image (34609160 bytes) from 0xa4ad0000 to 0x80080000 … [0011.095] I> Done
[0011.095] E> fdt_open_into fail (FDT_ERR_BADMAGIC)
[0011.096] E> Error (727449637) extracting the kernel DTB
[0011.117] I> Kernel EP: 0x80080000, DTB: 0x90000000

Are you sure I need to put my eks.img into bootloader folder ? because each time I use flahs.sh the eks.img is rewritten.
I think my eks.img is not used due to rewrite by flahs.sh.

Do you know if it is normal that flash.sh change my eks.img file ?

Do you you think I need to add setting to flash.sh to use my custom eks.img and avoid rewriting eks.img from flash.sh ?

If I put two keys into eks.img how the bootloader choose the key for kernel, and how the kernel choose the right key for decrypt disk ? encrypt kernel is the first one and encryption disk is the second one ?

Thank you for your help.
Julien.

[JerryChang]

hello JulienMoinard,

I’ve download the L4T source package and check again, it’s r32_release_v5.1 source package, which used two symmetric key files in the implementation.
the download link to the sources should be correct, could you please helps to confirm this also.

@JulienMoinard

[0011.095] I> Done
[0011.095] E> fdt_open_into fail (FDT_ERR_BADMAGIC)
[0011.096] E> Error (727449637) extracting the kernel DTB

so, here’s an error of verification failed.
please make sure that the eks.img you generated is using the same user_key in flash.sh.

may I know what’s the security settings you’d enable for your Jetson AGX Xavier?
you could share the fuse commands and messages, or, you may share the details of fuse info for reference,
thanks

[JulienMoinard]

Hello,

If I download your sources file the path of exemple is wrong can you confirm me the right path to find the script with two keys ?

But can you confirm me, it is normal that flash.sh edit the eks.img each time you run flash.sh script ?

Thank you.

[JerryChang]

hello JulienMoinard,

it’s public_sources.tbz2 download to your local host,
please check you’re actually extract r32.5.1 source package, please also review the linkage.
for example,
https://developer.nvidia.com/embedded/l4t/r32_release_v5.1/r32_release_v5.1/sources/t186/public_sources.tbz2

there’re several packages, please un-tar trusty_src.tbz2 package for the tools, showing the content as below.

trusty/app/nvidia-sample/hwkey-agent/CA_sample/tool/gen_ekb/

$ ll
total 24
drwxr-xr-x 2 jerry jerry 4096 May  6 10:21 ./
drwxr-xr-x 3 jerry jerry 4096 Feb 20 00:34 ../
-rwxr-xr-x 1 jerry jerry  606 Feb 20 00:34 example.sh*
-rwxr-xr-x 1 jerry jerry 5792 Feb 20 00:34 gen_ekb.py*
-rw-r--r-- 1 jerry jerry 3805 Feb 20 00:34 README

so,
you need to edit example.sh to generate a proper eks.img,
for example,

1. kek2_key that is flashed to your board; (all 0’s is for un-fused board);
2. don’t change fv_ekb, i.e. please use bad66eb4484983684b992fe54a648bb8;
3. echo “<your user_key” to sym.key
4. echo “<your_disk_encrypt_key>” to sym2.key
5. execute ./example.sh

please also note that key format is different used in eks.img generation and flash.sh.
for example,
if a key, ffeeddccbbaa99887766554433221101 is used to generate eks.img.
the corresponding key, 0xffeeddcc 0xbbaa9988 0x77665544 0x33221101 MUST be used as user_key in flash command.

please share the details messages if you still meet failures,
thanks

[JulienMoinard]

Hello JerryChang,

With your instructions it seems to be better, In fact I have changed the fixed-vector and I need to use the default one like you advise me.
I can confirm to you that I found gen_eks.py in the archives like you explain so thank you so much.

authenticate_oem_payload: Decrypt the binary
[0011.357] I> Kernel hdr @0xa4ad0000
[0011.357] I> Kernel dtb @0x90000000
[0011.357] I> decompressor handler not found
[0011.357] I> Copying kernel image (34609160 bytes) from 0xa4ad0000 to 0x80080000 … [0011.368] I> Done
[0011.369] I> Updated bpmp info to DTB

But I am always stuck on NVIDIA logo , kernel is loaded

0011.626] I> Kernel EP: 0x80080000, DTB: 0x90000000
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.9.201-tegra (prod@ubuntu) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167e f90065c0eeb7f91977701] (Linaro GCC 7.3-2018.05) ) #1 SMP PREEMPT Thu Mar 18 20:17:56 CET 2021
[ 0.000000] Boot CPU: AArch64 Processor [4e0f0040]
[ 0.000000] OF: fdt:memory scan node memory, reg size 48,
[ 0.000000] OF: fdt: - 80000000 , 2c000000
[ 0.000000] OF: fdt: - ac200000 , 44800000
[ 0.000000] OF: fdt: - 100000000 , 180000000
[ 0.000000] earlycon: tegra_comb_uart0 at MMIO32 0x000000000c168000 (options ‘’)
[ 0.000000] bootconsole [tegra_comb_uart0] enabled
[ 0.000000] Found tegra_fbmem: 00800000@a069f000
[ 0.000000] Found lut_mem: 00002008@a069b000

The uart console stay block here without more messages ?

Do you have and idea ?
Thank you.

[JerryChang]

hello JulienMoinard,

could you please attach detail bootloader messages for reference,
thanks

[JulienMoinard]

Hello,

Please find detail bootloader messages.

Julien.
console.txt (23.8 KB)

[JerryChang]

hello JulienMoinard,

wait! are you working with Jetson Xavier NX instead of Jetson AGX Xavier?

[0001.871] I> create_pm_ids: id: 3668-0001-200-G, len: 15

[JulienMoinard]

Yes I am working on Jetson Xavier NX production (with eMMC) but the procedure is the same ? Not ?

[JerryChang]

hello JulienMoinard,

yes, the procedure should be the same.

although this topic is filed for Jetson AGX Xavier to enable SecureBoot. you’re having an advance use-case to enable user keys and corresponding eks.img to fully test the encryption/decryption.
please initial another discussion thread for following-up, please mark this as a see-also topic.
thanks

[JulienMoinard]

Dear JerryChang,

Thank you for your help with generation of eks.img .

For full encryption (bootloader,kernel and disk) on JetsonNX production module (eMMC) please follow this new topic.

Stuck at loading kernel after enable SecureBOOT and disk encryption

After enable secureBoot by generating custom eks.img (with 2 keys) and place it to bootloader folder and fuse keys needed with odmfuse.sh keys I am able to flash board with this command : sudo ./flash.sh -i /data/keys/disk.key -u /data/keys/rsa.key -v /data/keys/sbk.key --user_key /data/keys/user.hex cti/xavier-nx/photon-encrypted mmcblk0p1 I am able to start my board and the kernel was verified correctly; authenticate_oem_payload: Decrypt the binary [0011.357] I> Kernel hdr @0xa4ad0000 …

[dcapers44]

@JerryChang I have tried doing as you suggest back on April 13. After trying to boot after flashing the Jetson is frozen on the NVIDIA loge screen.

I used the EKB generation tool to create an EKS image:

python3 gen_ekb.py \
    -kek2_key KEK2_EKB.txt\
    -fv FV_EKB.txt \
    -in_sym_key User_Key_EKB.txt \
    -in sym_key2 DEK_EKB.txt\
    -out eks.img

FV_EKB.txt is set to bad66eb4484983684b992fe54a648bb8

And all of the in parameter file are formatted as stated in

@JerryChang

hello dcapers44,

moreover,
you have to first replace the eks.img in bootloader folder with the eks.img you built using your own user_key for image flashing.
please also note that key format is different used in eks.img generation and flash.sh.

for example,
if a key, ffeeddccbbaa99887766554433221101 is used to generate eks.img.
the corresponding key, 0xffeeddcc 0xbbaa9988 0x77665544 0x33221101 MUST be used as user_key in flash command.

I then proceeded to flash the jetson:

sudo cp ~/Linux_for_Tegra/source/public/atf_and_trusty/trusty/trusty/app/nvidia-sample/hwkey-agent/CA_sample/tool/gen_ekb/eks.img ~/Linux_for_Tegra/bootloader

sudo BOARDID=2888 FAB=400 BOARDSKU=0001 BOARDREV=H.0 ./flash.sh -u RSA_Key.pem -v SBK.txt --user_key User_Key.txt jetson-agx-xavier-devkit mmcblk0p1

Can someone please assist…

[JerryChang]

@dcapers44

After trying to boot after flashing the Jetson is frozen on the NVIDIA loge screen.

hello dcapers44,

may I have confirmation,
so, you’re able to enable sercureBoot. but you got booting failed after replace EKS image?
could you please also gather bootloader logs for reference.
thanks

[dcapers44]

dcapers44

@JerryChang I gave you confirmation that I was able to enable SecureBoot back on April 12.

Apr 12

@JerryChang I was able to boot my Jetson AG Xavier with the info you provided in your last post… Thank you.

Now I want to bring up the SecureBoot using a user key.

Yes, I did get booting to fail after replacing EKS image.

Where are the bootloader logs located so I can provide them to you?

[JerryChang]

@dcapers44

Where are the bootloader logs located so I can provide them to you?

hello dcapers44,

please setup serial console to gather UART logs and attach to the thread. just as same as you did in comment #3.

[JerryChang]

hello dcapers44,

this should be a related issue of not booting-up with customize eks image, Topic 177180.
here’s a bug fix in decrypting buffer, could you please apply this patch for the CBoot sources.
please also check below for reference,
thanks

Black Screen after enable SecureBOOT and disk encryption

hello JulienMoinard, here’s a bug fix in decrypting buffer, could you please apply this patch for the cboot sources. for example, 0001-lib-tegrabl_auth-fix-a-bug-in-decrypting-buffer.patch (1.8 KB) please download CBoot sources and check the readme file for the instructions to build CBoot binary. for example, $L4T_Sources/r32.5/Linux_for_Tegra/source/public/cboot/CBoot_Standalone_Readme_t194.txt thanks

[dcapers44]

@JerryChang error when trying to build CBoot Binary… I don’t understand

dcapers@NUC-Ubuntu-18:~/nvidia/Linux_for_Tegra/source/public/cboot$ make -C ./bootloader/partner/t18x/cboot PROJECT=t186 TOOLCHAIN_PREFIX="${CROSS_COMPILE}" DEBUG=2 BUILDROOT="${PWD}"/out NV_BUILD_SYSTEM_TYPE=l4t NOECHO=@
[sudo] password for dcapers: 
make: Entering directory '/home/dcapers/nvidia/Linux_for_Tegra/source/public/cboot/bootloader/partner/t18x/cboot'
LKROOT=. LKINC=". " make -rR -f ./engine.mk -I. 
make[1]: Entering directory '/home/dcapers/nvidia/Linux_for_Tegra/source/public/cboot/bootloader/partner/t18x/cboot'
engine.mk:40: *** TEGRA_TOP undefined.  Stop.
make[1]: Leaving directory '/home/dcapers/nvidia/Linux_for_Tegra/source/public/cboot/bootloader/partner/t18x/cboot'
makefile:13: recipe for target '_top' failed
make: *** [_top] Error 2
make: Leaving directory '/home/dcapers/nvidia/Linux_for_Tegra/source/public/cboot/bootloader/partner/t18x/cboot'

I was following the instructions from the CBoot_Standalone_Readme.txt:

To build the CBoot binary:

1. Extract the CBoot-standalone source with the command:
   mkdir cboot
   tar -xjf cboot_src.tbz2 -C cboot
   cd cboot

2. Export the cross compiler tools with the following enviroment variables:
   export CROSS_COMPILE=<your_64-bit_ARM_toolchain_triple>

   Where: <your_64-bit_ARM_toolchain_triple> can be: ‘aarch64-linux-gnu-’

3. Set the TEGRA_TOP and TOP environment variables:
   export TEGRA_TOP=$PWD
   export TOP=$PWD

4. Build the T186 CBoot binary, lk.bin, with the command:
   make -C ./bootloader/partner/t18x/cboot PROJECT=t186 TOOLCHAIN_PREFIX="${CROSS_COMPILE}" DEBUG=2 BUILDROOT="${PWD}"/out NV_BUILD_SYSTEM_TYPE=l4t NOECHO=@

[JerryChang]

@dcapers44

engine.mk:40: *** TEGRA_TOP undefined. Stop.

hello dcapers44,

here’s undefine error of TEGRA_TOP, how does $ echo $TEGRA_TOP shows?