Postmortem: Jetson Xavier AGX will not get past Boot ROM after burning PKC+SBK+KEK256

We are working with a customer to enable secure boot (and later disk encryption).

For our customer’s Jetson Xavier AGK Dev Kit, using L4T-tegra-32.5.2, we generated the encryption keys with:

#!/bin/bash

echo "Generating PKC (Public Key Cryptogrpahy = RSA 3k Private key pem)"
 openssl genrsa -out rsa_priv.pem 3072
echo "Generating PKC (Public Key Cryptogrpahy = RSA 3k  Public key pem)"
 openssl rsa -in rsa_priv.pem -outform PEM -pubout -out rsa_pub.pem

echo "Generating KEK2 (Key Encryption Key)"
 openssl rand -rand /dev/urandom -hex 16 > kek2_hex_file

echo "Generating KEK256 (Key Encryption Key)"
 openssl rand -rand /dev/urandom -hex 32 > kek256_hex_file

echo "Generating SBK (Secure Boot Key)"
raw_hex=$(openssl rand -rand /dev/urandom -hex 16)
echo "0x${raw_hex:0:8} 0x${raw_hex:8:8} 0x${raw_hex:16:8} 0x${raw_hex:(-8)}" > sbk_hex_file

echo "Generating user_key"
raw_hex=$(openssl rand -rand /dev/urandom -hex 16)
echo "0x${raw_hex:0:8} 0x${raw_hex:8:8} 0x${raw_hex:16:8} 0x${raw_hex:(-8)}" > user_key_hex_file

We then tested the keys with:

#!/bin/sh
sudo FAB=400 BOARDID=2888 BOARDSKU=0001 BOARDREV=L.0 ./odmfuse.sh -i 0x19 --noburn --auth SBKPKC -k rsa_priv.pem --KEK256 kek256_hex_file --KEK2 kek2_hex_file -S sbk_hex_file jetson-agx-xavier-devkit

which produced the expected fuseblob.tbz2 with no errors.

We proceeded to test the keys before burning fuses with:

#!/bin/sh

sudo ./odmfuse.sh -i 0x19 --test -k rsa_priv.pem --KEK256 kek256_hex_file --KEK2 kek2_hex_file -S sbk_hex_file jetson-agx-xavier-devkit

Again, no errors.

So we dropped the --test and burned the fuses. Again no errors.

After this the board would never come out of Boot ROM. When trying to flash, we see the following (a sure sign the board is in an unrecoverable state):

Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands

[   0.0147 ] Parsing partition layout
[   0.0165 ] tegraparser_v2 --pt secureflash.xml.tmp
[   0.0194 ]
[   0.0195 ] Boot Rom communication
[   0.0211 ] tegrarcm_v2 --chip 0x19 0 --rcm rcm_1_signed.rcm --rcm rcm_2_signed.rcm
[   0.0227 ] BR_CID: 0xd8021911645995822400000015028280
[   0.0242 ] Bootrom returned error 22
[   0.0252 ] Boot Rom communication failed
[   0.0252 ]
Error: Return value 22

What could be the cause of the board failing to come out of Boot ROM? What is wrong with the scripts we used to generate the keys, test the keys, and ultimately burn the fuses? We do not want to “brick” another expensive (and in fact no longer available to purchase without long lead times) board.

hello ticotimo,

to burning Xavier series, please program all the fuse, and odm production modes at once.
please also refer to Topic 123506 and Topic 176834 as see-also. thanks

I have updated to 32.7.1, but I still have a couple of concerns:

  1. tegrasign_v3.py has #!/usr/bin/env python but this is not valid in a default Ubuntu 18.04 installation. Was this intended to be #!/usr/bin/env python3?

  2. Why is the odmfuse_pkc.xml from the --noburn mode different than the --test mode? The fuseblob.tbz2 version seems to be missing several fields. Attached are the two output logs (the command run is at the top of each log).
    gen-factory.log (76.9 KB)
    test.log (82.7 KB)

<genericfuse MagicId="0x45535546" version="1.0.0">
<fuse name="Kek2" size="16" value="206f34b0fd5072c15e683d5a0e6e3a77" />
<fuse name="Kek256" size="32" value="0a90d71dcdfb5930ebc8e7a6a85ae80eb140b691d0c9e7751f676443d560376f" />
<fuse name="SecurityMode" size="4" value="0x1" />
</genericfuse>

compared to

<genericfuse MagicId="0x45535546" version="1.0.0">
<fuse name="SecureBootKey" size="16" value="0x94f769b99cb6903309cfda062751216b" />
<fuse name="Kek2" size="16" value="206f34b0fd5072c15e683d5a0e6e3a77" />
<fuse name="Kek256" size="32" value="0a90d71dcdfb5930ebc8e7a6a85ae80eb140b691d0c9e7751f676443d560376f" />
<fuse name="PublicKeyHash" size="32" value="0xed32eb0a8e1cb8de800361a4618a9753051285afa795881d0c9cd6dda6f17ce8" />
<fuse name="BootSecurityInfo" size="4" value="0x6" />
<fuse name="SecurityMode" size="4" value="0x1" />
</genericfuse>
  1. Is RSA 3072 bit key known to work with odmfuse.sh? I want to make certain it has been tested.

  2. Can you confirm the following command is correct and safe to execute? I do not want to “brick” another board.

sudo ./odmfuse.sh -i 0x19 -c SBKPKC -p -k ../rsa_priv.pem -S ../sbk_hex_file --KEK2 ../kek2_hex_file --KEK256 ../kek256_hex_file jetson-agx-xavier-devkit

hello ticotimo,

Q1>
ya, that should be using pyhton3, it should be comment messages did not up to date.

Q2>

  • command-line with --noburn.
    • sudo FAB=400 BOARDID=2888 BOARDSKU=0001 BOARDREV=L.0 ./odmfuse.sh -i 0x19 -p --noburn --auth SBKPKC -k ../rsa_priv.pem --KEK2 ../kek2_hex_file --KEK256 ../kek256_hex_file -S ../sbk_hex_file jetson-agx-xavier-devkit
    • this command-line is running at offline mode, in the offline mode, the --auth options is used to indicate the board fuse status, because there is no boards connected, so odmfuse.sh doesn’t have a chance to know the board’s fuse status.
    • in this scenario… since the target has burned with PKC+SBK, -k is used to sign the encrypted images; -S is the command to encrypt the images. in the end, there only --KEK2 and KEK256 will be burned to the board, then it appears in the odmfuse_pkc.xml.
  • command-line with --test.
    • sudo ./odmfuse.sh --test -i 0x19 -c SBKPKC -p -k ../rsa_priv.pem -S ../sbk_hex_file --KEK2 ../kek2_hex_file --KEK256 ../kek256_hex_file jetson-agx-xavier-devkit
    • since there is neither board info nor --noburn in the command, odmfuse.sh will run in the online mode, which means a board must be connected; -c options is no needed in the online mode since odmfuse.sh is now able to get the board’s fuse status from the target. besides, the option -c is obsolete now. it’s replace by --auth options.
    • I assume you’re test this pipeline with unfused target, right? you should assign --auth NS options to the command-line due to neither PKC nor SBK is burned.
    • following above, for these options in the command-line, all fuse info will be burn to the target, such as… -p, -k, -S, -KEK2, -KEK256. so, that’s why odmfuse_pkc.xml include all those fuse information.

Q3>
the 3072-bit RSA key option is supported only on Jetson Xavier series.

Q4>
please use the multi-steps approaches to burn the fuse and flash the target.
for example, (1) using odmfuse.sh to create the fuse blob, (2) review all the fuse info and messages, running fusecmd.sh to burn the fuse to the device actually, (3) it’s flash.sh script to create the image blob, and (4) run flashcmd.txt to flash the target. please see-also this thread for reference, Jetson Xavier NX DEVKIT secureboot enabled - #7 by JerryChang

1 Like

hello ticotimo,

please revisit comment #5,
I’ve revise my previous comments for making all answers in the same reply.
thanks

Aha! So the --auth NS is the current state of the board (unfused target). And as you said, this is only needed in ‘off-line’ mode because no communication is happening with the board.

I was also not realizing that the -c option is not needed in ‘on-line’ mode because the actual state of the fuses is being read. I thought this was needed to set the mode (from NS to PKC or SBKPKC).

So the proper command is:

sudo FAB=400 BOARDID=2888 BOARDSKU=0001 BOARDREV=L.0 ./odmfuse.sh -i 0x19 -p --noburn --auth NS -k ../rsa_priv.pem --KEK2 ../kek2_hex_file --KEK256 ../kek256_hex_file -S ../sbk_hex_file jetson-agx-xavier-devkit

?
This will then generate the proper odmfuse_pkc.xml with all the fuse options included, so that one can proceed with fusecmd.sh.

Ok. The result makes sense now (fuseblob.tbz2 is identical to --test mode):

<genericfuse MagicId="0x45535546" version="1.0.0">
<fuse name="SecureBootKey" size="16" value="0x94f769b99cb6903309cfda062751216b" />
<fuse name="Kek2" size="16" value="206f34b0fd5072c15e683d5a0e6e3a77" />
<fuse name="Kek256" size="32" value="0a90d71dcdfb5930ebc8e7a6a85ae80eb140b691d0c9e7751f676443d560376f" />
<fuse name="PublicKeyHash" size="32" value="0xed32eb0a8e1cb8de800361a4618a9753051285afa795881d0c9cd6dda6f17ce8" />
<fuse name="BootSecurityInfo" size="4" value="0x6" />
<fuse name="SecurityMode" size="4" value="0x1" />
</genericfuse>

NOTE: you must run sudo ./fusecmd.sh

@JerryChang NOTE: neither sudo ./flash.sh –-no-flash --sign -u rsa_pub.pem -v sbk_hex_file jetson-agx-xavier-devkit mmcblk0p1 nor sudo ./flash.sh –-no-flash --sign -u rsa_priv.pem -v sbk_hex_file jetson-agx-xavier-devkit mmcblk0p1 works. Both tell me that the RSA or SBK key has not been provided. The rsa is 3k (3072) if that matters.

However, sudo ./flash.sh --no-flash -u ../rsa_priv.pem -v ../sbk_hex_file jetson-agx-xavier-devkit mmcblk0p1 does successfully generate the flashcmd.txt:

./tegraflash.py --bl nvtboot_recovery_cpu_t194.bin_sigheader.encrypt.signed --bct br_bct_BR.bct --applet rcm_2_signed.rcm --applet_softfuse rcm_1_signed.rcm --cmd "secureflash;reboot"  --cfg secureflash.xml --chip 0x19 --mb1_bct mb1_bct_MB1.bct_sigheader.encrypt.signed --mem_bct mem_rcm.bct_sigheader.encrypt.signed --mb1_cold_boot_bct mb1_cold_boot_bct_MB1.bct_sigheader.encrypt.signed --mem_bct_cold_boot mem_coldboot_sigheader.bct.signed  --bins "mb2_bootloader nvtboot_recovery_t194.bin_sigheader.encrypt.signed; mts_preboot preboot_c10_prod_cr_sigheader.bin.encrypt.signed; mts_mce mce_c10_prod_cr_sigheader.bin.encrypt.signed; mts_proper mts_c10_prod_cr_sigheader.bin.encrypt.signed; bpmp_fw bpmp_t194_sigheader.bin.encrypt.signed; bpmp_fw_dtb tegra194-a02-bpmp-p2888-a01_sigheader.dtb.encrypt.signed; spe_fw spe_t194_sigheader.bin.encrypt.signed; tlk tos-trusty_t194_sigheader.img.encrypt.signed; eks eks_sigheader.img.encrypt.signed; bootloader_dtb tegra194-p2888-0001-p2822-0000_sigheader.dtb.encrypt.signed"

And sudo bash ./flashcmd.txt did successfully flash the board (after the fuses had been burned with sudo ./flashcmd.sh).

It seems to me that the commands should be either providing better warnings or throwing errors. This was a very confusing and nerve wracking experience. It should not be.

I will also note that when I tried to perform the same operations with 32.6.1 I came across the error in orrupted-kernel-headers-package-in-l4t-32-6-1. This is on a bare-metal Ubuntu 18.04 host.

hello ticotimo,

it’s the --sign to create signed images to your local host instead of flashing the target.

        --sign --------------- Sign images and store them under "bootloader/signed"
                               directory. The board will not be physically flashed.

your workable commands has removed that so the image blob it’s finally created.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.