Boot issue after capsule update with both disk-encryption and secureboot enabled from r35.4.1 to r35.6.0

Hi @KevinFFF ,

Referring to post :

I created a separate ticket with boot-issue along with steps to reproduce:

  1. Fuse a Jetson Xavier NX board with KEK, PKC,SBK keys
  2. Generate a QSPI+NVME massflash image from r35.4.1 and flash the jetson board
#On r35.4.1, generated the eks.img as follows 
pushd $ROOT_DIR/keys/optee
# This is the default initial vector for EKB.
echo "bad66eb4484983684b992fe54a648bb8" > fv_ekb_t194

python3 gen_ekb.py -chip t194 -kek2_key ../sd/kek2.txt \
        -fv fv_ekb_t194 \
        -in_sym_key ../sym_t194.key \
        -in_sym_key2 ../sym2_t194.key \
        -out eks_t194.img
#Copy the eks_t194.img into bootloader
cp eks_t194.img $ROOT_DIR/Linux_for_Tegra/bootloader/eks.img
echo "2. Copy ekb.key for fused board..."
cp $ROOT_DIR/keys/sym2_t194.key $ROOT_DIR/Linux_for_Tegra/ekb.key
popd



# Massflash image creation command using r35.4.1

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ROOTFS_ENC=1 BOARDID=3668 FAB=301 BOARDSKU=0001 BOARDREV=E.0 ./tools/kernel_flash/l4t_initrd_flash.sh -u $ROOT_DIR/keys/jetson.pem -v $ROOT_DIR/keys/sbk_key.txt  -p "-i ./ekb.key -c bootloader/t186ref/cfg/flash_l4t_t194_qspi_p3668.xml" --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_nvme_rootfs_enc.xml -S 40GiB --no-flash --massflash 8 --showlogs jetson-xavier-nx-devkit-emmc nvme0n1p1

#Command used to flash the massflash image on the board with r35.4.1
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --massflash 
  1. Once the jetson board is flashed with r35.4.1 massflash image, download the bsp r35.6.0 and generate a BUP payload along with capsule to update it’s bootloader.

Note: Before generating the BUP payload, eks.img is changed to r35.6.0 version with auth_key

# jetson_board_spec.cfg

t19x_spec=(
    # jetson-xavier-nx-devkit-emmc:
    'boardid=3668;fab=100;boardsku=0001;boardrev=;chiprev=2;board=jetson-xavier-nx-devkit-emmc;rootdev=mmcblk0p1'
    # jetson-xavier-nx-devkit-emmc A03:
    'boardid=3668;fab=301;boardsku=0001;boardrev=;chiprev=2;board=jetson-xavier-nx-devkit-emmc;rootdev=mmcblk0p1'
)

or (rootdev=nvme0n1p1) - tried both but none of them works ! 

t19x_spec=(
    # jetson-xavier-nx-devkit-emmc:
    'boardid=3668;fab=100;boardsku=0001;boardrev=;chiprev=2;board=jetson-xavier-nx-devkit-emmc;rootdev=nvme0n1p1'
    # jetson-xavier-nx-devkit-emmc A03:
    'boardid=3668;fab=301;boardsku=0001;boardrev=;chiprev=2;board=jetson-xavier-nx-devkit-emmc;rootdev=nvme0n1p1'
)


#eks.img is changed in r35.6.0 with auth_t194.key being used as all zeros.

echo "bad66eb4484983684b992fe54a648bb8" > fv_ekb_t194
echo "00000000000000000000000000000000" > auth_t194.key
python3 gen_ekb.py -chip t194 -kek2_key ../sd/kek2.txt \
        -fv fv_ekb_t194 \
        -in_sym_key ../sym_t194.key \
        -in_sym_key2 ../sym2_t194.key \
        -in_auth_key auth_t194.key \
        -out eks_t194.img
#Copy the eks_t194.img into bootloader
cp eks_t194.img $ROOT_DIR/Linux_for_Tegra/bootloader/

cp $ROOT_DIR/keys/sym2_t194.key $ROOT_DIR/Linux_for_Tegra/ekb.key

popd


#Generated the payload

$ sudo ./l4t_generate_soc_bup.sh -u $ROOT_DIR/keys/jetson.pem -v $ROOT_DIR/keys/sbk_key.txt t19x
$ ./generate_capsule/l4t_generate_soc_capsule.sh -i bootloader/payloads_t19x/bl_only_payload -o ./TEGRA_BL.Cap t194

capsule_generation_logs.txt (2.9 KB)

bup_generation_logs.txt (202.1 KB)

  1. Transfer the TEGRA_BL.Cap to target board and run the below steps:
sudo mkdir -p /opt/nvidia/esp
esp_uuid=$(lsblk -o name,partlabel,uuid | grep "nvme0n1" | awk '{ if($2 == "esp") print $3 }')
sudo mount UUID=$esp_uuid /opt/nvidia/esp
sudo mkdir -p /opt/nvidia/esp/EFI/UpdateCapsule
sudo cp TEGRA_BL.Cap /opt/nvidia/esp/EFI/UpdateCapsule

cd /sys/firmware/efi/efivars/
printf "\x07\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00" > /tmp/var_tmp.bin
sudo dd if=/tmp/var_tmp.bin of=OsIndications-8be4df61-93ca-11d2-aa0d-00e098032b8c bs=12;sync
sudo umount /opt/nvidia/esp

sudo reboot 

  1. On reboot, the Update progress bar starts (goes to 100% ) but fails :

Boot-logs:


Jetson UEFI firmware (version 4.1-33958178 built on 2023-08-01T19:34:02+00:00)
ESC   to enter Setup.
F11   to enter Boot Manager Menu.
Enter to continue boot.

Update Progress - 100% **************************************************����
[0000.036] W> RATCHET: MB1 binary ratchet value 4 is larger than ratchet level 2 from HW fuses.
[0000.044] I> MB1 (prd-version: 2.6.0.0-t194-41334769-cab45716)
[0000.049] I> Boot-mode: Coldboot
[0000.052] I> Platform: Silicon
[0000.055] I> Chip revision : A02P
[0000.058] I> Bootrom patch version : 15 (correctly patched)
[0000.063] I> ATE fuse revision : 0x200
[0000.067] I> Ram repair fuse : 0x0
[0000.070] I> Ram Code : 0x0
[0000.072] I> rst_source: 0xb, rst_level: 0x1
[0000.077] I> Boot-device: QSPI (instance: 0)
[0000.081] I> Qspi flash params source = brbct
[0000.085] I> Qspi clock source : pllp
[0000.088] I> Qspi-0 initialized successfully
[0000.092] I> Boot chain mechanism: A/B
[0000.096] I> Current Boot-Chain Slot: 1
[0000.099] I> BR-BCT Boot-Chain: 1, status: 0. update flag: 0
[0000.105] I> Qspi flash params source = brbct
[0000.113] W> PROD_CONFIG: device prod data is empty in MB1 BCT.
[0000.118] I> Temperature = 47500
[0000.121] W> Skipping boost for clk: BPMP_CPU_NIC
[0000.126] W> Skipping boost for clk: BPMP_APB
[0000.130] W> Skipping boost for clk: AXI_CBB
[0000.134] W> Skipping boost for clk: AON_CPU_NIC
[0000.138] W> Skipping boost for clk: CAN1
[0000.141] W> Skipping boost for clk: CAN2
[0000.146] I> Boot-device: QSPI (instance: 0)
[0000.150] I> Qspi flash params source = mb1bct
[0000.154] I> Qspi clock source : pllc_out0
[0000.158] I> Qspi-0 reinitialized
[0000.161] I> Qspi flash params source = mb1bct
[0000.176] I> Non-ECC region[0]: Start:0x80000000, End:0x100000000
[0000.183] W>  Thermal config not found in BCT
[0000.191] W>  MEMIO rail config not found in BCT
[0000.201] I> Qspi flash params source = mb1bct
[0000.217] I> Qspi flash params source = mb1bct
[0000.262] I> Qspi flash params source = mb1bct
[0000.851] I> Qspi flash params source = mb1bct
[0000.879] I> Qspi flash params source = mb1bct
[0000.922] W>  Platform config not found in BCT
[0000.949] I> Qspi flash params source = mb1bct
[0000.987] I> MB1 done

����main enter
SPE VERSION #: R01.00.18 Created: Jan 29 2021 @ 14:18:27
HW Function test
Start Scheduler.
in late init
��
  [0000.996] I> Welcome to MB2(TBoot-BPMP) (version: default.t194-mobile-8e4b789e)
[0000.997] I> DMA Heap @ [0x526fa000 - 0x52ffa000]
[0000.997] I> Default Heap @ [0xd486400 - 0xd48a400]
[0000.998] E> DEVICE_PROD: Invalid value data = 70020000, size = 0.
[0001.004] W> device prod register failed
[0001.008] I> gpio framework initialized
[0001.011] I> tegrabl_gpio_driver_register: register 'nvidia,tegra194-gpio' driver
[0001.019] I> tegrabl_gpio_driver_register: register 'nvidia,tegra194-gpio-aon' driver
[0001.026] I> No valid sdcard_params in mb1_bct
[0001.031] I> Boot_device: QSPI_FLASH instance: 0
[0001.035] I> qspi flash-0 params source = boot args
[0001.041] I> QSPI-0l initialized successfully
[0001.044] I> sdmmc-3 params source = safe params
[0001.385] I> sdmmc DDR50 mode
[0001.403] I> Found 41 partitions in QSPI_FLASH (instance 0)
[0001.420] W> Cannot find any partition table for 00000003
[0001.421]  > PARTITION_MANAGER: Failed to publish partition.
[0001.437] I> Found 22 partitions in SDMMC_USER (instance 3)
[0001.438] I> Active Boot chain : 1
[0001.455] I> RSA PSS signature check: OK
[0001.500] I> RSA PSS signature check: OK
[0002.020] I> Relocating BR-BCT
[0002.021]  > DEVICE_PROD: device prod is not initialized.
[0002.047] E> I2C: slave not found in slaves.
[0002.048] E> I2C: Could not write 0 bytes to slave: 0x00ae with repeat start true.
[0002.049] E> I2C_DEV: Failed to send register address 0x00000000.
[0002.050] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xae at 0x00000000 via instance 0.
[0002.051] E> eeprom: Failed to read I2C slave device
[0002.054] I> Failed to read CVB eeprom data @ AE
[0002.058] I> Retrying CVB eeprom read @ AC ...
[0002.063] E> I2C: slave not found in slaves.
[0002.067] E> I2C: Could not write 0 bytes to slave: 0x00ac with repeat start true.
[0002.075] E> I2C_DEV: Failed to send register address 0x00000000.
[0002.081] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xac at 0x00000000 via instance 0.
[0002.090] E> eeprom: Failed to read I2C slave device
[0002.095] I> Failed to read CVB eeprom data @ AC
[0002.114] I> RSA PSS signature check: OK
[0002.133] I> RSA PSS signature check: OK
[0002.158] I> RSA PSS signature check: OK
[0002.215] I> Relocating OP-TEE dtb from: 0x6bfff720 to 0x70050000, size: 1008
[0002.216] I> [0] START: 0x80000000, SIZE: 0x2f000000
[0002.216] I> [1] START: 0xaf010000, SIZE: 0x189f0000
[0002.217] I> [2] START: 0xc7b00000, SIZE: 0xc0000
[0002.217] I> [3] START: 0xca000000, SIZE: 0x800000
[0002.218] I> dram_block larger than 80000000
[0002.220] I> [4] START: 0x100000000, SIZE: 0x180000000
[0002.231] I> Setting NS memory ranges to OP-TEE dtb finished.
[0002.236] I> RSA PSS signature check: OK
[0002.239] I> found decompressor handler: lz4
[0002.489] I> RSA PSS signature check: OK
[0002.521] I> RSA PSS signature check: OK
[0002.522] I> EKB detected (length: 0x410) @ VA:0x52709400
[0002.524] I> Setting EKB blob info to OPTEE dtb finished.
��NOTICE:  BL31: v2.6(release):5e1f8b33d
NOTICE:  BL31: Built : 01:45:47, Aug 28 2024
I/TC: Physical secure memory base 0xcb040000 size 0xf00000
I/TC: 
I/TC: Non-secure external DT found
I/TC: OP-TEE version: 3.22 (gcc version 9.3.0 (Buildroot 2020.08)) #2 Wed Aug 28 08:55:09 UTC 2024 aarch64
I/TC: WARNING: This OP-TEE configuration might be insecure!
I/TC: WARNING: Please check https://optee.readthedocs.io/en/latest/architecture/porting_guidelines.html
I/TC: Primary CPU initializing
E/TC:0 0 ekb_extraction_process:321 Tried all EKB_RKs but still can't extract the EKB image.
E/TC:0 0 jetson_user_key_pta_init:1039 jetson_user_key_pta_init: Failed (ffff000f).
E/TC:0 0 call_initcalls:43 Initcall __text_start + 0x000f92b0 failed
I/TC: Primary CPU switching to normal world boot
��
  [0003.288] I> Welcome to NVDisp-Init
[0003.289] I> NVDisp-Init version: t194-51f071e0
[0003.289] I> CPU-BL Params @ 0xca020000
[0003.289] I>  0) Base:0x00000000 Size:0x00000000
[0003.289] I>  1) Base:0xc8100000 Size:0x00100000
[0003.290] I>  2) Base:0xc9800000 Size:0x00200000
[0003.290] I>  3) Base:0xc8600000 Size:0x00200000
[0003.293] I>  4) Base:0xc8000000 Size:0x00100000
[0003.297] I>  5) Base:0xc7f00000 Size:0x00100000
[0003.302] I>  6) Base:0xc9400000 Size:0x00400000
[0003.306] I>  7) Base:0xc9000000 Size:0x00400000
[0003.310] I>  8) Base:0xc7e00000 Size:0x00100000
[0003.315] I>  9) Base:0xc7d00000 Size:0x00100000
[0003.319] I> 10) Base:0xca800000 Size:0x00800000
[0003.324] I> 11) Base:0x40000000 Size:0x00040000
[0003.328] I> 12) Base:0xc7c00000 Size:0x00100000
[0003.333] I> 13) Base:0x40046000 Size:0x00002000
[0003.337] I> 14) Base:0x40048000 Size:0x00002000
[0003.342] I> 15) Base:0xaf000000 Size:0x00004000
[0003.346] I> 16) Base:0x4004a000 Size:0x00002000
[0003.351] I> 17) Base:0xc7a00000 Size:0x00100000
[0003.355] I> 18) Base:0x4004c000 Size:0x00002000
[0003.360] I> 19) Base:0xc9a00000 Size:0x00600000
[0003.364] I> 20) Base:0x4004e000 Size:0x00002000
[0003.368] I> 21) Base:0xc7bc0000 Size:0x0000c000
[0003.373] I> 22) Base:0x00000000 Size:0x00000000
[0003.377] I> 23) Base:0xc7be0000 Size:0x00020000
[0003.382] I> 24) Base:0xcc000000 Size:0x02000000
[0003.386] I> 25) Base:0x40050000 Size:0x00002000
[0003.391] I> 26) Base:0x40040000 Size:0x00006000
[0003.395] I> 27) Base:0xc8c00000 Size:0x00400000
[0003.400] I> 28) Base:0xc8400000 Size:0x00200000
[0003.404] I> 29) Base:0xc8800000 Size:0x00400000
[0003.409] I> 30) Base:0xc7bd0000 Size:0x00010000
[0003.413] I> 31) Base:0x00000000 Size:0x00000000
[0003.418] I> 32) Base:0xf8000000 Size:0x08000000
[0003.422] I> 33) Base:0xce000000 Size:0x2a000000
[0003.427] I> 34) Base:0xcb000000 Size:0x01000000
[0003.431] I> 35) Base:0xae000000 Size:0x01000000
[0003.435] I> 36) Base:0xa0000000 Size:0x0e000000
[0003.440] I> 37) Base:0xca000000 Size:0x00800000
[0003.444] I> 38) Base:0x80000000 Size:0x20000000
[0003.449] I> 39) Base:0xb0000000 Size:0x08000000
[0003.453] I> 40) Base:0x00000000 Size:0x00000000
[0003.458] I> 41) Base:0x00000000 Size:0x00000000
[0003.462] I> 42) Base:0xc8200000 Size:0x00200000
[0003.467] I> 43) Base:0x00000000 Size:0x00000000
[0003.471] I> 44) Base:0x00000000 Size:0x00000000
[0003.476] I> 45) Base:0x00000000 Size:0x00000000
[0003.480] GIC-SPI Target CPU: 0
[0003.483] Interrupts Init done
[0003.486] calling constructors
[0003.489] initializing heap
[0003.491] I> Heap: [0xa0a60000 ... 0xadf00000]
[0003.496] initializing threads
[0003.499] initializing timers
[0003.501] creating bootstrap completion thread
[0003.506] top of bootstrap2()
[0003.508] CPU: MIDR: 0x4E0F0040, MPIDR: 0x80000000
[0003.513] initializing platform
[0003.516] E> DEVICE_PROD: Invalid value data = 0, size = 0.
[0003.522] W> device prod register failed
[0003.525] I> Bl_dtb @0xadf00000
[0003.528] I> gpio framework initialized
[0003.541] I> tegrabl_gpio_driver_register: register 'nvidia,tegra194-gpio' driver
[0003.549] I> tegrabl_gpio_driver_register: register 'nvidia,tegra194-gpio-aon' driver
[0003.555] I> fixed regulator driver initialized
[0003.574] I> register 'maxim' power off handle
[0003.578] I> virtual i2c enabled
[0003.578] I> registered 'maxim,max20024' pmic
[0003.579] I> tegrabl_gpio_driver_register: register 'max20024-gpio' driver
[0003.579] I> Boot-device: QSPI
[0003.579] I> Boot_device: QSPI_FLASH instance: 0
[0003.580] I> configure_qspi_clk: qparams clk_src = 5, clk_div = 0, clk_src_freq = 800000000, interface_freq = 50000000
[0003.588] I> configure_qspi_clk: FORCING INTERFACE_FREQ TO 133000000!!
[0003.595] I> QSPI source rate = 204000 Khz
[0003.598] I> Requested rate for QSPI clock = 34000 Khz
[0003.603] I> BPMP-set rate for QSPI clk = 34000 Khz
[0003.608] I> tx_clk_tap_delay : 0
[0003.611] I> rx_clk_tap_delay : 16
[0003.614] I> QSPI Flash Size = 32 MB
[0003.622] I> Qspi initialized successfully
[0003.622] I> qspi flash-0 params source = boot args
[0003.627] W> No board IDs available
[0003.629] E> Failed to get board id info!
[0003.635] I> sdmmc-3 params source = safe params
[0003.642] I> Found 41 partitions in QSPI_FLASH (instance 0)
[0003.644] W> Cannot find any partition table for 00000003
[0003.648] E> Failed to publish 00000003
[0003.657] I> Found 22 partitions in SDMMC_USER (instance 3)
[0003.680] I> regulator 'vdd-hdmi-5v0' already enabled
[0003.690] I> regulator 'vdd-hdmi-5v0' already enabled
[0003.690] I> hdmi cable connected
[0003.698] W> set volts not configured for 'vdd-1v0'
[0003.707] W> set volts not configured for 'vdd-1v8-hs'
[0003.707] I> retrieved tmds range from prod_list_hdmi_soc
[0003.709] E> cannot find any other nvdisp nodes
[0003.727] I> edid read success
[0003.739] I> edid read success
[0003.739] I> width = 640, height = 480, frequency = 25174825
[0003.740] I> width = 1024, height = 768, frequency = 65000000
[0003.740] I> width = 1920, height = 1200, frequency = 154000000
[0003.741] I> width = 1920, height = 1080, frequency = 148500000
[0003.741] I> width = 1920, height = 1080, frequency = 148351648
[0003.745] I> width = 1920, height = 1080, frequency = 148351648
[0003.751] I> width = 1280, height = 720, frequency = 74175824
[0003.757] I> width = 1280, height = 720, frequency = 74175824
[0003.762] I> width = 720, height = 480, frequency = 26973026
[0003.768] I> width = 720, height = 480, frequency = 26973026
[0003.773] I> width = 720, height = 576, frequency = 26973026
[0003.779] I> width = 640, height = 480, frequency = 25174825
[0003.784] I> Best mode Width = 1920, Height = 1080, freq = 148351648
[0003.796] I> hdmi_enable, starting HDMI initialisation
[0003.800] I> hdmi_enable, HDMI initialisation complete
[0003.809] initializing target
[0003.810] calling apps_init()
[0003.811] starting app kernel_boot_app
[0003.811] I> Kernel type = Normal

Jetson UEFI firmware (version 6.0-37391689 built on 2024-08-28T08:47:11+00:00)






























































��I/TC: Reserved shared memory is disabled
I/TC: Dynamic shared memory is enabled
I/TC: Normal World virtualization support is disabled
I/TC: Asynchronous notifications are disabled
E/TC:? 0 jetson_user_key_pta_uefi_vars_auth:920 UEFI variable auth key not set !
E/TC:? 0 stmm_handle_variable_authentication:910 Failed to get signed CMAC ffff0008

ASSERT [FvbNorFlashStandaloneMm] /dvs/git/dirty/git-master_linux/out/nvidia/optee.t194-uefi/StandaloneMmOptee_RELEASE/edk2-nvidia/Silicon/NVIDIA/Drivers/FvbNorFlashDxe/VarIntCheck.c(922): (()

Thanks !

#Regenerated eks_t194.img with a real auth_t194.key. 


openssl rand -hex 16 > auth_t194.key

python3 gen_ekb.py -chip t194 -kek2_key ../sd/kek2.txt -fv fv_ekb_t194   -
in_sym_key ../sym_t194.key -in_sym_key2 ../sym2_t194.key -in_auth_key auth_t194.key -out eks_t194.img

cp eks_t194.img $ROOT_DIR/Linux_for_Tegra/bootloader

Now the CMAC validation error goes away and I can boot into the system.

1 Like

Good to hear that, you need to re-generate and update eks image in r35.6.0 before generate capsule payload.

1 Like

Just one ques;

Why does the capsule update status shows still 0 after capsule update successfullly ?

Shouldn’t this be status=1 for successfully update according to the documentation and table mentioned below. ?

# Before update 

sudo nvbootctrl dump-slots-info
Current version: 35.4.1
Capsule update status: 0
Current bootloader slot: A
Active bootloader slot: A
num_slots: 2
slot: 0,             status: normal
slot: 1,             status: unbootable 


# After update 
sudo nvbootctrl dump-slots-info
Current version: 35.6.0
Capsule update status: 0
Current bootloader slot: B
Active bootloader slot: B
num_slots: 2
slot: 0,             status: normal

slot: 1,             status: normal


0 - No Capsule update
1 - Capsule update successfully
2 - Capsule install successfully but boot new firmware failed
3 - Capsule install failed 

Info- I booted from USB (active boot-media), copied the TEGRA_BL.Cap from usb rootfs to esp partition of NVME, and rebooted.

On reboot, the capsule update status shows 0.

Probably, it is because the active-boot-media was USB on the previous run and NVME after reboot.

However, it doesn’t mattter to me as the update is working, but the status might be expecting the active-boot-media to be same before and after reboot. I will test it out,

sudo nvbootctrl dump-slots-info
Current version: 35.6.0
Capsule update status: 1
Current bootloader slot: A
Active bootloader slot: A
num_slots: 2
slot: 0,             status: normal
slot: 1,             status: normal

Ok; I tried with active-boot media as NVME and placed TEGRA_BL.Cap on esp partition before reboot.

It gives now the correct update status as 1.

I think in my use case where I apply the bootloader update and rootfs update while booting from USB, this status thinks as no capsule update .i.e 0.

But it is OK for me as my bootloader and rootfs indeed gets updated.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.