Boot Failure When Op-Tee is Disabled

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
,
1

Orin Nano devkit fails to boot with the follow stack trace when optee support is disabled in atf. The same unit boots as expected when using the optee tos image from r36.4.0.

ERROR:   Exception reason=0 syndrome=0xbe000011
ERROR:   **************************************
ERROR:   RAS Uncorrectable Error in IOB, base=0xe010000:
ERROR:          Status = 0xec000612
ERROR:   SERR = Error response from slave: 0x12
ERROR:          IERR = CBB Interface Error: 0x6
ERROR:          Overflow (there may be more errors) - Uncorrectable
ERROR:          MISC0 = 0xc4460040
ERROR:          MISC1 = 0x1cc860000000000
ERROR:          MISC2 = 0x0
ERROR:          MISC3 = 0x0
ERROR:          ADDR = 0x8000000003270000
ERROR:   **************************************
ERROR:   sdei_dispatch_event returned -1
ERROR:   **************************************
ERROR:   RAS Uncorrectable Error in ACI, base=0xe01a000:
ERROR:          Status = 0xe8000904
ERROR:   SERR = Assertion failure: 0x4
ERROR:          IERR = FillWrite Error: 0x9
ERROR:          Overflow (there may be more errors) - Uncorrectable
ERROR:          ADDR = 0x8000000003270000
ERROR:   **************************************
ERROR:   sdei_dispatch_event returned -1
ERROR:   Powering off core

Steps to reproduce.

  1. Fetch atf source from l4t r36.4.0
  2. Modify nvbuild.sh to remove the SPD build flag from line 90
    nv-tegra.nvidia Code Review - tegra/optee-src/atf.git/blob - nvbuild.sh
  3. Build with a command similar to:
    NV_TARGET_BOARD=generic CROSS_COMPILE_AARCH64_PATH=../aarch64--glibc--stable-2022.08-1 ./nvbuild.sh
  4. Build tos image with a command similar to
    python gen_tos_part_img.py --monitor bl31.bin tos-mon-only_t234.img
  5. Copy tos-mon-only_t234.img to l4t r36.4.0 bootloader folder and change tos_t234.img symlink to point to it.
  6. Flash devkit as normal, like
    ./flash.sh jetson-orin-nano-devkit mmcblk0p1

This is the same failure that was observed in the following thread, where the user built atf without optee support.

The address listed is 0x3270000, which per the kernel dt is the spi channel the boot eeprom is on. Is there something in the closed source boot stack that routes eeprom access through optee? If so, is this something configurable in the bct configs? I’m trying to support the no-spd case and potentially develop support for other spd’s like trusty.

And another related question. Is there any plans to upstream t234 support to the main atf project? T210, t186, and t194 are all supported very well on upstream. It’s disappointing and a disruption of my workflow to have to use a vendor specific fork for t234. And worse, one that has no commit history, which would allow for easier porting to other versions.

3

hello Steel01,

let’s check whether you’ve compile atf and op-tee correctly.
here’re my steps for your reference,

Prerequisites.
Extract both atf_src.tbz2 and nvidia-jetson-optee-source.tbz2 source package.
$ apt install python3-pyelftools
$ sudo apt remove python3-cryptography && pip3 install cryptography (it’s for the old python cryptography issue)

Setting the CROSS_COMPILE Environment Variable
$ export CROSS_COMPILE_AARCH64_PATH=/home/jerry/L4T/l4t-gcc/aarch64--glibc--stable-2022.08-1
$ export CROSS_COMPILE_AARCH64=/home/jerry/L4T/l4t-gcc/aarch64--glibc--stable-2022.08-1/bin/aarch64-buildroot-linux-gnu-
$ export CROSS_COMPILE=/home/jerry/L4T/l4t-gcc/aarch64--glibc--stable-2022.08-1/bin/aarch64-buildroot-linux-gnu-
$ export UEFI_STMM_PATH=$OUT/JP_6/Linux_for_Tegra/bootloader/standalonemm_optee_t234.bin
$ export NV_TARGET_BOARD=t234

Build commands
$ ./nv_public_src_build.sh -p t234
$ ./nv_public_src_build_tos.sh -p t234 -u ~/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/standalonemm_optee_t234.bin -s ~/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/nv_tegra/tos-scripts/gen_tos_part_img.py

4 
let’s check whether you’ve compile atf and op-tee correctly.

Please read my post. The point is to NOT include optee. To build and boot atf without a secure dispatcher. If optee is included, then it does boot as expected.

5

hello Steel01,

sorry, we don’t support this use-case to disable op-tee.

6

For those finding this thread via search, I did find a solution. The exception is being generated by edk2.

Optee support in edk2 can be removed by disabling the SECURITY_MM_OPTEE config option. VARIABLE_EMULATED then needs enabled to replace some of the functionality provided by optee. With this config, t234 can be booted without optee. Or with an alternate secure os.

8

I have found a further solution to this issue.

By modifying tegra234-mb2-bct-scr-p3767-0000.dts (or tegra234-mb2-bct-scr-p3701-0000.dts for the AGX Orin series) to remove the CBB_CENTRAL_CBB_FIREWALL_QSPI0_BLF sections (reg@5138 and reg@5139 as of l4t r36.4.1), non-secure edk2 will be able to access the qspi in the absence of optee, allowing an unmodified edk2 image from l4t r36.4.1 to work without optee.

I’ve been trying to figure out how the firewall config works for some time now, since I have been unable to find any documentation on it. Based on this, given that the base firewall config has a value of 0x0010000b for the qspi sections and the files referenced above override that to 0x00100009, it can be inferred that bit 2 controls ns uefi access.

This solution works better for my purposes because it allows for persistent efi variable storage. The previous solution forced emulated variable storage, which does not persist across warm reboots, much less cold boots. Which breaks things like requesting a boot slot switch.