OP-TEE setup questions for the atf_and_optee_README.txt

Hi, I am currently trying to setup OP-TEE on the Jetson AGX Orin Developer Kit.
I am working on the latest files of Jetpack 6.0 and Jetson Linux 36.3.
When going through the atf_and_optee_README.txt file, I executed nv_public_src_build.sh as said in the txt file and it made the following output:

...
================================================================================
Kernel Image: /home/andi/jetson-public-srcs/Linux_for_Tegra/source/src_out/kernel_src_build/kernel_out/kernel/kernel-jammy-src/arch/arm64/boot/Image
Kernel sources compiled successfully.
================================================================================
make: Leaving directory '/home/andi/jetson-public-srcs/Linux_for_Tegra/source/src_out/kernel_src_build/kernel_out/kernel'
Kernel sources compiled successfully.
Directory "nvethernetrm" is not found, exiting..
andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source$

My first question is:
Should there be something else happening besides compiling the kernel sources, because it continues with searching for the directory “nvethernetrm”?

My second question:
If nothing is expected afterwards, I continue with the next step. For the python script gen_tos_part_img.py, where can I find the bl31.bin file required to generate the tos.img file? The usage of the script is shown as follows:

./gen_tos_part_img.py \
       --monitor ./atf_build/arm-trusted-firmware/build/tegra/t<platform>/release/bl31.bin \
       --os ./optee/build/t<platform>/core/tee-raw.bin \
       --dtb ./optee/tegra<platform>-optee.dtb \
       --tostype optee \
       ./tos.img

Thank you for your help!

EDIT: please refer to below comment #5 for reference steps.


it’s created by arm-trusted-firmware.
as you can see, please also setup Setting the CROSS_COMPILE Environment Variable for cross-complie.

hello andib,

you should also extract both atf_src.tbz2 and nvidia-jetson-optee-source.tbz2 source package,
in practical, these two should be located together, for instance, I’ll create atf_and_optee/ folder, and extract both of them into the same folder.

please note that you’ve to setup the CROSS_COMPILE Environment Variable
$ export CROSS_COMPILE_AARCH64_PATH=/home/jerry/L4T/l4t-gcc/aarch64--glibc--stable-2022.08-1
$ export CROSS_COMPILE_AARCH64=/home/jerry/L4T/l4t-gcc/aarch64--glibc--stable-2022.08-1/bin/aarch64-buildroot-linux-gnu-
$ export CROSS_COMPILE=/home/jerry/L4T/l4t-gcc/aarch64--glibc--stable-2022.08-1/bin/aarch64-buildroot-linux-gnu-
$ export UEFI_STMM_PATH=$OUT/JP_6/Linux_for_Tegra/bootloader/standalonemm_optee_t234.bin
$ export NV_TARGET_BOARD=t234

you’ve running nv_public_src_build.sh and then nv_public_src_build_tos.sh .
for building all source tarballs, including the atf_src.tbz2 , you should run the nv_public_src_build.sh script.

BTW,
if you’re seeing an error when running nv_public_src_build_tos.sh.
that’s due to the cryptography python package in your build system is too old.
re-cap as below…

".../optee/build/t234/export-ta_arm64/scripts/sign_encrypt.py", line 562, in parse
    self.subkey_key = rsa.RSAPublicNumbers(e, n).public_key()
TypeError: public_key() missing 1 required positional argument: 'backend'

you should running the following commands to resolve the build failure.
$ sudo apt remove python3-cryptography
$ pip3 install cryptography

hence,
I’m able to build the r36.3 ATF and OP-TEE sources.
for example,
$ ./nv_public_src_build_tos.sh -p t234 -u $OUT/JP-6/Linux_for_Tegra/bootloader/standalonemm_optee_t234.bin -s $OUT/JP-6/Linux_for_Tegra/nv_tegra/tos-scripts/gen_tos_part_img.py

here’s the output directory src_out has created.
for example,

/source/src_out$ tree -L 1
.
├── atf_src_build
├── img.bin
├── kernel_src_build
├── optee_src_build
└── tos-t234.img

Hi JerryChang, thanks for your reply.
I have set the environment variables:

andi@andi-usb3:~$ export CROSS_COMPILE_AARCH64_PATH=/home/andi/toolchain/aarch64--glibc--stable-2022.08-1
andi@andi-usb3:~$ export CROSS_COMPILE_AARCH64=/home/andi/t
test.txt   toolchain/ 
andi@andi-usb3:~$ export CROSS_COMPILE_AARCH64=/home/andi/toolchain/aarch64--glibc--stable-2022.08-1/bin/aarch64-buildroot-linux-gnu-
andi@andi-usb3:~$ export CROSS_COMPILE=/home/andi/t
test.txt   toolchain/ 
andi@andi-usb3:~$ export CROSS_COMPILE=/home/andi/toolchain/aarch64--glibc--stable-2022.08-1/bin/aarch64-buildroot-linux-gnu-
andi@andi-usb3:~$ export UEFI_STMM_PATH=jetson-linux/Linux_for_Tegra/bootloader/standalonemm_optee_t234.bin

and also removed python3-cryptography and did pip3 install cryptography.

When executing the script to build the tos, I get the following:

andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source$ ./nv_public_src_build_tos.sh -p t234 -u ~/jetson-linux/Linux_for_Tegra/bootloader/standalonemm_optee_t234.bin -s ~/jetson-linux/Linux_for_Tegra/nv_tegra/tos-scripts/gen_tos_part_img.py 
Can't find the ATF image: /home/andi/jetson-public-srcs/Linux_for_Tegra/source/src_out/atf_src_build/arm-trusted-firmware/-t234/tegra/t234/release/bl31.bin
Did you run nv_public_src_build.sh?

When I executed nv_public_src_build.sh, these are the last lines of the execution:

  LD [M]  sound/soc/rockchip/snd-soc-rockchip-rt5645.ko
  LD [M]  sound/soc/rockchip/snd-soc-rockchip-spdif.ko
  LD [M]  sound/soc/sh/rcar/snd-soc-rcar.ko
  LD [M]  sound/soc/sunxi/sun4i-i2s.ko
  LD [M]  sound/soc/sunxi/sun4i-spdif.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra-audio-graph-card.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra-pcm.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra186-asrc.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra186-dspk.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-admaif.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-adx.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-ahub.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-amx.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-dmic.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-i2s.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-mixer.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-mvc.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-ope.ko
  LD [M]  sound/soc/tegra/snd-soc-tegra210-sfc.ko
================================================================================
Kernel Image: /home/andi/jetson-public-srcs/Linux_for_Tegra/source/src_out/kernel_src_build/kernel_out/kernel/kernel-jammy-src/arch/arm64/boot/Image
Kernel sources compiled successfully.
================================================================================
make: Leaving directory '/home/andi/jetson-public-srcs/Linux_for_Tegra/source/src_out/kernel_src_build/kernel_out/kernel'
Kernel sources compiled successfully.
Directory "nvethernetrm" is not found, exiting..

For me, it just creates this output directory:

andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source/src_out$ tree -L 1
.
└── kernel_src_build

1 directory, 0 files
andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source/src_out$ 

I think the problem is in the nv_public_src_build.sh, because as you can see in the output of the script it builds the kernel sources but exits afterwards:

Directory "nvethernetrm" is not found, exiting..

hello andib,

I had this failure reported as well, but, it’s not the root cause of your TOS building failure.

it looks an incorrect path has used, or a NULL property of NV_TARGET_BOARD.
please try build again with adding below environment variable, $ export NV_TARGET_BOARD=t234
and… please gather the complete error logs if you still seeing failures.

Hi JerryChang,
unfortunately the error persists.
I added a txt file with everything I just did.
session.txt (492 KB)

As you can see, in the src_out contrary to your output, only the kernel_src_build folder appears. The rest is not created.
The arm_trusted_firmware folder that you can see in my txt file was created when I manually unpacked the atf_src.tbz2.
Is there something I am missing or need to download?

hello andib,

you should execute nv_public_src_build.sh and then nv_public_src_build_tos.sh.
and… it looks some misunderstanding, I meant we may ignore this message… Directory "nvethernetrm" is not found, exiting..

besides, regarding to below error reports.

Can't find the ATF image: /home/andi/jetson-public-srcs/Linux_for_Tegra/source/src_out/atf_src_build/arm-trusted-firmware/t234-t234/tegra/t234/release/bl31.bin

please try running find commands,
for instance, here’re the path of bl31 binary files located.
$ find src_out/ -name bl31.bin
src_out/atf_src_build/arm-trusted-firmware/t234-t194/tegra/t194/release/bl31.bin
src_out/atf_src_build/arm-trusted-firmware/t234-t234/tegra/t234/release/bl31.bin

Afterwards when I execute the build_tos script, I still get that error:

andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source$ ./nv_public_src_build_tos.sh -p t234 -u ~/jetson-linux/Linux_for_Tegra/bootloader/standalonemm_optee_t234.bin -s ~/jetson-linux/Linux_for_Tegra/nv_tegra/tos-scripts/gen_tos_part_img.py
Can't find the ATF image: /home/andi/jetson-public-srcs/Linux_for_Tegra/source/src_out/atf_src_build/arm-trusted-firmware/t234-t234/tegra/t234/release/bl31.bin
Did you run nv_public_src_build.sh?
andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source$

and also, the bl31.bin file can’t be found:

andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source$ find src_out/ -name bl31.bin
andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source$ 

the src_out folder is still only holding the kernel_src_build directory, the atf_build directory is still missing:

andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source/src_out$ tree -L 1
.
└── kernel_src_build

1 directory, 0 files
andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source/src_out$ 

since I cannot reproduce this locally, please double confirm you’ve downloaded the [Driver Package (BSP) Sources] package from jetson-linux-r363.
besides, may I know what’s your ubuntu version? I’m tested locally with 20.04 without failures.
$ lsb_release -a

No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.5 LTS
Release:	20.04

I downloaded the package again and get the same error. I am using Ubuntu 22.04:

andi@andi-usb3:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.4 LTS
Release:	22.04
Codename:	jammy

Do you know where exactly the atf_src_build folder comes from?
Because for me it is not appearing and it seems like it is needed for the nv_public_src_build_tos.sh script.
Because of that output Can't find the ATF image: /home/andi/Downloads/Linux_for_Tegra/source/src_out/atf_src_build/arm-trusted-firmware/t234-t234/tegra/t234/release/bl31.bin Did you run nv_public_src_build.sh?

I ran nv_public_src_build.sh again but still there is only this one directory after executing nv_public_src_build.sh:

andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/src_out$ tree -L 1
.
└── kernel_src_build

1 directory, 0 files
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/src_out$ 

hello andib,

yes, I’ve running nv_public_src_build.sh and then nv_public_src_build_tos.sh.
actually, for building all source tarballs, including the atf_src.tbz2, you should run the nv_public_src_build.sh script.

I think I just found a workaround:
I did the following:

andi@andi-usb3:~/Downloads/Linux_for_Tegra/source$ sudo -E ./nvbuild.sh 
Building atf sources ..
Building atf sources for t194
make: *** /home/andi/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/: No such file or directory.  Stop.
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source$ tar -xvjf atf_src.tbz2

After that, a folder arm_trusted_firmware appeared and I executed

andi@andi-usb3:~/Downloads/Linux_for_Tegra/source$ sudo -E ./nvbuild.sh

Now, I finally find the bl31.bin file:

andi@andi-usb3:~/Downloads/Linux_for_Tegra/source$ cd arm-trusted-firmware/
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware$ ls
bl1   bl32            docs     lib          make_helpers       readme.rst  tools
bl2   changelog.yaml  drivers  license.rst  package.json       services
bl2u  common          fdts     licenses     package-lock.json  t234-t194
bl31  dco.txt         include  Makefile     plat               t234-t234
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware$ cd t234-t2
bash: cd: t234-t2: No such file or directory
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware$ cd t234-t234/
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234$ ls
tegra
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234$ cd tegra/
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234/tegra$ ls
t234
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234/tegra$ cd t234/
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234/tegra/t234$ ls
release
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234/tegra/t234$ cd release/
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234/tegra/t234/release$ ls
bl31  bl31.bin  lib  libc  libwrapper  romlib
andi@andi-usb3:~/Downloads/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234/tegra/t234/release$ 

I will try to build now the tos image again using this command:

./gen_tos_part_img.py \
       --monitor ./atf_build/arm-trusted-firmware/build/tegra/t<platform>/release/bl31.bin \
       --os ./optee/build/t<platform>/core/tee-raw.bin \
       --dtb ./optee/tegra<platform>-optee.dtb \
       --tostype optee \
       ./tos.img

And try to use the bl31.bin file that I just created.
I will then come back with updates.

hello andib,

BTW, I did not use sudo for running those build scripts.

Hi,
it finally created tos.bin:

andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source/jetson-optee-srcs$ python3 gen_tos_part_img.py --monitor ~/jetson-public-srcs/Linux_for_Tegra/source/arm-trusted-firmware/t234-t234/tegra/t234/release/bl31.bin --os ./optee/build/t234/core/tee-raw.bin --dtb ./optee/tegra234-optee.dtb --tostype optee ./tos.img
Generating Trusted OS Partition Image File
Generate TOS Image File for boot-wrapper.
andi@andi-usb3:~/jetson-public-srcs/Linux_for_Tegra/source/jetson-optee-srcs$ ls
 gen_tos_part_img.py   img.bin   nvcommon_build.sh   optee   optee_src_build.sh  ' --os'   tos.img

The img.bin was created before, I think there I made a mistake in the command. But I think now the tos.img will work and I will now flash the AGX Orin devkit with it.

hello andib,

I think another important step is extract atf_src.tbz2 and nvidia-jetson-optee-source.tbz2 source package,
in practical, these two should be located together, for instance, I’ll create atf_and_optee/ folder, and extract both of them into the same folder.

let me add that to my previous comment #5.
hope it helps others for building ATF and OP-TEE sources.

1 Like

Hi JerryChang,

thank you so much for your help!
Afterwards, I executed apply_binaries.sh and /tools/l4t_flash_prerequisites.sh and then I flashed using sudo ./flash.sh jetson-agx-orin-devkit internal
After that, it also booted contrary to my previous flashing attempts. When it booted successfully, I set it up and ran sudo xtest, which resulted in 31066 subtests of which 0 failed, 133 test cases of which 0 failed and 0 tests were skipped.
Seems like it works fine :)
Thank you so much for your efforts!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.