Building new Trusted Applications (TA) using Trusty on Jetson Xavier NX

Hello!

I have a Jetson Xavier NX board, and I followed the quick-start guide given in the documentation to install whatever I was supposed to. Now I have a Linux terminal, which I believe is Ubuntu 18.04. My output of “uname -a” is

Linux xavier 4.9.253-tegra #1 SMP PREEMPT Mon Jul 26 12:19:28 PDT 2021 aarch64 aarch64 aarch64 GNU/Linux

Now, I want to build a Trusted Application on this using Trusty. (I am running all this on the Xavier board)

I downloaded L4T Driver Package (BSP) Sources from here - https://developer.nvidia.com/embedded/l4t/r32_release_v6.1/sources/t186/public_sources.tbz2 - and unzipped it.

Inside that, I found trusty_src.tbz2 and unzipped it to find a bunch of folders. I went to nvidia-sample/hwkey-agent to find a bunch of files. I then went to CA_Sample in the same folder and ran “make” to get a binary in /out. Then as the README mentioned, I tried to generate a random number and I was successfully able to.

Now, my question is this - how do I make a modification to this hwkey-agent app? As I understand, the CA_Sample folder contains the files for the client application and rest of the files belong to the TA (i.e in the case of hwkey-agent, rng_srv.c, ipc.c and so on.)

I made a modification to the rng_srv.c where I added a line printing “Hello World!”. How do I compile and run this so that this “Hello World” also gets printed?

I found a text file by the name “atf_and_trusty_README.txt”, but that, I think, is to build on a different machine such a laptop running Linux, so I am not sure how that is helpful when we are building apps on the Xavier board itself.

I also saw this link - Welcome — Jetson Linux<br/>Developer Guide 34.1 documentation - and here on Step 6, it says that I need to. “Rebuild Trusty” but how do I do this? I couldn’t understand that part.

Any help would be really appreciated.

Tagging @JerryChang

hello gokulnath136,

I’ve never verify building TAs on the Jetson Xavier,
please refer to the instruction, atf_and_trusty_README.txt, it show the steps to build Trusty sources on the x86 host machine,

you may have a try,
you’ll need to download cross compile toolchains, (both 32-bit and 64-bit version) to build the sources.
thanks

Hi, thank you for the reply.

Here’s what I understand:

  1. Install the required toolchains on a host machine (I am currently using Ubuntu 20.04). Here’s what I have installed:

gokul@cssl:~$ echo $CROSS_COMPILE
/home/gokul/Trusty/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
gokul@cssl:~$ echo $CROSS_COMPILE_AARCH64
/home/gokul/Trusty/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
gokul@cssl:~$ echo $CROSS_COMPILE_ARM
/home/gokul/Trusty/gcc-linaro-7.3.1-2018.05-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-

  1. I followed the instructions on this page i.e wrote a manifest.c, added source files to the app directory, created a rules.mk etc. I named my app gokul and placed it at /home/gokul/Trusty/Linux_for_Tegra/source/public/atf_and_trusty/trusty/trusty/app/sample/gokul. Post that, I added the line TRUSTY_ALL_USER_TASKS += sample/gokul to trusty/device/nvidia/t186/project/t186-l4t.mkas mentioned. But, I also added the same line to trusty/tegra/public/project/t186/t186-l4t.mk as

ifeq ($(filter l4t-partner-ote, $(TRUSTY_VARIANT)),)
TRUSTY_ALL_USER_TASKS +=
nvidia-sample/hwkey-agent
nvidia-sample/luks-srv
sample/gokul

as it seemed to me that all the apps were present there as well. (Should I remove it from this file and keep it only where the documentation asks it to be?)

  1. I, then, followed the instructions on atf_and_trusty.txt and was able to build both the lk.bin and bl31.bin file, placed it on in the bootloaderfolder after running the TOS_image Python script and then flashed it using sudo ./flash.sh jetson-xavier mmcblk0p1

The board booted up normally, and I then transferred these files that I had made via SSH to the Xavier board and tried to run it, but it threw me an error saying that it was unable to find the library. (I checked, and all the libraries were present).

The only step I added extra was the line that I appended in trusty/tegra/public/project/t186/t186-l4t.mk. Could that have been the issue why my app did not run? Is there something more I need to install in my toolchain as well?

Once again, thank you for taking the time to help me.

hello gokulnath136,

instead of full flash, it’ll be quicker to update secure-os partition to apply the changes.
since you’re having a Jetson Xavier NX, please assign the correct board name to the pipeline.
for example, $ sudo ./flash.sh -k secure-os jetson-xavier-nx-devkit mmcblk0p1

please try again, and please share the test results.
thanks

Hi, I ran this and got this error:

L4T BSP Information:
R32 , REVISION: 6.1
Target Board Information:
Name: jetson-xavier-nx-devkit, Board Family: t186ref, SoC: Tegra 194,
OpMode: production, Boot Authentication: NS,
Disk encryption: disabled ,

copying soft_fuses(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-soft-fuses-l4t.cfg)… done.
./tegraflash.py --chip 0x19 --applet “/home/gokul/Trusty/Linux_for_Tegra/bootloader/mb1_t194_prod.bin” --skipuid --soft_fuses tegra194-mb1-soft-fuses-l4t.cfg --bins “mb2_applet nvtboot_applet_t194.bin” --cmd “dump eeprom boardinfo cvm.bin;reboot recovery”
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands

[ 0.0029 ] Generating RCM messages
[ 0.0050 ] tegrahost_v2 --chip 0x19 0 --magicid MB1B --appendsigheader /home/gokul/Trusty/Linux_for_Tegra/bootloader/mb1_t194_prod.bin zerosbk
[ 0.0057 ] Header already present for /home/gokul/Trusty/Linux_for_Tegra/bootloader/mb1_t194_prod.bin
[ 0.0083 ]
[ 0.0103 ] tegrasign_v3.py --getmode mode.txt --key None
[ 0.0104 ] Assuming zero filled SBK key
[ 0.0125 ] tegrasign_v3.py --file /home/gokul/Trusty/Linux_for_Tegra/bootloader/mb1_t194_prod_sigheader.bin --key None --length 1136 --offset 2960 --pubkeyhash pub_key.key
[ 0.0126 ] Assuming zero filled SBK key
[ 0.0168 ] tegrahost_v2 --chip 0x19 0 --updatesigheader /home/gokul/Trusty/Linux_for_Tegra/bootloader/mb1_t194_prod_sigheader.bin /home/gokul/Trusty/Linux_for_Tegra/bootloader/mb1_t194_prod_sigheader.hash zerosbk
[ 0.0199 ]
[ 0.0222 ] tegrabct_v2 --chip 0x19 0 --sfuse tegra194-mb1-soft-fuses-l4t.cfg.pdf sfuse.bin
[ 0.0231 ]
[ 0.0251 ] tegrarcm_v2 --listrcm rcm_list.xml --chip 0x19 0 --sfuses sfuse.bin --download rcm /home/gokul/Trusty/Linux_for_Tegra/bootloader/mb1_t194_prod_sigheader.bin 0 0
[ 0.0257 ] RCM 0 is saved as rcm_0.rcm
[ 0.0282 ] RCM 1 is saved as rcm_1.rcm
[ 0.0283 ] RCM 2 is saved as rcm_2.rcm
[ 0.0283 ] List of rcm files are saved in rcm_list.xml
[ 0.0284 ]
[ 0.0284 ] Signing RCM messages
[ 0.0306 ] tegrasign_v3.py --getmontgomeryvalues montgomery.bin --key None --list rcm_list.xml --pubkeyhash pub_key.key
[ 0.0307 ] Assuming zero filled SBK key
[ 0.0372 ] Copying signature to RCM mesages
[ 0.0391 ] tegrarcm_v2 --chip 0x19 0 --updatesig rcm_list_signed.xml
[ 0.0402 ]
[ 0.0403 ] Boot Rom communication
[ 0.0421 ] tegrarcm_v2 --chip 0x19 0 --rcm rcm_list_signed.xml --skipuid
[ 0.0428 ] RCM version 0X190001
[ 0.0438 ] Boot Rom communication completed
[ 1.0606 ]
[ 2.0665 ] tegrarcm_v2 --isapplet
[ 2.0698 ] Applet version 01.00.0000
[ 2.0885 ]
[ 2.0935 ] tegrarcm_v2 --ismb2
[ 2.1284 ]
[ 2.1336 ] tegrahost_v2 --chip 0x19 --align nvtboot_applet_t194.bin
[ 2.1371 ]
[ 2.1418 ] tegrahost_v2 --chip 0x19 0 --magicid PLDT --appendsigheader nvtboot_applet_t194.bin zerosbk
[ 2.1450 ] adding BCH for nvtboot_applet_t194.bin
[ 2.1570 ]
[ 2.1645 ] tegrasign_v3.py --key None --list nvtboot_applet_t194_sigheader.bin_list.xml --pubkeyhash pub_key.key
[ 2.1648 ] Assuming zero filled SBK key
[ 2.1794 ] tegrahost_v2 --chip 0x19 0 --updatesigheader nvtboot_applet_t194_sigheader.bin.encrypt nvtboot_applet_t194_sigheader.bin.hash zerosbk
[ 2.1897 ]
[ 2.1947 ] tegrarcm_v2 --download mb2 nvtboot_applet_t194_sigheader.bin.encrypt
[ 2.1977 ] Applet version 01.00.0000
[ 2.2170 ] Sending mb2
[ 2.2171 ] […] 100%
[ 2.2319 ]
[ 2.2343 ] tegrarcm_v2 --boot recovery
[ 2.2353 ] Applet version 01.00.0000
[ 2.2557 ]
[ 3.2614 ] tegrarcm_v2 --isapplet
[ 3.2851 ]
[ 3.2904 ] tegrarcm_v2 --ismb2
[ 3.2939 ] MB2 Applet version 01.00.0000
[ 3.3127 ]
[ 3.3178 ] tegrarcm_v2 --ismb2
[ 3.3213 ] MB2 Applet version 01.00.0000
[ 3.3571 ]
[ 3.3616 ] Retrieving board information
[ 3.3663 ] tegrarcm_v2 --oem platformdetails chip chip_info.bin
[ 3.3696 ] MB2 Applet version 01.00.0000
[ 3.3954 ] Saved platform info in chip_info.bin
[ 3.3999 ] Chip minor revision: 2
[ 3.4003 ] Bootrom revision: 0xf
[ 3.4009 ] Ram code: 0x0
[ 3.4009 ] Chip sku: 0xde
[ 3.4010 ] Chip Sample: non es
[ 3.4010 ]
[ 3.4018 ] Retrieving EEPROM data
[ 3.4020 ] tegrarcm_v2 --oem platformdetails eeprom cvm /home/gokul/Trusty/Linux_for_Tegra/bootloader/cvm.bin
[ 3.4047 ] MB2 Applet version 01.00.0000
[ 3.4275 ] Saved platform info in /home/gokul/Trusty/Linux_for_Tegra/bootloader/cvm.bin
[ 3.4622 ]
[ 3.4625 ] Rebooting to recovery mode
[ 3.4682 ] tegrarcm_v2 --ismb2
[ 3.4712 ] MB2 Applet version 01.00.0000
[ 3.4890 ]
[ 3.4891 ] Rebooting to recovery mode
[ 3.4935 ] tegrarcm_v2 --reboot recovery
[ 3.4961 ] MB2 Applet version 01.00.0000
[ 3.5400 ]
Board ID(3668) version(300) sku(0000) revision(B.0)
copying bctfile(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-bct-memcfg-p3668-0001-a00.cfg)… done.
copying bctfile1(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-memcfg-sw-override.cfg)… done.
copying device_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra19x-mb1-bct-device-qspi-p3668.cfg)… done.
copying misc_cold_boot_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-bct-misc-l4t.cfg)… done.
copying misc_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-bct-misc-flash.cfg)… done.
copying pinmux_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra19x-mb1-pinmux-p3668-a01.cfg)… done.
copying gpioint_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-bct-gpioint-p3668-0001-a00.cfg)… done.
copying pmic_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-bct-pmic-p3668-0001-a00.cfg)… done.
copying pmc_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra19x-mb1-padvoltage-p3668-a01.cfg)… done.
copying prod_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra19x-mb1-prod-p3668-0001-a00.cfg)… done.
copying scr_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-bct-scr-cbb-mini-p3668.cfg)… done.
copying scr_cold_boot_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-bct-scr-cbb-mini-p3668.cfg)… done.
copying bootrom_config(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-bct-reset-p3668-0001-a00.cfg)… done.
copying dev_params(/home/gokul/Trusty/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-br-bct-qspi.cfg)… done.
Existing bootloader(/home/gokul/Trusty/Linux_for_Tegra/bootloader/nvtboot_cpu_t194.bin) reused.
copying initrd(/home/gokul/Trusty/Linux_for_Tegra/bootloader/l4t_initrd.img)… done.
Making Boot image… done.
cp: cannot stat ‘/home/gokul/Trusty/Linux_for_Tegra/bootloader/tegra194-mb1-bct-ratchet-p3668.cfg’: No such file or directory
/home/gokul/Trusty/Linux_for_Tegra/bootloader/tegraflash.py --chip 0x19 --key --minratchet_config tegra194-mb1-bct-ratchet-p3668.cfg --cmd sign boot.img kernel
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands

[ 0.0003 ] Generating signature
[ 0.0025 ] tegrasign_v3.py --getmode mode.txt --key
[ 0.0025 ] Assuming zero filled SBK key : not reading
[ 0.0025 ] Generating ratchet blob
[ 0.0045 ] tegrabct_v2 --chip 0x19 0 --ratchet_blob ratchet_blob.bin --minratchet tegra194-mb1-bct-ratchet-p3668.cfg
[ 0.0053 ] Empty File tegra194-mb1-bct-ratchet-p3668.cfg
[ 0.0054 ] Failed to Parse file tegra194-mb1-bct-ratchet-p3668.cfg: 0x0000000a
[ 0.0054 ] File tegra194-mb1-bct-ratchet-p3668.cfg open failed
[ 0.0054 ]
Error: Return value 19
Command tegrabct_v2 --chip 0x19 0 --ratchet_blob ratchet_blob.bin --minratchet tegra194-mb1-bct-ratchet-p3668.cfg
l4t_sign_image.sh: Error: Unable to find the signed file generated by tegraflash.py
failed.

I tried looking for an answer in the forums, but couldn’t find anything.

Also, if I, say, add a printf in one of the TA files of the hwkey-agent app (for example in rng_srv.c), then would I be able to see the output on my screen? Or is the output from secure world not printed on the normal terminal? If then, where can I see it?

I was able to fix this. I was missing the file tegra194-mb1-bct-ratchet-p3668.cfg. I saw that a file was missing and the command I ran was sudo ./flash.sh --no-flash jetson-xavier-nx-devkit mmcblk0p1 to generate the file again as mentioned again, and ran this command - sudo ./flash.sh -k secure-os jetson-xavier-nx-devkit mmcblk0p1 again and was able to boot the secure-os successfully.