Jetson won't boot after update TOS image

hello jdiegodelgado,

it looks it’s stuck here for waiting system configuration.

[   32.806104] Please complete system configuration setup on the serial port provided by Jetson's USB device mode connection. e.g. /dev/ttyUSBx where x can 0, 1, 2 etc.

you may refer to Skipping oem-config.
could you please try running below $OUT/Linux_for_Tegra/tools/l4t_create_default_user.sh to creates a default user accounts for testing.

JerryChang,

I tested skipping the oem-config by running the /Linux_for_Tegra/tools/l4t_create_default_user.sh. It seems to work fine (please see the logs attached below).

But some logs called my attention, for instance this one:

E/TC:0 0 jetson_user_key_pta_init:700 jetson_user_key_pta_init: Failed (f0100006).

Why does jetson_user_key_pta_init failed? How do I know that the new Trusted OS is working properly?

minicom.cap (83.6 KB)

Also, I noticed that inside the Linux_for_Tegra/bootloader/ path, originally you find two symlink files:

  • tos_t194.img
  • tos_t234.img

They look like this: tos_t234.img -> tos-optee_t234.img.
When I created my new tos.img, should I renamed it for tos_t194.img, move it to the Linux_for_Tegra/bootloader/ path, and then made it a symlink that points to tos-optee_t194.img?

Best,

Hi @JerryChang ,

Can you please provide guidance on this doubt?

Thanks!

hello jdiegodelgado,

yes, for running gen_tos_part_img.py, you should given different TOS image names for different platforms.
you may see-also flash configuration file.
Orin series, TOSFILE="bootloader/tos_t234.img";
Xavier series, TOSFILE="bootloader/tos_t194.img";

you may looking bootloader logs.
there’ll be BL31 and also OP-TEE version and built times. it shall be match your host machine build time after your TOS image has updated.

had you flash user_key which is specified in the eks.img.
there’s EKB (Encrypted Binary Blob) stores two keys, one is the kernel encryption key, and another one is the LUKS key for disk encryption support.
it’s boot-up process to derive keys from SE keyslots. this is error reported if there’s no keys available.

@JerryChang ,

Thanks again for your support. I was able to fixed the jetson_user_key_pta_init failed by generating the EKB key.
Now my question would be regarding the step 3 of the atf_and_optee_README.txt, precisely on the Generating the tos.img with ATF and OP-TEE images section, which says:

3. Copy all the files under ./optee/install/t<platform> to the target.
Now my next question is about the "helloworld ta" that I added to the OP-TEE sou

Where exactly should I copy these files within the Jetson Xavier AGX platform?

Best,

hello jdiegodelgado,

there are 3 folders in optee/install/t<platform>, i.e. /bin, /lib and /usr.
just copy those binaries to the corresponding folders, /bin, /lib and /usr in the target board.

Hi @JerryChang ,

Thanks for your response. On Friday I copied the binaries to the corresponding folders in the target board. But I started to experience some issues when I tried to run a trusted application I got some errors.

This is using the op-sources from JetPack 5.1:

When I run xtest I got:

Run test suite with level=0
TEE test application started over default TEE instance
Failed to open TEE context: 0xffff0008

Then, I tried to run the tee-supplicant:

$ tee-supplicant
ERR [2340] TEES:main:905: failed to find an OP-TEE supplicant device

Then, I checked if the tee-supplicant was running, but it seems that it was not:

$ ps | grep supplicant
NO OUTPUT

So I tried to run it as daemon, but I got:

$ tee-supplicant -d
ERR [2492] TEES:main:891: make_daemon(): -1

Next, I checked the logs to find if something went wrong during boot process and checked that the tee supplicant did start at boot. But I did not see any error:
minicom.cap (162.0 KB)

Confirm that tee-supplicant did start at boot:
[ 9.539517] systemd[1]: Started OP-TEE Client Supplicant.

After that, I checked if the tee driver was created, which it was:

$ ls /dev/tee*
/dev/tee0  /dev/teepriv0

Then, I checked if the driver was loaded:

$ sudo dmesg | grep optee
[    4.186253] optee: probing for conduit method.
[    4.187181] optee: revision 3.19 (30cb55b2)
[    4.188252] optee: dynamic shared memory is enabled
[    4.189558] optee: initialized driver

Do you have any suggestions why xtest is failing and tee-supplicant seems not be running?

Best,

BTW, for the user TAs (i.e. the TA users create), they should be put under… /lib/optee_armtz.

@JerryChang ,

Thanks for your answer. But the problem that I’m facing is with JetPack5.1 OP-TEE fresh sources. Yesterday I re-flashed the board again, just to test the OP-TEE that comes with JetPack5.1, and check if it works.
On my last post I was debugging the built-in OP-TEE. I did not add anything from my side.

Any thoughts on why is it failing?
I will tag @KevinFFF @WayneWWW to know if they have more insights about this.

$ tee-supplicant
ERR [2340] TEES:main:905: failed to find an OP-TEE supplicant device
$ xtest
Run test suite with level=0
TEE test application started over default TEE instance
Failed to open TEE context: 0xffff0008

Best,

hello jdiegodelgado,

that’s error reported by getting device fd.
could you please check whether you have sysnode like below… /dev/teepriv*

Hi @JerryChang ,

Yes, the devices seems to be correctly created:

$ ls /dev/tee*
/dev/tee0  /dev/teepriv0

Best,

Hi @JerryChang ,

Is there anything else that I can tried from my side?

This error is the one that is getting my attention:

$ tee-supplicant
ERR [2340] TEES:main:905: failed to find an OP-TEE supplicant device

Best,

hello jdiegodelgado,

we are able to reproduce the same issue locally, and we’ve created an internal thread for tracking.

since optee has booted up properly according to the logs.
did you run these commands in root permission BTW? please have a try with… $ sudo tee-supplicant

@JerryChang ,

Thanks for your reply.

I tested it. The output seems to be like the one above:
ERR [4243] TEES:main:905: failed to find an OP-TEE supplicant device.

Did you find any insight on the internal thread for tracking?

Best,

hello jdiegodelgado,

this still look like a permission issue since the optee kernel driver is up and “/dev/teepriv0” exists.
the error reported means tee-supplicant cannot open the file.
please examine the permission of the device file, $ ls -la /dev/teepriv0

@JerryChang ,

Permissions:

nvidia@tegra-ubuntu:~$ ls -la /dev/teepriv0
crw------- 1 root root 243, 16 Apr 11 20:31 /dev/teepriv0

Best,

hello jdiegodelgado,

can it works by chmod to 664 or 644?

Hi @JerryChang ,

Changed the permissions for the one that you suggested, but problem persists.

$ ls -la /dev/teepriv0
crw-rw-r-- 1 root root 243, 16 Sep  8  2022 /dev/teepriv0
$ sudo tee-supplicant
ERR [8652] TEES:main:905: failed to find an OP-TEE supplicant device

Best,

hello jdiegodelgado,

oh… had you update tee-supplicant , or you’re using the default app under rootfs for verification?
please refer to atf_and_optee_README.txt for step-3. within [Verifying the Image] section.
this means all the user-space files under /install should copy to your target.

  1. Copy all the files under ./optee/install/t<platform> to the target.

furthermore, please share all steps in details if you still cannot resolve this issue.
for example,
the command to build the OP-TEE source package, OP-TEE dtb, ATF source code…etc, and, you should generate the tos.img with python script.
what exactly binaries you’ve replace with. finally, the steps you’ve done to flash the target.