How can I disable OP-TEE on Nvidia Jetson Xavier NX?

ptrk1 · February 21, 2023, 8:59am

Sure, here are my exact steps.

I first download the Driver Package and the Sample Root Filesystem from this link.

I then extract the Sample Root File System into the Linux_for_Tegra/rootfs directory with sudo permissions.

Then in the Linux_for_Tegra/rootfs directory, I run the following command:

sudo ./apply_binaries.sh

Then I modify p3668.conf.common as per your instructions:

please modify p3668.conf.common flash configuration file, TOSFILE="bootloader/tos-mon-only_t194.img";

Here is my modified p3668.conf.common file for your review: p3668.conf.common (6.1 KB)

In the Linux_for_Tegra/bootloader directory, I then run the following command as per your instructions:

ln -sf tos-mon-only_t194.img tos_t194.img

I verify the soft-link command succeeded by running ls -la on the Linux_for_Tegra/bootloader directory as shown below:

...
lrwxrwxrwx  1 ptrk ptrk    21 Feb 21 19:40 tos_t194.img -> tos-mon-only_t194.img

Finally, I make sure the Xavier is in Force Recovery Mode and begin the flash with the following command:

sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1

Unfortunately, the flash keeps getting stuck at the following stage as explained above:

[  10.0867 ] tegrarcm_v2 --boot recovery
[  10.0885 ] Applet version 01.00.0000
[  12.2685 ] tegrarcm_v2 --isapplet
[ 1022.1329 ] tegrarcm_v2 --ismb2
[ 2037.9416 ] tegradevflash_v2 --iscpubl
[ 2037.9452 ] CPU Bootloader is not running on device.
[ 3054.7497 ] tegrarcm_v2 --isapplet
[ 4069.5556 ] tegrarcm_v2 --ismb2
[ 5085.3636 ] tegradevflash_v2 --iscpubl
[ 5085.3669 ] CPU Bootloader is not running on device.

However, I have been able to successfully flash the device using the sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1 command if I don’t try to disable OP-TEE.

WayneWWW · February 21, 2023, 9:03am

Did you download the new tos mon only image or you are still using the default one…?

ptrk1 · February 21, 2023, 9:15am

Hi @WayneWWW ,

I have tried both using the default image and also the following image linked by @JerryChang below:

BTW,
you may try replacing tos-mon-only_t194.img with attachment, Topic240460_Feb06.7z (27.0 KB)

To use Jerry’s image, I simply untar the attachment and then run the following cp command in the Linux_for_Tegra/bootloader/ directory:

cp ~/Downloads/Topic240460_Feb06_tos-mon-only.img tos-mon-only_t194.img

This replaces the existing tos-mon-only_t194.img file in the Linux_for_Tegra/bootloader/ directory.

Then I run the following flash command:

sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1

Unfortunately, the flash gets stuck at the same spot as I mentioned in my earlier post.

ptrk1 · February 21, 2023, 11:22am

For completeness, attached are the logs for when I try to use Jerry’s image as explained in the previous post:
error-cpu-bootloader-jerry-img.txt (48.5 KB)

WayneWWW · February 21, 2023, 11:32am

please also check the serial console on your device and share log.

ptrk1 · February 22, 2023, 1:47am

Hi @WayneWWW, attached is the serial console’s output when I attempt to flash Jerry’s image as explained in the previous post:

serial-cpu-bootloader-jerry-img.txt (5.7 KB)

ptrk1 · February 23, 2023, 11:27am

Hi, just wondering if there’s any update on this issue? It would be great if someone could confirm if there’s an issue with the image I’ve been trying to flash or if I’m simply doing something wrong when flashing and the image is actually flashable. Happy to provide more details if it helps to get to the bottom of this. Thanks in advance!

JerryChang · February 24, 2023, 7:41am

hello ptrk1,

let’s back to your original request…
may I know what’s the actual use-case for looking physical address range reserved for OP-TEE?

FYI,
that carveout region for OP-TEE is allocated dynamically.
the OP-TEE reserved address range is used by security engine. kernel and user-space will not see, neither access to them.
even you replaces the image by tos-mon-only_t194.img, it’s the approach to disable OP-TEE, however, the reserved region is still there.
the issue you seen for image flash… ��SERT: plat/nvidia/tegra/soc/t194/plat_setup.c <543>
this should be mismatch binary file and the target board.

nevertheless, I would like to confirm your actual use-case before sending you another file for testing…

ptrk1 · February 24, 2023, 8:23am

Hi @JerryChang, thanks for getting back to me! For context, I’m trying to load and run the seL4 microkernel on the Nvidia Jetson Xavier device. When a small amount of physical memory (16MB from 0x80000000) is allocated to the microkernel, I am able to successfully run seL4Test, which is a set of tests to verify the kernel is working. However, when a larger amount of memory is allocated to the microkernel, I can no longer successfully run seL4Test and it crashes part way through.

My suspicion is that OP-TEE is being loaded somewhere in the DRAM Aperture and when seL4 is given access to a larger chunk of memory, it is inadvertently accessing/overwriting memory reserved for OP-TEE, which is causing seL4Test to fail.

As such, I would like to prevent any memory in the DRAM Aperture from being reserved for OP-TEE as this is a feature that is not required for my use case. Alternatively, if I knew the memory region of OP-TEE, I could also simply avoid allocating that region to seL4.

However, it is interesting that you mention the following since it seems to indicate that flashing the Xavier with tos-mon-only_t194.img doesn’t prevent memory from being reserved for OP-TEE?

even you replaces the image by tos-mon-only_t194.img, it’s the approach to disable OP-TEE, however, the reserved region is still there.

In this case, do you have any suggestions for an alternative approach? My ultimate goal is to have full access to the DRAM Aperture of the Xavier. I believe this scenario isn’t only applicable to me but also those who want to run standalone binaries bare metal on the Xavier and need full access to the Xavier’s physical memory range.

Thanks in advance!

JerryChang · February 24, 2023, 9:13am

it’s also very hard to figure out the allocation range since that carveout region for OP-TEE is allocated dynamically.

ptrk1 · February 24, 2023, 9:46am

it’s also very hard to figure out the allocation range since that carveout region for OP-TEE is allocated dynamically.

Understood. Do you have any suggestions with respect to preventing memory from being allocated for OP-TEE? For example, is it possible for someone to run a standalone binary bare metal on the Xavier without running into OP-TEE accidentally?

To me, this issue seems tied to the Nvidia UEFI bootloader. Using a different bootloader (such as U-Boot) would probably help solve my issue; however, there doesn’t seem to much information/guidance about using U-Boot with the Xavier (since its not officially supported by Nvidia). Interestingly enough, seL4 runs fine on the Jetson TX2, which uses U-Boot as the bootloader by default.

ptrk1 · February 27, 2023, 4:28am

Did older versions of Jetson Linux that were using C-Boot as the bootloader also reserve a portion of physical memory for OP-TEE on the Xavier?

JerryChang · March 5, 2023, 9:55am

hello ptrk1,

could you please have several test combinations to narrow down this failure.
for example, how “larger” will you see the failure?

BTW,
Xavier doesn’t use U-Boot.

system · March 19, 2023, 9:56am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.