Increase the size of the TZDRAM on TX2

Hi,

So I have a custom trust-zone OS and applications that I would like to run on the TX2, but the amount of memory provided to the OS is very small (6MB).

So my understanding from the TRM, is that the TZDRAM’s aperture is programmable using the memory controllers registers. I see that this is done in the ARM-TF’s BL31 source code provided by nvidia, and I have tried increasing the size by modifying the ARM-TF’s BL31, but the accessible region of memory doesn’t seem to be affected by this change.

Does the BPMP do some additional work to setup the TZDRAM’s carevout that I don’t know about? I am thinking this might be the case since the aperture params (base and size) are passed from the BPMP, but it’s unclear why this is done. Is this memory region configurable in some way?

Thanks!

hello richard.habeeb,

according to Topic 163562, this TZDRAM_BASE variable in the makefile is actually is TZSRAM not the TZDRAM.
please see the mb1 header file for the carveout-id, and you should setup serial console and check the bootloader logs for the carveouts.
for example,

typedef enum carve_out_type {
..
  	CARVEOUT_TZDRAM = 32
[0001.444] I> Welcome to Cboot
[0001.447] I> Cboot Version: t186-02737b3e
[0001.451] I> CPU-BL Params @ 0x275800000
...
[0001.597] I> 32) Base:0x276000000 Size:0x00600000
1 Like

Hi Jerry,

Thanks for your help. Reading between the lines of your post a bit, I realized that there is a configuration system for the MB1 that controls the carveout documented here. However when I compare the documentation of t186 and t194 MB1 configuration, I see that only the t194 seems to support parameters for tzdram size.

In short, the carveout.tzdram.size parameter seems to unavailable for the TX2. Is this true? Are there also parameters for adjusting the tzdram base? Is this possible to change in general or is it fixed in hardware?

My followup question would be this. What hardware system/controller is the MB1 interfacing with to setup this carveout?

Also, doesn’t the TX2 use TBoot or some proprietary Nvidia code for MB1 not CBoot? For anyone who is curious this head file is in the cboot sources (bootloader/partner/t18x/common/include/soc/t186/tegrabl_carveout_id.h)

Okay I have a solution to my original question, though admittedly this feels more like a hack. I’ll outline the steps.

So the only hardware needed to interface with to setup the TZDRAM aperture is the MC, which is programmed by ARM-TF BL31. If you update the function, “bl31_early_platform_setup” to modify the data structure passed from the BPMP: plat_params.tzdram_base, plat_params.tzdram_size, as well at the modify the entry_point_info_t data structure for the secure OS (this requires a messy const cast unfortunately). These assignments should happen before tegra_memctrl_tzdram_setup.

You will need to change the tzdram_base value because the carveout squeezes it between two other regions. You can find the original carveout printed out on the debug serial port. I used the following values:

  • tzdram_base = 0x260000000
  • tzdram_size = 0x10000000
  • bl32_ep.pc = 0x260100000

You can’t set the pc/link address of the bl32 kernel/TOS to be right at the start of the tzdram because it will be partially overwritten.

The final steps are to build the ARM-TF BL31, and create the combined TOS.img using the provided python tool gen_tos_part_img.py, then to flash just the secure OS partition with just this image:

sudo ./flash.sh -k secure-os --image path-to-tos.img jetson-tx2 mmcblk0p1

2 Likes

One addition to the solution steps I outline above. In order to communicate the TZDRAM base and size to the normal world, you need to update the carveout data structure in the global boot parameters; this can also be done in the BL31. This header files for this carveout data structure is defined in the source code for cboot.

  • Find the relevant header files for struct tegrabl_global_data.
  • Copy the relevant header files for this struct to the BL31 source.
  • Then calculate the address of this data structure. The address is stored in a fixed location defined in the scratch space; to do this recreate the functionality of platform_init_boot_param in the cboot source in BL31.
  • Finally use the defined struct to modify the carveout array with the new base and size of the TZDRAM.

If you don’t do this, you won’t be able to boot Normal World Linux.

3 Likes

Hi Richard,
Thanks for your sharing!
One question:(found the answer)
Did you have modified the memory layout in kernel dtb?
Seems the 0x260000000 ~ 0x270000000 which you allocated for tzdram conflict with kernel memory.

After checked the dtb file, the cboot already reserved the memory range. Needn’t manually handle it.

1.The original memory layout:
memory@80000000 {
device_type = “memory”;
reg = <0x0 0x80000000 0x0 0x70000000 //0x80000000 ~ 0xf0000000
0x0 0xf0200000 0x1 0x85600000 //0xf0200000 ~ 0x275800000
0x2 0x75e00000 0x0 0x200000 //0x275e0000 ~ 0x276000000
0x2 0x76600000 0x0 0x200000 //0x27660000 ~ 0x276800000
0x2 0x77000000 0x0 0x200000>; //0x27700000 ~ 0x277200000
};

  1. The rearrange tzdram memory layout
    reg = <0x0 0x80000000 0x0 0x70000000
    0x0 0xf0200000 0x1 0x6fe00000 // 0xf020,0000 ~ 0x2,6000,0000
    0x2 0x70000000 0x0 0x5800000
    0x2 0x75e00000 0x0 0xa00000
    0x2 0x77000000 0x0 0x200000>

BR.,
Jason

2 Likes