JP 5.0.2 missing ~1GB volatile memory

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
23

Hey @Bibek

okay, I understand that. Anyway it did not have any effect after flashing.

I changed the file:
Linux_for_Tegra\bootloader\t186ref\BCT\tegra194-memcfg-sw-override-l4t.cfg

And flashed with:

ROOTFS_AB=1 ./flash.sh -S 6GiB jetson-xavier-nx-devkit mmcblk0p1

Anyway I still have 6.7GB memory. The device tree still shows that the vpr-carveout is enabled, I am not sure if that should be updated if it is controlled through the bootloader.

24

Iā€™ve found the issue. The file you told me is orphaned. It is nowhere used in the l4t. The correct file is:
tegra194-memcfg-sw-override.cfg, without the ā€œ-l4tā€
You might want to delete the unused file in the next release.

Now we have:
Checked on release R32: Free shows 7959912 total memory
Checked on release R35: Free shows 7019444 total memory
Release R35 without VPR: 7710608 total memory

What else do we need to disable to get the last 250MB?

25

please dump your device tree using
dtc -I fs -O dts -o ./extracted.dts /proc/device-tree

and check the file extracted.dts for reserved memory

also check the cat /proc/iomem and share

26

Hey @Bibek ,

I have dumped both files for the current released and compared them to the R32. I could not see anything in the reserved-memory block which obviously causes the memory difference. I am not able to understand the whole device tree though.

Maybe you can see what is different?

extracted_r32.dts (259.9 KB)
extracted_r35.dts (385.9 KB)

iomem_r32.txt (7.5 KB)
iomem_r35.txt (7.2 KB)

27

@Bibek

any idea?

28

Hi, I am assuming, you have got the rel32 iomem data without being a root user, as a result all data are zero.
Please attach the dmesg logs also.
Delete the nodes under reserved-memory and see it helps.

29

Hey @Bibek

here is the dmesg from R35:

dmesg (106.9 KB)

I donā€™t have access to the dmesg of R32, do you need it too?

30

the log says, there is reserved memory of 463456K in rel-35 which is missing in rel-32

[ 0.000000] kernel: Memory: 7632992K/8161984K available (16320K kernel code, 3088K rwdata, 6512K rodata, 3712K init, 1082K bss, 463456K reserved, 65536K cma-reserved)

31

@Bibek

okay, thatā€™s interesting. Do you have an idea what causes that?
At least itā€™s the default state of your software.

In the dts in reserved-memory the only thing that seems different is the camdbg_carveout, but that already is set to disabledā€¦

32

As asked earlier, can you delete the cambdg_caeveout from dt,
it could be that the memory is reserved irrespective of state of the node.
check the driver for this compatible = ā€œnvidia,camdbg_carveoutā€;

33

@Bibek

unfortunately that did not help. I removed the camdbg carvout. The memory is still missing.

Here the dts and the dmesg:

R35_nocamdbg.dts (367.2 KB)
R35_nocamdbg_dmesg (108.4 KB)

34

[ 0.000000] kernel: Memory: 7633092K/8161984K available (16320K kernel code, 3088K rwdata, 6512K rodata, 3712K init, 1082K bss, 463356K reserved, 65536K cma-reserved)

above is from your log, I believe Rel-32 also have same or less memory available.

Not all of 8GB (8,388,608) will be seen by kernel as bootloader reserves many memories for other usecases and those are not visible to kernel.

35

@Bibek

I have more memory in R32

R32:
htop: 7.59GB
free: 7959912 (same as htop with /(1024*1024))
dmesg: 7190540K/8132608K

R35 without vpr carvout:
htop: 7.35GB
free: total 7710540 (same as htop with /(1024*1024))
dmesg: 7633096K/8161984K

Yes dmesg shows even more memory for R32, with less available. But in the end in the system you have 240MB less for actual use in R35. There has to be some reason for this or not?

Iā€™d understand it if you told me that R35 does something more or better than R32. But on the same hardware without improvements, Iā€™d expect to have the same memory.

36

there are many improvements in rel-35. the memory missing is allocated to optee, uefi and for some inter-r5-shmem usecases.
Not possible to be made available in rel-35.
You can stick to rel-32 and use upstream kernel to retain the 240MB of space and also move to latest kernel

37

Hey @Bibek

that is very unfortunate. We are planning on using Orin NX or Orin Nano for the finished product, those both are not supported by Rel32.
Itā€™s quite sad that the boards are not usable with the advertised memory. Our decision to use the board was based on itā€™s specifications. Every bit of missing memory will make our application slower.

  • We do not require optee, are you sure that it cannot be removed?
  • Why does the ARM implementation of UEFI burn system memory compared to the X64 implementation? I initially was happy that you adopt uefi, but the price of paying system memory is too high!
  • I have not read about inter-r5-shmem. What can we do with that?
40

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.