JP 5.0.2 missing ~1GB volatile memory

Hey @Bibek

thanks for your answer. In the Kernel the VPR is disabled and apparently in the device tree too:

\source\public\hardware\nvidia\soc\t19x\kernel-dts\tegra194-soc\tegra194-soc-memory.dtsi
which is included by tegra194-p3668-common.dtsi → tegra194-soc/tegra194-soc-cvm.dtsi → tegra194-soc/tegra194-soc-base.dtsi → tegra194-soc-memory.dtsi

		vpr: vpr-carveout {
			compatible = "nvidia,vpr-carveout";
			status = "disabled";
		};
	
		tegra-carveouts {
			compatible = "nvidia,carveouts-t19x";
			memory-region = <&generic_reserved &vpr>;
			status = "disabled";
		};

# CONFIG_TEGRA_VPR is not set

That seems to be the default of the L4T sources. Can you please guide me how to really disable it?

Hi,

In the BCT file present in BSP misc/tegra194-memcfg-sw-override-l4t.cfg,
make this variable as ZERO
McVideoProtectSizeMb = 0x00000000;

Hey @Bibek

I changed the file and recompiled the kernel (Wondering how it should change anything, the file is not even in the kernel source folder) , the device tree is unchanged and the 1GB is still missing…
Any idea what could be wrong?

the file is present in bootloader/t186ref/BCT and has nothing to do with kernel. this is a bootloader file and from here you need to disable vpr so that bootloader does not enable vrp in kernel during boot.
Just change the file and reflash the board

Hey @Bibek

okay, I understand that. Anyway it did not have any effect after flashing.

I changed the file:
Linux_for_Tegra\bootloader\t186ref\BCT\tegra194-memcfg-sw-override-l4t.cfg

And flashed with:

ROOTFS_AB=1 ./flash.sh -S 6GiB jetson-xavier-nx-devkit mmcblk0p1    

Anyway I still have 6.7GB memory. The device tree still shows that the vpr-carveout is enabled, I am not sure if that should be updated if it is controlled through the bootloader.

I’ve found the issue. The file you told me is orphaned. It is nowhere used in the l4t. The correct file is:
tegra194-memcfg-sw-override.cfg, without the “-l4t”
You might want to delete the unused file in the next release.

Now we have:
Checked on release R32: Free shows 7959912 total memory
Checked on release R35: Free shows 7019444 total memory
Release R35 without VPR: 7710608 total memory

What else do we need to disable to get the last 250MB?

please dump your device tree using
dtc -I fs -O dts -o ./extracted.dts /proc/device-tree

and check the file extracted.dts for reserved memory

also check the cat /proc/iomem and share

Hey @Bibek ,

I have dumped both files for the current released and compared them to the R32. I could not see anything in the reserved-memory block which obviously causes the memory difference. I am not able to understand the whole device tree though.

Maybe you can see what is different?

extracted_r32.dts (259.9 KB)
extracted_r35.dts (385.9 KB)

iomem_r32.txt (7.5 KB)
iomem_r35.txt (7.2 KB)

@Bibek

any idea?

Hi, I am assuming, you have got the rel32 iomem data without being a root user, as a result all data are zero.
Please attach the dmesg logs also.
Delete the nodes under reserved-memory and see it helps.

Hey @Bibek

here is the dmesg from R35:

dmesg (106.9 KB)

I don’t have access to the dmesg of R32, do you need it too?

the log says, there is reserved memory of 463456K in rel-35 which is missing in rel-32

[ 0.000000] kernel: Memory: 7632992K/8161984K available (16320K kernel code, 3088K rwdata, 6512K rodata, 3712K init, 1082K bss, 463456K reserved, 65536K cma-reserved)

@Bibek

okay, that’s interesting. Do you have an idea what causes that?
At least it’s the default state of your software.

In the dts in reserved-memory the only thing that seems different is the camdbg_carveout, but that already is set to disabled…

As asked earlier, can you delete the cambdg_caeveout from dt,
it could be that the memory is reserved irrespective of state of the node.
check the driver for this compatible = “nvidia,camdbg_carveout”;

@Bibek

unfortunately that did not help. I removed the camdbg carvout. The memory is still missing.

Here the dts and the dmesg:

R35_nocamdbg.dts (367.2 KB)
R35_nocamdbg_dmesg (108.4 KB)

[ 0.000000] kernel: Memory: 7633092K/8161984K available (16320K kernel code, 3088K rwdata, 6512K rodata, 3712K init, 1082K bss, 463356K reserved, 65536K cma-reserved)

above is from your log, I believe Rel-32 also have same or less memory available.

Not all of 8GB (8,388,608) will be seen by kernel as bootloader reserves many memories for other usecases and those are not visible to kernel.

@Bibek

I have more memory in R32

R32:
htop: 7.59GB
free: 7959912 (same as htop with /(1024*1024))
dmesg: 7190540K/8132608K

R35 without vpr carvout:
htop: 7.35GB
free: total 7710540 (same as htop with /(1024*1024))
dmesg: 7633096K/8161984K

Yes dmesg shows even more memory for R32, with less available. But in the end in the system you have 240MB less for actual use in R35. There has to be some reason for this or not?

I’d understand it if you told me that R35 does something more or better than R32. But on the same hardware without improvements, I’d expect to have the same memory.

there are many improvements in rel-35. the memory missing is allocated to optee, uefi and for some inter-r5-shmem usecases.
Not possible to be made available in rel-35.
You can stick to rel-32 and use upstream kernel to retain the 240MB of space and also move to latest kernel

Hey @Bibek

that is very unfortunate. We are planning on using Orin NX or Orin Nano for the finished product, those both are not supported by Rel32.
It’s quite sad that the boards are not usable with the advertised memory. Our decision to use the board was based on it’s specifications. Every bit of missing memory will make our application slower.

  • We do not require optee, are you sure that it cannot be removed?
  • Why does the ARM implementation of UEFI burn system memory compared to the X64 implementation? I initially was happy that you adopt uefi, but the price of paying system memory is too high!
  • I have not read about inter-r5-shmem. What can we do with that?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.