Modification of CMA unsuccessfull

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Hi All,

to allocate a larger Contiguous Memory Area, I set
cma=1512M@0-4G
in ‘os_args’ in the partition ‘kernel-dtb’ in ‘nvidia/nvidia_sdk/DRIVE_Software_10.0_Linux_OS_E3550/DRIVEOS/drive-t186ref-foundation/virtualization/pct/e3550b03-t194a/linux/linux_storage_emmc.cfg’
and flashed the target device.

However it fails to allocate the CMA as ‘cat /var/log/kern.log | grep -i cma’ tells me:

Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] cma: Failed to reserve 1512 MiB
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Kernel command line: fbcon=map:9 aurixfw=AFW root=/dev/vblkdev0p1 gpt rootwait ip=off rw gpt console=ttyS0,115200n8 tegra_keep_boot_clocks cma=1512M@0-4G disable_android_paranoid_network sdhci_tegra.en_boot_part_access=1  tegraid=19.1.2.0.0 uid=1645207840800000001028280 bl_debug_data=65536@0x7ffffd0000 earlycon=uart8250,mmio32,0x0c280000 console=ttyS0,115200n8 bl_prof_dataptr=2048@0x7fb56b7000
Sep 21 00:58:45 tegra-ubuntu kernel: [    0.000000] Memory: 28583172K/29225984K available (14014K kernel code, 2656K rwdata, 5020K rodata, 7168K init, 574K bss, 642812K reserved, 0K cma-reserved)
2

Hi fixed-term.friedemann.schestag,

I’m not familiar with the kernel parameters. Could you try if “cma=1585446912” works? Or could you try to enable more debug messages of dma-contiguous.c driver for debugging this issue? Thanks!

3

Thanks for the reply. The proposed solution has not worked. I suspect it is because the system is unable to find a contiguous block of 1512Mb in 32-bit memory space…

Is there a way to free up 32-bit address space so that CMA can find a large enough contiguous block?

below you can see my /proc/iomem:
01f00000-01f1ffff : MSSNVLINK_0 aperture
02200000-0220ffff : security
02210000-0221ffff : gpio
02430000-02446fff : /pinmux@2430000
02600000-0280ffff : /dma@2600000
02c00000-02c0ffff : /mc_sid@2c00000
03110000-0311ffff : /serial@3110000
03160000-031600ff : /i2c@3160000
03180000-031800ff : /i2c@3180000
03190000-031900ff : /i2c@3190000
031b0000-031b00ff : /i2c@31b0000
031c0000-031c00ff : /i2c@31c0000
031e0000-031e00ff : /i2c@31e0000
03210000-0321ffff : /spi@3210000
03230000-0323ffff : /spi@3230000
032c0000-032cffff : /pwm@32c0000
032f0000-032fffff : /pwm@32f0000
03300000-0330ffff : /spi@3300000
03400000-0341ffff : /sdhci@3400000
03520000-03520fff : padctl
03540000-03540fff : ao
03550000-03557fff : /xudc@3550000
03558000-03558fff : /xudc@3550000
03600000-0360ffff : /xhci@3610000
03610000-0365ffff : /xhci@3610000
03820000-038205ff : /efuse@3820000
03830000-0383ffff : /kfuse@3830000
03990000-0399ffff : 3990000.mipical
039c0000-039c000f : /tachometer@39c0000
03b80000-03b80fff : NVLW_TIOCTRL aperture
03b84000-03b84fff : NVLW_NVLIPT aperture
03b86000-03b86fff : NVLW_MINION aperture
03b90000-03b93fff : NVLW_NVL aperture
03b94000-03b94fff : NVLW_SYNC2X aperture
03b96000-03b96fff : NVLW_NVLTLC aperture
03e10000-03e1ffff : base
03e20000-03e2ffff : base
03e30000-03e3ffff : base
03e40000-03e4ffff : base
03e50000-03e5ffff : base
03e60000-03e6ffff : base
03e70000-03e7ffff : base
03e80000-03e8ffff : base
03e90000-03e9ffff : base
03ea0000-03eaffff : base
03eb0000-03ebffff : base
03ec0000-03ecffff : base
03ed0000-03edffff : base
03ee0000-03eeffff : base
03ef0000-03efffff : base
03f00000-03f0ffff : base
03f10000-03f1ffff : base
03f20000-03f2ffff : base
03f30000-03f3ffff : base
03f40000-03f4ffff : base
0b9f0000-0ba2ffff : rce-pm
0c1e0000-0c1effff : gte
0c240000-0c2400ff : /i2c@c240000
0c250000-0c2500ff : /i2c@c250000
0c280000-0c28ffff : serial
0c2f0000-0c2f0fff : security
0c2f1000-0c2f1fff : gpio
0c300000-0c303fff : /pinmux@2430000
0c310000-0c310143 : can-regs
0c311000-0c311031 : glue-regs
0c312000-0c312fff : msg-ram
0c320000-0c320143 : can-regs
0c321000-0c321031 : glue-regs
0c322000-0c322fff : msg-ram
0c340000-0c34ffff : /pwm@c340000
0c360000-0c3603ff : /pmc@c360000
0c370000-0c3705ff : /pmc@c370000
0c390000-0c392ffe : /pmc@c360000
0c3a0000-0c3a0ffe : /pmc@c360000
11000000-117fffff : /iommu@12000000
12000000-127fffff : /iommu@12000000
13e10000-13e1ffff : guest
14100000-1411ffff : appl
14160000-1417ffff : appl
14180000-1419ffff : appl
14800000-1480ffff : /host1x/isp@14800000
14b00000-14bfffff : /host1x/isp-thi@14b00000
15880000-158bffff : /host1x/nvdla0@15880000
158c0000-158fffff : /host1x/nvdla1@158c0000
15a00000-15a4ffff : /host1x/nvcsi@15a00000
15c10000-15e3ffff : /host1x/vi@15c10000
15f00000-15ffffff : /host1x/vi@15c10000
16000000-167fffff : /host1x/pva0
16800000-16ffffff : /host1x/pva1
17810000-1781ffff : usermode
30000000-3003ffff : config
30040000-3007ffff : atu_dma
30200000-31ffffff : MEM
31000000-31ffffff : PCI Bus 0001:01
31000000-31ffffff : 0001:01:00.0
36000000-3603ffff : config
36040000-3607ffff : atu_dma
36200000-37ffffff : MEM
36200000-367fffff : PCI Bus 0004:01
36200000-3623ffff : 0004:01:00.0
36240000-3624ffff : 0004:01:00.0
36240000-3624ffff : atlantic
36250000-36250fff : 0004:01:00.0
36250000-36250fff : atlantic
36400000-367fffff : 0004:01:00.0
36400000-367fffff : atlantic
38000000-3803ffff : config
38040000-3807ffff : atu_dma
38200000-39ffffff : MEM
38200000-389fffff : PCI Bus 0000:01
38200000-389fffff : PCI Bus 0000:02
38200000-387fffff : PCI Bus 0000:04
38200000-3823ffff : 0000:04:00.0
38240000-3824ffff : 0000:04:00.0
38240000-3824ffff : atlantic
38250000-38250fff : 0000:04:00.0
38250000-38250fff : atlantic
38400000-387fffff : 0000:04:00.0
38400000-387fffff : atlantic
38800000-389fffff : PCI Bus 0000:03
60000000-603fffff : sem-syncpt-shim
80000000-bfcfffff : System RAM
80080000-8131ffff : Kernel code
81a20000-81d4dfff : Kernel data
ffd00000-ffd0ffff : persistent_ram
ffd10000-ffd1ffff : persistent_ram
ffd20000-ffd2ffff : persistent_ram
ffd30000-ffd3ffff : persistent_ram
ffd40000-ffd4ffff : persistent_ram
ffd50000-ffd5ffff : persistent_ram
ffd60000-ffd6ffff : persistent_ram
ffd70000-ffd7ffff : persistent_ram
ffd80000-ffd8ffff : persistent_ram
ffd90000-ffd9ffff : persistent_ram
ffda0000-ffdaffff : persistent_ram
ffdb0000-ffdbffff : persistent_ram
ffdc0000-ffdcffff : persistent_ram
ffdd0000-ffddffff : persistent_ram
ffde0000-ffdeffff : persistent_ram
ffdf0000-ffdfffff : persistent_ram
ffe00000-ffe0ffff : persistent_ram
ffe10000-ffe1ffff : persistent_ram
ffe20000-ffe2ffff : persistent_ram
ffe30000-ffe3ffff : persistent_ram
ffe40000-ffe4ffff : persistent_ram
ffe50000-ffe5ffff : persistent_ram
ffe60000-ffe6ffff : persistent_ram
ffe70000-ffe7ffff : persistent_ram
ffe80000-ffefffff : persistent_ram
100000000-7b7ffffff : System RAM
1200000000-123fffffff : MEM
1200000000-1217ffffff : PCI Bus 0001:01
1200000000-120fffffff : 0001:01:00.0
1210000000-1211ffffff : 0001:01:00.0
1400000000-17ffffffff : MEM
1800000000-1bffffffff : MEM
1800000000-1805ffffff : PCI Bus 0000:01
1800000000-1803ffffff : 0000:01:00.1
1804000000-18043fffff : 0000:01:00.1
1804000000-18043fffff : switchtec
1804400000-18047fffff : 0000:01:00.1
1804800000-1804bfffff : PCI Bus 0000:02
1804800000-18049fffff : PCI Bus 0000:03
1804a00000-1804bfffff : PCI Bus 0000:04
3fc0000000-3fffffffff : hmm_device0

4

By default, I see below from kernel messages and /proc/iomem. It seems info from “/proc/iomem” cannot tell why it fails to reserve 1.5GB.

[ 0.000000] cma: Reserved 512 MiB at 0x000000009fc00000

80000000-bfcfffff : System RAM

Could you try to enable more debug messages of dma-contiguous.c driver for debugging this issue?

5

Yes of course, how do I do that?

6

Could you refer to https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt and set loglevel=8 to enable more debug messages?