Memory for NvMap

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Is there a way to enlarge the size of available memory for nvmap driver? We are using nvmap_dmabuf as an exporter for dmabuf which is used in a custom kernel module for dma transfers from an external device. It is desirable that the physical memory used to be contiguous. As for now it’s not the case, it usually comes in two disjoint chunks. So is there a way to ensure that it is physically contiguous? Meminfo says that only 74524kB is NvMapMemFree, but there are a lot of available CMA memory which is apparently not used by nvmap_dmabuf.

3

hello zhyltsov,

may I know which JetPack release you’re working with?
please check the release tag for the release version,

$ cat /etc/nv_tegra_release
# R32 (release), REVISION: 5.1, GCID: 26202423, BOARD: t210ref, EABI: aarch64, DATE: Fri Feb 19 16:45:52 UTC 2021

may I know what’s your use-case?
I’ve check the memory info of my Nano platform, here’s the NvMapMemFree by default available,
for example,

$ cat /proc/meminfo | grep NvMap
NvMapMemFree:     269824 kB
NvMapMemUsed:     445324 kB
4

Hello. Yes, the release tag is the same as the one in your reply. But the available memory for NvMap is much less.

We have a pcie-connected device which does video capturing and intend to use jetson-nano for post-processing. The device itself is FPGA-controlled and as for now has very limited DMA abilities (it only works with physically contiguous memory). So the question is whether we can do something about the way the memory for nvmap_dmabuf is allocated to ensure it is contiguous?

5

hello zhyltsov,

there’s debugfs to analysis the NvMap usage, could you please share below for reference,
it dumps the memory usage of all clients from the specific mem_heap.
for example, # cat /sys/kernel/debug/nvmap/iovmm/clients

6

Output just after system booting:

CLIENT PROCESS PID SIZE
user compiz 7278 58688K
user Xorg 6826 133824K
user gnome-shell 6128 61312K
user Xorg 5904 42752K
user nvargus-daemon 5232 76K
total 296652K

Output with our application (app62) running:

CLIENT PROCESS PID SIZE
user app62 7905 64092K
user compiz 7278 68160K
user Xorg 6826 139008K
user gnome-shell 6128 61312K
user Xorg 5904 42752K
user nvargus-daemon 5232 76K
total 375400K

7

hello zhyltsov,

please use the API, NvBufferCreateEx to allocates a hardware buffer.
you may also check Multimedia API Sample Applications for reference,
thanks

8

Well, yeah, I’ve been doing exactly that: calling NvBufferCreateEx and using 12_camera_v4l2_cuda as a reference. It kinda works but only when the memory allocated by NvBufferCreateEx comes in one piece (physically) which is often not the case. So what can I do about that?

10

Hi,
It looks similar to this topic:
High CPU usage for video capturing - #14 by DaneLLL

NvBuffer is hardware DMA buffer and need to consider data alignment. Not able to be as flexible as CPU buffer. In the topic it is UYVY format with width > 4096. Another example is YUV420 in 1920x1080. The buffer is:

Y: pitch=2048 width=1920 height=1080
U: pitch=1024 width=960 height=540
V: pitch=1024 width=960 height=540

You may call NvBufferGetParams() to get the information.

11

Could you please elaborate on the similarities you see with my problem? I am well aware that NvBuffer created with NvBufferLayout_Pitched as layout parameter will be pitched and its size will be different.

12

So let’s put it that way: is there a way (for example via device tree) to reserve more memory for nvmap_dmabuf to use while allocating NvBuffer?

14

hello zhyltsov,

by default, NvMap allocates from System Heap. please check $ cat /proc/meminfo | grep MemAvailable for available memory size.
there is reserved-memory node in device tree, which reserves memory for carveouts. size in the node to reserve the amount of memory, NvMap only manages for VPR carveout. if you allocates it from reserved-memory, then it applies to your use-case.
for example,

        reserved-memory {
                ...
                vpr-carveout {
                        compatible = "nvidia,vpr-carveout";
                        size = <0x0 0x19000000>;
                        alignment = <0x0 0x400000>;
                        alloc-ranges = <0x0 0x80000000 0x0 0x70000000>;
15

VPR memory sounds nice, it’s CMA and there is a lot of it available. The question now is how to get the privilege for our application to use it since it’s protected?

16

VPR memory is limited to specific clients like NvDec, hence PCIe device can’t use it.
Can you check if “NVRM_MEM_HANDLE_SET_CONTIGUOUS_ATTR” is available as part of libnvrm_mem library?

1 Like
17

What would be the best way to do it?

18

Set this attribute while creating handle(before calling NvRmMemHandleAllocAttr)
NVRM_MEM_HANDLE_SET_CONTIGUOUS_ATTR(NvRmMemAttr,1);

19

So, we’ve ended up implementing a scatter-gather DMA on the FPGA side, so contiguous allocations are no longer of concern. Still it would be nice to have a solid reference for things that can and cannot be controlled when allocating nvbuffers.