Valgrind Errors/Leaks with jetson_multimedia_api Sample

christianR · May 18, 2021, 1:23pm

Hi,
i got a lot of error/leaks reported when i run the jetson_multimedia_api 02_video_dec_cuda sample with valgrind.
This is what i did:

sudo apt install nvidia-l4t-jetson-multimedia-api     #nvidia-l4t-jetson-multimedia-api (32.5.1-20210219084708)    
sudo chmod -R o+rwX /usr/src/jetson_multimedia_api
cd /usr/src/jetson_multimedia_api/samples/02_video_dec_cuda/
make
valgrind \
    --keep-debuginfo=yes \
    --leak-check=full \
    --track-origins=yes \
    --show-leak-kinds=all \
    --log-file=valgrind-out.txt \
    ./video_dec_cuda /usr/src/jetson_multimedia_api/data/Video/sample_outdoor_car_1080p_10fps.h264 H264     --disable-rendering

Valgrind Output:
valgrind-out.txt (320.8 KB)

LEAK SUMMARY:
==19390== definitely lost: 10,816 bytes in 184 blocks
==19390== indirectly lost: 2,008 bytes in 23 blocks
==19390== possibly lost: 3,664 bytes in 35 blocks
==19390== still reachable: 242,779 bytes in 385 blocks
==19390== of which reachable via heuristic:
==19390== newarray : 1,536 bytes in 16 blocks
==19390== suppressed: 0 bytes in 0 blocks
==19390==
==19390== For lists of detected and suppressed errors, rerun with: -s
==19390== ERROR SUMMARY: 212 errors from 209 contexts (suppressed: 0 from 0)

Version used:

cat /etc/nv_tegra_release
# R32 (release), REVISION: 5.1, GCID: 26202423, BOARD: t186ref, EABI: aarch64, DATE: Fri Feb 19 16:50:29 UTC 2021
nvidia-l4t-jetson-multimedia-api: 32.5.1-20210219084708
nvidia-l4t-multimedia: 32.5.1-20210219084708
valgrind: 3.16.1
g++: 7.5.0

do i need to worry about these errors/leaks or is this an valgrind issue?

DaneLLL · May 19, 2021, 12:53am

Hi,
Please refer to 00_video_decode sample. There is an option for stress test:

        -s <loop-count>      Stress test [Default = 1]

We have the test in each release to ensure the stability and check memory leak.

We now encourage users to use NvBufferTransform(). In 02_video_dec_cuda sample, NvVideoConverter is used and might be the reason of the leak. Suggest use NvBuffer APIs.

christianR · May 19, 2021, 11:37am

Does this mean it is not recommended to use NvVideoConverter? because right now i would use NvVideoConverter to convert from Block Linear Memory to Pitch Linear Memory (need CU_EGL_FRAME_TYPE_PITCH instead of CU_EGL_FRAME_TYPE_ARRAY). I could use cuMemcpy2D (TYPE_ARRAY->TYPE_DEVICE) to bypass this, but is this recommended? NvBufferTransform() doesn’t seem to support such transformations.

Back to the memory leak.
I tested with the 00_video_decode sample and got leaks a well:

valgrind ./video_decode H264 -s 1 --disable-rendering /usr/src/jetson_multimedia_api/data/Video/sample_outdoor_car_1080p_10fps.h264

==30152== HEAP SUMMARY:
==30152==     in use at exit: 270,979 bytes in 667 blocks
==30152==   total heap usage: 3,036 allocs, 2,369 frees, 26,194,875 bytes allocated
==30152==
==30152== LEAK SUMMARY:
==30152==    definitely lost: 12,880 bytes in 185 blocks
==30152==    indirectly lost: 6,136 bytes in 25 blocks
==30152==      possibly lost: 3,664 bytes in 35 blocks
==30152==    still reachable: 248,299 bytes in 422 blocks
==30152==                       of which reachable via heuristic:
==30152==                         newarray           : 1,536 bytes in 16 blocks
==30152==         suppressed: 0 bytes in 0 blocks

valgrind ./video_decode H264 -s 10 --disable-rendering /usr/src/jetson_multimedia_api/data/Video/sample_outdoor_car_1080p_10fps.h264

==30173== HEAP SUMMARY:
==30173==     in use at exit: 441,899 bytes in 2,546 blocks
==30173==   total heap usage: 14,900 allocs, 12,354 frees, 241,716,794 bytes allocated
==30173==
==30173== LEAK SUMMARY:
==30173==    definitely lost: 128,800 bytes in 1,850 blocks
==30173==    indirectly lost: 61,360 bytes in 250 blocks
==30173==      possibly lost: 3,664 bytes in 35 blocks
==30173==    still reachable: 248,075 bytes in 411 blocks
==30173==                       of which reachable via heuristic:
==30173==                         newarray           : 1,536 bytes in 16 blocks
==30173==         suppressed: 0 bytes in 0 blocks

DaneLLL · May 20, 2021, 2:10am

Hi,
You can do block linear to pitch linear conversion through NvBufferTransform(). If the source NvBuffer is block liner and destination NvBuffer is pitch linear, it does the conversion. We would suggest use NvBuffer APIs.

For memory leak, could you also try

$ cat /proc/meminfo

Would like to know if other tool also shows the issue.

christianR · May 20, 2021, 7:34am

./video_decode H264 -s 100000 --disable-rendering /usr/src/jetson_multimedia_api/data/Video/sample_outdoor_car_1080p_10fps.h264

cat /proc/meminfo Output:

~20sec (Start):

MemTotal:       32686756 kB
MemFree:        20219048 kB
MemAvailable:   30718804 kB
Buffers:          321820 kB
Cached:          6241032 kB
SwapCached:            0 kB
Active:          2386652 kB
Inactive:        4573048 kB
Active(anon):     459184 kB
Inactive(anon):   185248 kB
Active(file):    1927468 kB
Inactive(file):  4387800 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      16343360 kB
SwapFree:       16343360 kB
Dirty:                40 kB
Writeback:             0 kB
AnonPages:        381324 kB
Mapped:           314252 kB
Shmem:            247588 kB
Slab:             662196 kB
SReclaimable:     529176 kB
SUnreclaim:       133020 kB
KernelStack:        8176 kB
PageTables:         7388 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    32686736 kB
Committed_AS:    2789672 kB
VmallocTotal:   263061440 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:    118784 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
NvMapMemFree:    4041748 kB
NvMapMemUsed:     296820 kB
CmaTotal:         753664 kB
CmaFree:          686872 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

~5min:

MemTotal:       32686756 kB
MemFree:        20214844 kB  (-4204 kB to Start)
MemAvailable:   30712284 kB
Buffers:          321820 kB
Cached:          6241044 kB
SwapCached:            0 kB
Active:          2392160 kB
Inactive:        4573060 kB
Active(anon):     464692 kB
Inactive(anon):   185248 kB
Active(file):    1927468 kB
Inactive(file):  4387812 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      16343360 kB
SwapFree:       16343360 kB
Dirty:                20 kB
Writeback:             0 kB
AnonPages:        386820 kB
Mapped:           314272 kB
Shmem:            247588 kB
Slab:             662152 kB
SReclaimable:     529152 kB
SUnreclaim:       133000 kB
KernelStack:        8144 kB
PageTables:         7368 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    32686736 kB
Committed_AS:    2816952 kB
VmallocTotal:   263061440 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:    120832 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
NvMapMemFree:    4039444 kB
NvMapMemUsed:     297844 kB
CmaTotal:         753664 kB
CmaFree:          686872 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

~15min.:

MemTotal:       32686756 kB
MemFree:        20205716 kB  (-13332 kB to Start)
MemAvailable:   30703164 kB
Buffers:          321820 kB
Cached:          6241080 kB
SwapCached:            0 kB
Active:          2401428 kB
Inactive:        4573096 kB
Active(anon):     473960 kB
Inactive(anon):   185248 kB
Active(file):    1927468 kB
Inactive(file):  4387848 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      16343360 kB
SwapFree:       16343360 kB
Dirty:                28 kB
Writeback:             0 kB
AnonPages:        396088 kB
Mapped:           314340 kB
Shmem:            247588 kB
Slab:             662296 kB
SReclaimable:     529124 kB
SUnreclaim:       133172 kB
KernelStack:        8112 kB
PageTables:         7380 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    32686736 kB
Committed_AS:    2807016 kB
VmallocTotal:   263061440 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:    126976 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
NvMapMemFree:    4039444 kB
NvMapMemUsed:     297332 kB
CmaTotal:         753664 kB
CmaFree:          686872 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

~30min:

MemTotal:       32686756 kB
MemFree:        20193868 kB  (-25180 kB to Start)
MemAvailable:   30691340 kB
Buffers:          321820 kB
Cached:          6241128 kB
SwapCached:            0 kB
Active:          2413632 kB
Inactive:        4573144 kB
Active(anon):     486164 kB
Inactive(anon):   185248 kB
Active(file):    1927468 kB
Inactive(file):  4387896 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      16343360 kB
SwapFree:       16343360 kB
Dirty:                16 kB
Writeback:             0 kB
AnonPages:        408164 kB
Mapped:           314428 kB
Shmem:            247588 kB
Slab:             662232 kB
SReclaimable:     529100 kB
SUnreclaim:       133132 kB
KernelStack:        8208 kB
PageTables:         7412 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    32686736 kB
Committed_AS:    2794356 kB
VmallocTotal:   263061440 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:    139264 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
NvMapMemFree:    4039444 kB
NvMapMemUsed:     296820 kB
CmaTotal:         753664 kB
CmaFree:          686872 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

~60min:

MemTotal:       32686756 kB
MemFree:        20169956 kB  (-49092 kB to Start)
MemAvailable:   30667464 kB
Buffers:          321820 kB
Cached:          6241232 kB
SwapCached:            0 kB
Active:          2437636 kB
Inactive:        4573248 kB
Active(anon):     510168 kB
Inactive(anon):   185248 kB
Active(file):    1927468 kB
Inactive(file):  4388000 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      16343360 kB
SwapFree:       16343360 kB
Dirty:                 8 kB
Writeback:             0 kB
AnonPages:        432296 kB
Mapped:           314588 kB
Shmem:            247588 kB
Slab:             662248 kB
SReclaimable:     529032 kB
SUnreclaim:       133216 kB
KernelStack:        8112 kB
PageTables:         7456 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    32686736 kB
Committed_AS:    2852556 kB
VmallocTotal:   263061440 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:    161792 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
NvMapMemFree:    4039444 kB
NvMapMemUsed:     296820 kB
CmaTotal:         753664 kB
CmaFree:          686872 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

top shows an increace in “RES” as well for the video_decode process

christianR · March 23, 2022, 3:59pm

are there plans for issue or is this already fixed?

DaneLLL · March 24, 2022, 1:32am

Hi,
Please apply this as a quick fix:
Memory Leak (Alloc/free mismatch) in Tegra multimedia API (encoder) - #6 by DaneLLL

system · April 20, 2022, 5:23am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Memory corruption problems nvv4l2decoder/nvjpegdec in R32.4.4 + R35.4.1 Jetson TX2 decoder	4	702	December 4, 2023
Nvvideoconvert Memory Compatibility Error DeepStream SDK	6	585	September 6, 2022
Possible multimedia api regression with decode interlace source Jetson TX2 mmapi , nvbugs	28	2674	February 9, 2022
Error in Deepstream 6.1 DeepStream SDK deepstream	21	528	June 6, 2024
Gstreamer use nvdewarper frame lag and tear DeepStream SDK cudnn , jetson , deepstream	23	78	November 18, 2024
NvBufSurfTransform failed with error -2 while converting buffer DeepStream SDK	2	1749	October 12, 2021
Mc-err: (255) csr_vicsrd: EMEM address decode error Jetson AGX Orin camera , gstreamer	11	211	July 19, 2024
Argus - Memory Leak when retrieving NV12 Buffer Jetson AGX Orin camera	4	496	March 11, 2024
Memory leak even after calling NvBufSurfaceUnMap DeepStream SDK deepstream	15	52	March 20, 2025
omxh264dec/nvvidconv: 34.7MB per minute memory leak/crash when decoding Interlaced H264 video Jetson Nano	27	3073	October 14, 2021

Valgrind Errors/Leaks with jetson_multimedia_api Sample

Related topics