On the AGX Orin, tegra release R36.3.0.
I changed the build options in Rules.mk to activate the Address Sanitizer support :
CPPFLAGS := -fno-omit-frame-pointer -fsanitize=address -fno-pie -O1 -g
LDFLAGS := -fno-omit-frame-pointer -fsanitize=address -no-pie
The 01_video_encode sample will produce the same result as Possible bug inside libnvfnet.so or nearby from 2021.
After removing the libv4l2_nvargus.so, ASAN detects a buffer overflow:
/usr/src/jetson_multimedia_api/samples/01_video_encode$ ./video_encode ../../data/Picture/nvidia-logo.yuv 1920 1080 H265 /tmp/test.h265
Creating Encoder in blocking mode
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 8
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
892744264
842091865
NvVideo: H265 : Profile : 1
NVMEDIA: Need to set EMC bandwidth : 846000
Could not read complete frame from input file
File read complete.
NvVideo: bBlitMode is set to TRUE
=================================================================
==147102==ERROR: AddressSanitizer: heap-buffer-overflow on address 0xffff94034288 at pc 0xffff9d6cab3c bp 0xffff81b2e4f0 sp 0xffff81b2e548
READ of size 680 at 0xffff94034288 thread T4
#0 0xffff9d6cab38 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
#1 0xffff958aacec (/usr/lib/aarch64-linux-gnu/nvidia/libnvmmlite_video.so+0x1acec)
#2 0xffff99649b74 (/usr/lib/aarch64-linux-gnu/nvidia/libnvos.so+0x9b74)
#3 0xffff999cd5c4 in start_thread nptl/pthread_create.c:442
#4 0xffff99a35ed8 (/lib/aarch64-linux-gnu/libc.so.6+0xe5ed8)
0xffff94034288 is located 0 bytes to the right of 1032-byte region [0xffff94033e80,0xffff94034288)
allocated by thread T0 here:
#0 0xffff9d73a2f4 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
#1 0xffff99582604 in NvMMLiteBlockCreateStream (/usr/lib/aarch64-linux-gnu/nvidia/libnvmmlite_utils.so+0x2604)
#2 0x523100 in NvVideoEncoder::setOutputPlaneFormat(unsigned int, unsigned int, unsigned int) /usr/src/jetson_multimedia_api/samples/common/classes/NvVideoEncoder.cpp:140
#3 0x42105c in encode_proc /usr/src/jetson_multimedia_api/samples/01_video_encode/video_encode_main.cpp:1988
#4 0x42105c in main /usr/src/jetson_multimedia_api/samples/01_video_encode/video_encode_main.cpp:2634
#5 0xffff999773f8 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
#6 0xffff999774c8 in __libc_start_main_impl ../csu/libc-start.c:392
#7 0x406fac in _start (/usr/src/jetson_multimedia_api/samples/01_video_encode/video_encode+0x406fac)
Thread T4 created by T2 here:
#0 0xffff9d6e6188 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0xffff99649eb0 (/usr/lib/aarch64-linux-gnu/nvidia/libnvos.so+0x9eb0)
#2 0xffff99a35ed8 (/lib/aarch64-linux-gnu/libc.so.6+0xe5ed8)
Thread T2 created by T0 here:
#0 0xffff9d6e6188 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0xffff99649eb0 (/usr/lib/aarch64-linux-gnu/nvidia/libnvos.so+0x9eb0)
#2 0x425328 in encode_proc /usr/src/jetson_multimedia_api/samples/01_video_encode/video_encode_main.cpp:2359
#3 0x425328 in main /usr/src/jetson_multimedia_api/samples/01_video_encode/video_encode_main.cpp:2634
#4 0xffff999773f8 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
#5 0xffff999774c8 in __libc_start_main_impl ../csu/libc-start.c:392
#6 0x406fac in _start (/usr/src/jetson_multimedia_api/samples/01_video_encode/video_encode+0x406fac)
SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827 in __interceptor_memcpy
Shadow bytes around the buggy address:
0x200ff2806800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x200ff2806810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x200ff2806820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x200ff2806830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x200ff2806840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x200ff2806850: 00[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff2806860: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff2806870: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x200ff2806880: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x200ff2806890: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x200ff28068a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==147102==ABORTING