Formatting images to feed into NvVideoEncoder (Tegra multimedia API)

Hello,

there are 2 camera , first camera with MMAPI/FFMPEG functions and second run with a C++/Gstreamer functions (see3cam CU130).

That’s work and record well … but , when i stop first camera , there have a crash after closes of the NvVideoConverter and the NvVideoEncoder.

Show log :

WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvConvertHEVC	:01645:	 convertFrame():	recv Image with an null imageData buffer 
WARN: NvConvertHEVC	:01272:	 LoopConvertFrame():	LoopConvertFrame Close
WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
WARN: NvConvertHEVC	:00171:	 cleanmBufferI420():	 DROP BUFFER I420 IMAGE
WARN: NvConvertHEVC	:00171:	 cleanmBufferI420():	 DROP BUFFER I420 IMAGE
WARN: NvConvertHEVC	:00171:	 cleanmBufferI420():	 DROP BUFFER I420 IMAGE
WARN: NvConvertHEVC	:01275:	 LoopConvertFrame():	LoopConvertFrame cleanup DONE
WARN: NvConvertHEVC	:01277:	 LoopConvertFrame():	LoopConvertFrame image : END 

CloseHEVC m_Encoder_HEVC
ERROR: NvInterfaceHEVC	:00156:	 CloseHEVC():	wait for stopping Encoder 
ERROR: NvEncoderHEVC	:01894:	 encodeFrame():	Could not read complete frame from input stream
ERROR: NvEncoderHEVC	:01925:	 encodeFrame():	File read complete.

ERROR: NvEncoderHEVC	:01351:	 StopStream():	Send EOS to ENC output plane
ERROR: NvEncoderHEVC	:01364:	 StopStream():	Send -1 to waitForDQThread EOS 
ERROR: NvEncoderHEVC	:01370:	 StopStream():	 Stop ENC output plane
ERROR: NvEncoderHEVC	:01375:	 StopStream():	capture_plane.setStreamStatus
INFO: NvEncoderHEVC	:01380:	 StopStream():	Camera video streaming off ...

WARN: NvEncoderHEVC#011:01748:#011 LoopCompressFrame():#011LoopCompressFrame DONE
kernel: [  734.119206] autorecord[2061]: unhandled level 2 translation fault (11) at 0x00000018, esr 0x92000006
kernel: [  734.119222] pgd = ffffffc0dee10000
kernel: [  734.119244] [00000018] *pgd=000000016c131003, *pmd=0000000000000000
kernel: [  734.119261] 
kernel: [  734.119269] CPU: 1 PID: 2061 Comm: autorecord Not tainted 3.10.96+ #1
kernel: [  734.119275] task: ffffffc0c4f34c00 ti: ffffffc0c4f38000 task.ti: ffffffc0c4f38000
kernel: [  734.119284] PC is at 0x7f8009d01c
kernel: [  734.119287] LR is at 0x7f8009d01c
kernel: [  734.119292] pc : [<0000007f8009d01c>] lr : [<0000007f8009d01c>] pstate: 60000000
kernel: [  734.119295] sp : 0000007f0f63fe40
kernel: [  734.119298] x29: 0000007f0f641610 x28: 0000007f0f640890 
kernel: [  734.119304] x27: 0000000000000000 x26: 0000007f0f640ad8 
kernel: [  734.119309] x25: 0000007f0f640a90 x24: 0000000000000001 
kernel: [  734.119313] x23: 0000000000000000 x22: 0000007f44f2fee0 
kernel: [  734.119318] x21: 0000000000000001 x20: 0000007f180007e8 
kernel: [  734.119322] x19: 0000007f0f640558 x18: 0000000000000000 
kernel: [  734.119327] x17: 0000007f8a27a340 x16: 0000007f800b0328 
kernel: [  734.119331] x15: 0000000000000000 x14: 0000000000000000 
kernel: [  734.119335] x13: 0000000000000870 x12: 0000000000000000 
kernel: [  734.119340] x11: 0000000000000000 x10: 0000000000000f00 
kernel: [  734.119344] x9 : 0000000000000004 x8 : 0000007f0f640760 
kernel: [  734.119349] x7 : 0000000000000000 x6 : 000000000000003f 
kernel: [  734.119353] x5 : 0000000000000040 x4 : ffffffffffffffe0 
kernel: [  734.119357] x3 : 0000000000000020 x2 : 0000000000000000 
kernel: [  734.119362] x1 : 0000000000000000 x0 : 0000007f0f640558 
kernel: [  734.119366] 
kernel: [  734.119388] Library at 0x7f8009d01c: 0x7f8008f000 /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so
kernel: [  734.119398] Library at 0x7f8009d01c: 0x7f8008f000 /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so
kernel: [  734.119407] vdso base = 0x7f8b454000
[Switching to Thread 0x7f3bffee20 (LWP 2219)]
0x0000007fac14701c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so
(gdb) bt
#0  0x0000007fac14701c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so
#1  0x0000007fac13bef0 in NvDdk2dBlitExt () from /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so
#2  0x0000007f672fcd38 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_video.so
#3  0x0000007f672cfa20 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_video.so
#4  0x0000007f672bab7c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_video.so
#5  0x0000007f67279fa4 in NvMMLiteBlockDoWork () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_utils.so
#6  0x0000007f6728ba10 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite.so
#7  0x0000007f4835293c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
#8  0x0000007f48347e54 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
#9  0x0000007f48315224 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
#10 0x0000007f4831f4a4 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
#11 0x0000007fac0ed40c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#12 0x0000007fb6ef7fc4 in start_thread (arg=0x7fac0ed3d0) at pthread_create.c:335
#13 0x0000007fb6e702e0 in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:89

can i run 2 cameras with Nvidia Converter ?!

I didn’t find information about NvRmMemHanldeAllocAttr in MMAPI

kernel: [  663.604911] nvmap_alloc_handle: PID 1932: autorecord: WARNING: All NvMap Allocations must have a tag to identify the subsystem allocating memory.Plase pass the tag to the API call NvRmMemHanldeAllocAttr() or relevant.

Hi Syd, do you allocate one NvVideoConverter for two cameras? Or each camera has its own NvVideoConverter?

The first camera (photonis/pleora) used functions based on git and haven’t V4L2 drivers support. I allocate and it have own NvVideoConverter and NvVideoEncoder.

Second camera (see3cam CU130) , use C++/Gstreamer functions and have own pipeline with gst_element_factory_make , like this :

gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,with=3840,heigth=2160,pixelformat="MJPG" ! jpegdec ! queue ! clockoverlay ! tee ! queue ! videorate ! video/x-raw,framerate=30/1 ! omxh264enc ! queue ! avimux ! filesink location="test.avi" sync=false

Note : the queue between omxh264enc and avimux is used for the multi-files algorithm ( queue/avimux link between pad source and pad sink ).
I tested each camera alone, that works fine.

I make many test :

  • Then i stop first camera before ; there are a Segment Fault crash, like in previous post.
  • Then i stop second camera before , there are no problem and first camera record and stop without problem.

I try to understand link between NvVideoConverter/NvVideoEncoder and libnvomx libnvmmlite_utils libnvmmlite libnvmmlite_video libnvddk_2d_v2

With Gstreamer, i didn’t know for allocate in NvVideoConverter/NvVideoEncoder. May be add explicit NvVideoConverter before omxh264enc plugins in Gstreamer Pipeline.

i see you are fixed “libv4l2_nvvidconv: set mapping failed” issu in r28.2.

But, I use a old mmapi , so i add li_lin tips in mmapi and i fix v4l2_enc_hw_preset_type like patch exemple.

i dev for TX1/TX2 , but i can testing in TX2 when i have this tegra , of course I fix problems on if there are.

i check this post for decode / converter
https://devtalk.nvidia.com/default/topic/1006857/jetson-tx1/-mmapi-under-what-conditions-lsquo-conv-gt-capture_plane-qbuffer-rsquo-will-block/

but i have always same issu :

Thread 37 "apps" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f3bffee20 (LWP 18782)]
0x0000007fac14301c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so

NvVideoConverter and NvVideoEncoder close well.

in GDB , i have :

CloseHEVC
CloseHEVC m_Converter_HEVC
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvConvertHEVC	:01757:	 convertFrame():	recv Image with an null imageData buffer 
WARN: NvConvertHEVC	:01384:	 LoopConvertFrame():	LoopConvertFrame Close
ERROR: NvConvertHEVC	:00199:	 cleanup():	try to cleanup
ERROR: NvConvertHEVC	:00205:	 cleanup():	try to stop m_ContextDevice/conv0
ERROR: NvConvertHEVC	:00210:	 cleanup():	try to StopStream
ERROR: NvConvertHEVC	:00212:	 cleanup():	abort Context Device
ERROR: NvConvertHEVC	:00874:	 conv0_capture_dqbuf_thread_callback():	conv0_capture_dqbuf_thread_callback START
ERROR: NvConvertHEVC	:00924:	 conv0_capture_dqbuf_thread_callback():	==> conv0_capture_dqbuf_thread_callback CONV0
ERROR: NvConvertHEVC	:00217:	 cleanup():	waitFor thread capture_plane
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvInterfaceHEVC	:00131:	 CloseHEVC():	wait for stopping Converter 
ERROR: NvConvertHEVC	:00221:	 cleanup():	waitFor thread output_plane
ERROR: NvConvertHEVC	:00226:	 cleanup():	stop thread capture_plane
[Thread 0x7f628efe20 (LWP 19230) exited]
[Thread 0x7f620efe20 (LWP 19231) exited]
ERROR: NvConvertHEVC	:00260:	 cleanup():	try to delete m_ContextDevice/conv0
ERROR: NvConvertHEVC	:00303:	 cleanup():	delete conv0 Device
ERROR: NvConvertHEVC	:00311:	 cleanup():	clean conv1_output_plane_buf_queue Device
ERROR: NvConvertHEVC	:00326:	 cleanup():	delete conv1_output_plane_buf_queue Device
ERROR: NvConvertHEVC	:00333:	 cleanup():	delete conv1 Device
ERROR: NvConvertHEVC	:00354:	 cleanup():	App run was successful
ERROR: NvConvertHEVC	:00194:	 cleanBufferImage():	 DROP BUFFER IMAGE
ERROR: NvConvertHEVC	:00172:	 cleanmBufferI420():	 DROP BUFFER I420 IMAGE
WARN: NvConvertHEVC	:01387:	 LoopConvertFrame():	LoopConvertFrame cleanup DONE
WARN: NvConvertHEVC	:01389:	 LoopConvertFrame():	LoopConvertFrame image : END 
[Thread 0x7f67bfae20 (LWP 19229) exited]
CloseHEVC m_Encoder_HEVC
ERROR: NvInterfaceHEVC	:00156:	 CloseHEVC():	wait for stopping Encoder 
ERROR: NvEncoderHEVC	:01897:	 encodeFrame():	Could not read complete frame from input stream
ERROR: NvEncoderHEVC	:01928:	 encodeFrame():	File read complete.
[Thread 0x7f495fee20 (LWP 19233) exited]
ERROR: NvEncoderHEVC	:01352:	 StopStream():	Send EOS to ENC output plane
ERROR: NvEncoderHEVC	:01365:	 StopStream():	Send 2000 to waitForDQThread EOS 
ERROR: NvEncoderHEVC	:01373:	 StopStream():	 Stop ENC output plane
ERROR: NvEncoderHEVC	:01378:	 StopStream():	capture_plane.setStreamStatus
INFO: NvEncoderHEVC	:01383:	 StopStream():	Camera video streaming off ...
ERROR: NvEncoderHEVC	:01318:	 cleanup():	delete ENC was calling
[Thread 0x7f618efe20 (LWP 19232) exited]
INFO: NvEncoderHEVC	:01339:	 cleanup():	App run was successful

WARN: NvEncoderHEVC	:01751:	 LoopCompressFrame():	LoopCompressFrame DONE
[Thread 0x7f683fae20 (LWP 19228) exited]

Thread 37 "autorecord" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f3bffee20 (LWP 19248)]
0x0000007fac14201c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so
(gdb) bt full
#0  0x0000007fac14201c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so
No symbol table info available.
#1  0x0000007fac136ef0 in NvDdk2dBlitExt () from /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so
No symbol table info available.
#2  0x0000007f6734cd38 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_video.so
No symbol table info available.
#3  0x0000007f6731fa20 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_video.so
No symbol table info available.
#4  0x0000007f6730ab7c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_video.so
No symbol table info available.
#5  0x0000007f672c9fa4 in NvMMLiteBlockDoWork () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_utils.so
No symbol table info available.
#6  0x0000007f672dba10 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvmmlite.so
No symbol table info available.
#7  0x0000007f4833e93c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
No symbol table info available.
#8  0x0000007f48333e54 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
No symbol table info available.
#9  0x0000007f48301224 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
No symbol table info available.
#10 0x0000007f4830b4a4 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
No symbol table info available.
#11 0x0000007fac0e840c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
No symbol table info available.
#12 0x0000007fb6ef5fc4 in start_thread (arg=0x7fac0e83d0) at pthread_create.c:335
        pd = 0x7fac0e83d0
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {546467474976, 546683477336, 0, 546683477327, 548347478992, 548530135040, 546683483408, 546671940400, 1, 546467474976, 546467472912, 12755117748680772276, 0, 12755117750644732720, 0, 
                0, 0, 0, 0, 0, 0, 0}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#13 0x0000007fb6e6e2e0 in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:89

CloseHEVC() function close convert and encode thread , but there are not log for the end of CloseHEVC function.

i search if this destroyed the Gstreamer pipeline (like equal index problems) , leak memory or HW HEVC closing issu ?

if somebody have idea … --’

Hi Syd,
Not sure but seems you are combining MM APIs and gstreamer? This is not verified and supported.

FFMPEG is from 3rdparty and we don’t have enough experience to give suggestion. Hope other users can help you.
For USB3 camera see3con CU130, you can refer to tegra_multimedia_api\samples\12_camera_v4l2_cuda

Hi DaneLLL,

Thanx for this information.
I can’t rewrite all code for moving Gstreamer to MMAPI … It’s the money time !!!

So, I find a solution , i add rules : stop see3cam CU130 fisrt , and close camera with MMAPI after. I lost a few seconds video on see3cam CU130.

But I have crash then Gstreamer run alone ( see3cam CU130 ) … how i can clean GPU memory without reboot the Tegra ?!

Thank you for your interest !

Sorry, bad typing , we must read see3cam instead of see3con

Where can I find source code Nvidia (git) for understand how Gstreamer work with OMX and HEVC component on TX1 / TX2?!

thx

hi Syd, We don’t support OMX-IL layer.

thx for information.

I find OMX git and OMX gstreamer plugins git ([url]https://github.com/GStreamer/gst-omx[/url]), but I try to fix that for furth version.

Today, search to fix strong probleme … freeze and lose frames while several seconds (5’’ to 10’') , that’s every minute.