SIGSEGV from nvcompositor (pp_vic:src) during repeated PLAYING→NULL→PLhttps://forums.developer.nvidia.com/t/gst-launch-1AYING cycles on Jetson Orin NX

Robotics & Edge Computing Jetson Systems Jetson Orin NX
1

I am debugging a recurring segmentation fault from nvcompositor on a Jetson Orin NX.

Environment

  • Device: NVIDIA Jetson Orin NX 16GB

  • Pipeline uses NVMM / VIC / nvv4l2h264enc

Problem

The pipeline works correctly during the first iteration.

After:

  • EOS

  • PLAYING → NULL

  • PLAYING again

I eventually get:

Thread 71 "pp_vic:src" received signal SIGSEGV
0x0000ffffd4474ee8 in ??
from /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcompositor.so

The crash consistently occurs from:

pp_vic:src

which appears to be the nvcompositor streaming thread.

Pipeline structure

Video branch

filesrc
 -> decodebin
 -> nvvideoconvert
 -> capsfilter(memory:NVMM)
 -> queue
 -> nvcompositor sink_0

Overlay branch

videotestsrc
 -> capsfilter
 -> videoconvert
 -> cairooverlay
 -> nvvideoconvert
 -> capsfilter(memory:NVMM)
 -> queue
 -> nvcompositor sink_1

The overlay is stitched below the main video.

Main processing branch

nvcompositor
 -> nvvideoconvert
 -> capsfilter
 -> nvvideoconvert
 -> capsfilter(NV12)
 -> nvv4l2h264enc
 -> queue
 -> h264parse
 -> mp4mux
 -> filesink

Main loop

The pipeline is initialized only once.

Each iteration does:

gst_element_set_state(pipeline, GST_STATE_READY);
gst_element_set_state(pipeline, GST_STATE_PAUSED);
gst_element_set_state(pipeline, GST_STATE_PLAYING);

wait_for_eos();

gst_element_set_state(pipeline, GST_STATE_NULL);

Then the next iteration starts again.

Important details

  • Uses request pads on nvcompositor

  • Request pads are released during destroy

  • flush-on-eos=TRUE on queues

  • NVMM caps everywhere before compositor

  • videotestsrc is configured with:

    • is-live=FALSE

    • num-buffers=N

Suspected issue

I suspect one of:

  • VIC thread still active during NULL transition

  • nvcompositor internal race

  • EOS synchronization issue between overlay/video branches

I also noticed:

NvVideo: bBlitMode is set to TRUE

before the crash.

Questions

  1. Is nvcompositor known to have teardown races across repeated PLAYING->NULL cycles on Jetson?

  2. Is videotestsrc problematic inside an Aggregator pipeline?

  3. Is there a recommended teardown order for:

    • nvcompositor

    • nvv4l2h264enc

    • mp4mux

  4. Should the pipeline instead be fully recreated each iteration rather than reused?

GDB debug output :

Post processing has been started ....
Starting iteration 0
Opening in BLOCKING MODE 
[New Thread 0xffffed0aee60 (LWP 26712)]
[New Thread 0xffffec863e60 (LWP 26713)]
[New Thread 0xffffdfffee60 (LWP 26714)]
[New Thread 0xffffdf7eee60 (LWP 26715)]
[New Thread 0xffffdefdee60 (LWP 26716)]
[New Thread 0xffffde7cee60 (LWP 26717)]
[New Thread 0xffffddfbee60 (LWP 26718)]
[New Thread 0xffffdd47ee60 (LWP 26719)]
[New Thread 0xffffdcc3fe60 (LWP 26720)]
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
[New Thread 0xffffb99f5e60 (LWP 26721)]
[New Thread 0xffffb91e5e60 (LWP 26722)]
[New Thread 0xffffb89d5e60 (LWP 26723)]
NvMMLiteBlockCreate : Block : BlockType = 261 
[New Thread 0xffffb81c5e60 (LWP 26724)]
[New Thread 0xffffb61bfe60 (LWP 26725)]
Linked lOvCamPPDecodebin pad to capsPad
[New Thread 0xffffb5149e60 (LWP 26726)]
NvMMLiteOpen : Block : BlockType = 4 
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
[New Thread 0xffff89a1ee60 (LWP 26727)]
[New Thread 0xffff8920ee60 (LWP 26728)]
H264: Profile = 66 Level = 0 
[New Thread 0xffff81608e60 (LWP 26729)]
[New Thread 0xffff80df8e60 (LWP 26730)]
NVMEDIA: Need to set EMC bandwidth : 2872000 
NvVideo: bBlitMode is set to TRUE 
[Thread 0xffffb81c5e60 (LWP 26724) exited]
[Thread 0xffff8920ee60 (LWP 26728) exited]
EOS for: /app/VideoSamples/mk/GstreamerPipelineTest/OV_sampleviDeo_0.mp4
Elapsed time: 70.1156 seconds
Post Processing completed for iteration 0
[Thread 0xffff89a1ee60 (LWP 26727) exited]
Resetting pipeline for next iteration 0
[Thread 0xffffb89d5e60 (LWP 26723) exited]
[Thread 0xffffb91e5e60 (LWP 26722) exited]
[Thread 0xffffb99f5e60 (LWP 26721) exited]
[Thread 0xffff80df8e60 (LWP 26730) exited]
[Thread 0xffff81608e60 (LWP 26729) exited]
[Thread 0xffffed0aee60 (LWP 26712) exited]
[Thread 0xffffec863e60 (LWP 26713) exited]
[Thread 0xffffdf7eee60 (LWP 26715) exited]
[Thread 0xffffdefdee60 (LWP 26716) exited]
[Thread 0xffffdfffee60 (LWP 26714) exited]
[Thread 0xffffdcc3fe60 (LWP 26720) exited]
[Thread 0xffffdd47ee60 (LWP 26719) exited]
[Thread 0xffffddfbee60 (LWP 26718) exited]
[Thread 0xffffde7cee60 (LWP 26717) exited]
Re-initializing pipeline for next iteration 0
Starting iteration 1
Opening in BLOCKING MODE 
[New Thread 0xffffde7cee60 (LWP 26731)]
[New Thread 0xffffddfbee60 (LWP 26732)]
[New Thread 0xffffdd47ee60 (LWP 26733)]
[New Thread 0xffffdcc3fe60 (LWP 26734)]
[New Thread 0xffffed0aee60 (LWP 26735)]
[New Thread 0xffffec89ee60 (LWP 26736)]
[New Thread 0xffffdfffee60 (LWP 26737)]
[New Thread 0xffffdf7eee60 (LWP 26738)]
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
[New Thread 0xffffb9ffee60 (LWP 26739)]
[New Thread 0xffffb97eee60 (LWP 26740)]
[New Thread 0xffffb8fdee60 (LWP 26741)]
NvMMLiteBlockCreate : Block : BlockType = 261 
[New Thread 0xffffb87cee60 (LWP 26742)]
Linked lOvCamPPDecodebin pad to capsPad
[New Thread 0xffffa7ffee60 (LWP 26743)]
NvMMLiteOpen : Block : BlockType = 4 
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 

Thread 24 "pp_vic:src" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffddfbee60 (LWP 26732)]
0x0000ffffee194ee8 in ?? () from /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcompositor.so
(gdb) bt
#0  0x0000ffffee194ee8 in  () at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcompositor.so
#1  0x0000fffff593ee5c [PAC] in  () at /usr/lib/aarch64-linux-gnu/libgstvideo-1.0.so.0
#2  0x0000fffff5b29038 in  () at /usr/lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#3  0x0000fffff7d2180c in  () at /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#4  0x0000fffff7b08ab8 in g_thread_pool_thread_proxy (data=<optimized out>) at ../glib/gthreadpool.c:350
#5  0x0000fffff7b08064 in g_thread_proxy (data=0xffffe8000e40) at ../glib/gthread.c:831
#6  0x0000fffff75bd5c8 in  () at /usr/lib/aarch64-linux-gnu/libc.so.6
#7  0x0000fffff7625edc in  () at /usr/lib/aarch64-linux-gnu/libc.so.6
(gdb)

currentPPoptplying
currentPPoptplying10027×569 441 KB

Any debugging suggestions would help a lot — I’ve been stuck on this for weeks.

for full code : VICpipeline

4

Hi,
Since nvcompositor plugin is open source, please download the source code and add debug prints to check further. Please download the packge and follow README to build the plugin:

Jetson Linux Release 36.5 | NVIDIA Developer
Driver Package (BSP) Sources

Please also try the sample commands in developer guide and see if the issue still appears:
Accelerated GStreamer — NVIDIA Jetson Linux Developer Guide

5

As i am running this entire pipeline inside the docker container (on deepstream:7.0-triton-multiarch)
and i am construct and initialize the pipeline once
And then i am reusing the pipeline by changing the src , sink locations and the piepline states.

I think the Nvcompositor src thread holds the previous NVMM buffer memory eventhough I flush and cleanup the ncompositor before setting it to null.

And the NVcompositor works fine for PLAYING → NULL → pipeline dies

But it doesn’t work for Playing → NULL → Playing cycle

And I have added some debug prints in the nvcompositor.c file and here they are

GST_DEBUG_LOGS


0:00:11.208544416  5315 0xffff780010f0 WARN            nvcompositor gstnvcompositor.c:1324:do_nvcomposite:<pp_vic> About to NvBufSurfTransformMultiInputBufCompositeBlend, dst_surf=0xffff6c488db0 input_count=1
Flushed compositor sink_0 pad for next iteration 0
Flushed compositor sink_1 pad for next iteration 0
Flushed compositor pads for next iteration 0
EOS for: /app/VideoSamples/mk/GstreamerPipelineTest/OV_sampleviDeo_0.mp4
Elapsed time: 11.1127 seconds
Post Processing completed for iteration 0
Resetting pipeline for next iteration 0
Flushed compositor sink_0 pad for next iteration 0
Flushed compositor sink_1 pad for next iteration 0
^[[FFlushed compositor pads for next iteration 0
0:00:14.741151616  5315 0xaaaae0e691b0 WARN            nvcompositor gstnvcompositor.c:1072:gst_nvcompositor_stop:<pp_vic> nvcompositor STOP called — freeing pool=0xffff6c4856d0
0:00:14.741230912  5315 0xaaaae0e691b0 WARN            nvcompositor gstnvcompositor.c:1079:gst_nvcompositor_stop:<pp_vic> nvcompositor pool freed and NULL'd
0:00:14.741253280  5315 0xaaaae0e691b0 WARN            nvcompositor gstnvcompositor.c:1083:gst_nvcompositor_stop:<pp_vic> freeing comp_params: src_rect=0xaaaae0dff030 dst_rect=0xaaaae0dff140 alpha=0xaaaae0dff250 color_bg=0xaaaae0dff2a0
0:00:14.741267136  5315 0xaaaae0e691b0 WARN            nvcompositor gstnvcompositor.c:1098:gst_nvcompositor_stop:<pp_vic> nvcompositor STOP complete
Starting iteration 1
NvMMLiteOpen : Block : BlockType = 261 
NvMMLiteBlockCreate : Block : BlockType = 261 
Opening in BLOCKING MODE 
Opening in BLOCKING MODE 
Linked lOvCamPPDecodebin pad to capsPad
NvMMLiteOpen : Block : BlockType = 4 
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
0:00:16.034511808  5315 0xffff78000e40 WARN            nvcompositor gstnvcompositor.c:913:gst_nvcompositor_negotiated_caps:<pp_vic> Release old pool
0:00:16.034587968  5315 0xffff78000e40 WARN            nvcompositor gstnvcompositor.c:916:gst_nvcompositor_negotiated_caps:<pp_vic> negotiated_caps: out=3840x2260 fmt=19 — old pool released, decide_allocation will create new one

From this i have infered that the gst_nvcompositor_init() started when the pipeline is started
do_nvcomposite() → works fine during playing …
When the pipeline goes NULL , gst_nvcompositor_stop() will execute and remove all the reference pointers.
Here the crash occurs when i am again set the pipeline playing , init() will not execute and it directly calls the do_nvcomposite() → it tries to access the NULL pointer causing the SIGSEG Fault.