Nvcompositor does not work with NVMM attribute on jetpack 6

parg · September 16, 2024, 1:13pm

Hello

We are trying to run a gstreamer pipeline on L4T36.3 with jetpack 6 using nvcompositor on NVIDIA Jetson Orin NX and it gets stuck. However, it was running fine on L4T35.4.1 with jetpack 5.

Gstreamer pipeline:

GST_DEBUG=3 gst-launch-1.0 -v\
    nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160 \
                      sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 ! \
    queue ! nvvidconv ! queue ! autovideosink \
    nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_0 \
    nvv4l2camerasrc device=/dev/video1 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_1

Errors on L4T36.3 with jetpack 6 (GST_DEBUG=3):

GST_DEBUG=3 gst-launch-1.0 -v     nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160                       sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 !     queue ! nvvidconv ! queue ! autovideosink     nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_0     nvv4l2camerasrc device=/dev/video1 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_1
Setting pipeline to PAUSED ...
0:00:00.062510287  6500 0xaaab0f3c8300 FIXME                default gstutils.c:4025:gst_pad_create_stream_id_internal:<nvv4l2camerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is live and does not need PREROLL ...
0:00:00.062651443  6500 0xaaab0f3c8360 FIXME                default gstutils.c:4025:gst_pad_create_stream_id_internal:<nvv4l2camerasrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.063027613  6500 0xaaab0f3c8360 WARN         nvv4l2camerasrc gstnvv4l2camerasrc.cpp:684:gst_nv_v4l2_camera_fixate:<nvv4l2camerasrc1> Unable to fixate caps structure
0:00:00.063088127  6500 0xaaab0f3c8300 WARN         nvv4l2camerasrc gstnvv4l2camerasrc.cpp:684:gst_nv_v4l2_camera_fixate:<nvv4l2camerasrc0> Unable to fixate caps structure
New clock: GstSystemClock
0:00:00.063280964  6500 0xaaab0f3c82a0 WARN         videoaggregator gstvideoaggregator.c:1140:gst_video_aggregator_find_best_format:<comp> Nothing compatible with video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.063323557  6500 0xaaab0f3c82a0 WARN            nvcompositor gstnvcompositor.c:826:gst_nvcompositor_fixate_caps:<comp> This pad is invalid
0:00:00.063335173  6500 0xaaab0f3c82a0 WARN            nvcompositor gstnvcompositor.c:826:gst_nvcompositor_fixate_caps:<comp> This pad is invalid
0:00:00.063435368  6500 0xaaab0f3c81e0 FIXME               basesink gstbasesink.c:3395:gst_base_sink_default_event:<autovideosink0-actual-sink-xvimage> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
/GstPipeline:pipeline0/GstNvCompositor:comp.GstAggregatorPad:src: caps = video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1, height=(int)1, framerate=(fraction)30/1, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1, height=(int)1, framerate=(fraction)30/1, colorimetry=(string)sRGB
Redistribute latency...
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1, height=(int)1, framerate=(fraction)30/1, colorimetry=(string)sRGB
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage.GstPad:sink: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1, height=(int)1, framerate=(fraction)30/1, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)2160, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)2160, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)2160, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)2160, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
Caught SIGSEGV
Spinning.  Please run 'gdb gst-launch-1.0 6500' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

Any Idea what that could be?

Thanks and best regards
Gianluca

DaneLLL · September 18, 2024, 2:56am

Hi,
We can run the command successfully:

gst-launch-1.0 -v\
    nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160 \
                      sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 ! \
    queue ! nvvidconv ! video/x-raw ! xvimagesink \
    videotestsrc is-live=1 ! 'video/x-raw, format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_0 \
    videotestsrc is-live=1 ! 'video/x-raw, format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_1

Please give it a try. Would like to clarify whether the failure is related to nvv4l2camerasrc.

parg · September 18, 2024, 7:43am

Hi @DaneLLL

We already checked with 2x videotestsrc and I can confirm that it is working flawlessly with two test sources.

Indeed it somehow is the combination of nvv4l2camerasrc and nvcompositor, because streaming the two image halfs independently from /dev/video0 or /dev/video1 is working without any problems using this pipeline:

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! queue ! nvvidconv ! autovideosink

Furthermore we are able to get the pipeline running by replacing nvv4l2camerasrc with v4l2src (with some weird white artefacts…):

GST_DEBUG=3 gst-launch-1.0 -v \
    nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160 \
                      sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 ! \
    queue ! nvvidconv ! queue ! autovideosink \
    v4l2src device=/dev/video0 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! queue ! nvvidconv ! comp.sink_0 \
    v4l2src device=/dev/video1 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! queue ! nvvidconv ! comp.sink_1

But this is longterm not sufficient.

Thanks

DaneLLL · September 18, 2024, 8:05am

Hi,
If you use Jetpack 6.0GA(r36.3), the white artefacts are known and there is a patch:

Jetson/L4T/r36.3.x patches - eLinux.org
[Camera][VI][JP-6.0/r36.3] Fixes for raw image corruption (line artifacts) through VI-mode

Please rebuild kernel with the patch and try again.

parg · September 18, 2024, 8:33am

Thanks

However longterm we want to use nvv4l2camerasrc. Is there a way to get that running?

DaneLLL · September 19, 2024, 5:26am

Hi,
Please try the command like this:
Using `nvcompositor` with `nvarguscamerasrc` leads to segfault in `nvcompositor` - #5 by DaneLLL
NvCompositor SIGSEGV - #14 by Honey_Patouceul

Please clearly set the caps to video/x-raw(memory:NVMM),format=RGBA to source pad of nvvidconv and nvcmpositor plugin.

And we can run the command with two USB cameras:

gst-launch-1.0 -v \
    nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 \
                      sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 ! 'video/x-raw(memory:NVMM),format=RGBA,width=2560,height=720' ! \
    queue ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! queue ! nv3dsink sync=0 \
    nvv4l2camerasrc device=/dev/video2 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)30/1, width=(int)1280, height=(int)720' ! \
    nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_0 \
    nvv4l2camerasrc device=/dev/video6 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)30/1, width=(int)1280, height=(int)720' ! \
    nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_1

system · October 9, 2024, 6:59am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
JetPack 6.1 + (IMX219) + Jetson Orin NX + nvcompositor segfaults Jetson Orin NX camera	11	90	January 10, 2025
Nvcompositor segfaults Jetson AGX Orin gstreamer	7	88	December 5, 2024
Nvcompositor is not working on Jetpack 4.6 Jetson Xavier NX gstreamer	10	1822	November 4, 2021
Gstreamer using nvcomposer to filesink hangs, but nvcomposer to nvoverlaysink works fine Jetson Nano camera , gstreamer , nvbugs	13	3433	October 15, 2021
RTSP Gstreamer pipeline with nvv4l2h264 Jetson AGX Orin camera , gstreamer	5	2141	January 11, 2023
Nvcompositor with opencv not work Jetson Nano opencv , gstreamer	11	633	January 20, 2023
Nvcompositor hang Jetson Xavier NX gstreamer	13	975	November 16, 2022
Gst for merging 4 webcams sometimes works, and sometimes it doesn't Jetson Nano camera , gstreamer	5	2265	February 2, 2022
NvCompositor SIGSEGV Jetson Xavier NX gstreamer	17	2279	May 5, 2023
Gstreamer Image compositor with video streams dosen't start with preview Jetson Nano gstreamer	10	1287	November 23, 2022

Nvcompositor does not work with NVMM attribute on jetpack 6

Related topics