Nvcompositor does not work with NVMM attribute on jetpack 6

Hello

We are trying to run a gstreamer pipeline on L4T36.3 with jetpack 6 using nvcompositor on NVIDIA Jetson Orin NX and it gets stuck. However, it was running fine on L4T35.4.1 with jetpack 5.

Gstreamer pipeline:

GST_DEBUG=3 gst-launch-1.0 -v\
    nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160 \
                      sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 ! \
    queue ! nvvidconv ! queue ! autovideosink \
    nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_0 \
    nvv4l2camerasrc device=/dev/video1 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_1

Errors on L4T36.3 with jetpack 6 (GST_DEBUG=3):

GST_DEBUG=3 gst-launch-1.0 -v     nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160                       sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 !     queue ! nvvidconv ! queue ! autovideosink     nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_0     nvv4l2camerasrc device=/dev/video1 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_1
Setting pipeline to PAUSED ...
0:00:00.062510287  6500 0xaaab0f3c8300 FIXME                default gstutils.c:4025:gst_pad_create_stream_id_internal:<nvv4l2camerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is live and does not need PREROLL ...
0:00:00.062651443  6500 0xaaab0f3c8360 FIXME                default gstutils.c:4025:gst_pad_create_stream_id_internal:<nvv4l2camerasrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.063027613  6500 0xaaab0f3c8360 WARN         nvv4l2camerasrc gstnvv4l2camerasrc.cpp:684:gst_nv_v4l2_camera_fixate:<nvv4l2camerasrc1> Unable to fixate caps structure
0:00:00.063088127  6500 0xaaab0f3c8300 WARN         nvv4l2camerasrc gstnvv4l2camerasrc.cpp:684:gst_nv_v4l2_camera_fixate:<nvv4l2camerasrc0> Unable to fixate caps structure
New clock: GstSystemClock
0:00:00.063280964  6500 0xaaab0f3c82a0 WARN         videoaggregator gstvideoaggregator.c:1140:gst_video_aggregator_find_best_format:<comp> Nothing compatible with video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.063323557  6500 0xaaab0f3c82a0 WARN            nvcompositor gstnvcompositor.c:826:gst_nvcompositor_fixate_caps:<comp> This pad is invalid
0:00:00.063335173  6500 0xaaab0f3c82a0 WARN            nvcompositor gstnvcompositor.c:826:gst_nvcompositor_fixate_caps:<comp> This pad is invalid
0:00:00.063435368  6500 0xaaab0f3c81e0 FIXME               basesink gstbasesink.c:3395:gst_base_sink_default_event:<autovideosink0-actual-sink-xvimage> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
/GstPipeline:pipeline0/GstNvCompositor:comp.GstAggregatorPad:src: caps = video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1, height=(int)1, framerate=(fraction)30/1, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1, height=(int)1, framerate=(fraction)30/1, colorimetry=(string)sRGB
Redistribute latency...
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1, height=(int)1, framerate=(fraction)30/1, colorimetry=(string)sRGB
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage.GstPad:sink: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = video/x-raw, width=(int)1, height=(int)1, framerate=(fraction)30/1, format=(string)YUY2, colorimetry=(string)2:4:7:1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)RGBA, width=(int)1, height=(int)1, framerate=(fraction)30/1, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)2160, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)2160, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)2160, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)2160, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
Caught SIGSEGV
Spinning.  Please run 'gdb gst-launch-1.0 6500' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

Any Idea what that could be?

Thanks and best regards
Gianluca

1 Like

Hi,
We can run the command successfully:

gst-launch-1.0 -v\
    nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160 \
                      sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 ! \
    queue ! nvvidconv ! video/x-raw ! xvimagesink \
    videotestsrc is-live=1 ! 'video/x-raw, format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_0 \
    videotestsrc is-live=1 ! 'video/x-raw, format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! nvvidconv ! comp.sink_1

Please give it a try. Would like to clarify whether the failure is related to nvv4l2camerasrc.

Hi @DaneLLL

We already checked with 2x videotestsrc and I can confirm that it is working flawlessly with two test sources.

Indeed it somehow is the combination of nvv4l2camerasrc and nvcompositor, because streaming the two image halfs independently from /dev/video0 or /dev/video1 is working without any problems using this pipeline:

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! queue ! nvvidconv ! autovideosink

Furthermore we are able to get the pipeline running by replacing nvv4l2camerasrc with v4l2src (with some weird white artefacts…):

GST_DEBUG=3 gst-launch-1.0 -v \
    nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160 \
                      sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 ! \
    queue ! nvvidconv ! queue ! autovideosink \
    v4l2src device=/dev/video0 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! queue ! nvvidconv ! comp.sink_0 \
    v4l2src device=/dev/video1 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)60/1, width=(int)1920, height=(int)2160' ! queue ! nvvidconv ! comp.sink_1

But this is longterm not sufficient.

Thanks

Hi,
If you use Jetpack 6.0GA(r36.3), the white artefacts are known and there is a patch:

Jetson/L4T/r36.3.x patches - eLinux.org
[Camera][VI][JP-6.0/r36.3] Fixes for raw image corruption (line artifacts) through VI-mode

Please rebuild kernel with the patch and try again.

Thanks

However longterm we want to use nvv4l2camerasrc. Is there a way to get that running?

Hi,
Please try the command like this:
Using `nvcompositor` with `nvarguscamerasrc` leads to segfault in `nvcompositor` - #5 by DaneLLL
NvCompositor SIGSEGV - #14 by Honey_Patouceul

Please clearly set the caps to video/x-raw(memory:NVMM),format=RGBA to source pad of nvvidconv and nvcmpositor plugin.

And we can run the command with two USB cameras:

gst-launch-1.0 -v \
    nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 \
                      sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 ! 'video/x-raw(memory:NVMM),format=RGBA,width=2560,height=720' ! \
    queue ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! queue ! nv3dsink sync=0 \
    nvv4l2camerasrc device=/dev/video2 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)30/1, width=(int)1280, height=(int)720' ! \
    nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_0 \
    nvv4l2camerasrc device=/dev/video6 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, framerate=(fraction)30/1, width=(int)1280, height=(int)720' ! \
    nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_1

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.