Another nvvidconv to v4l2sink issue

henry.fuller · March 31, 2023, 7:22pm

I’m trying to do colorspace conversion with nvvidconv and gstreamer and send the result to a v4l2loopback device. If I replace the v4l2sink with a fakesink the pipeline works.

Here is the smallest repro pipeline I can come up with:

GST_DEBUG=2 gst-launch-1.0 -v videotestsrc ! "video/x-raw, width=2880, height=1860, framerate=(fraction)30/1, format=UYVY" !  nvvidconv ! "video/x-raw(memory:NVMM),format=BGRx" ! nvvidconv ! "video/x-raw" ! fpsdisplaysink video-sink="v4l2sink device=/dev/video10" sync=false

I have read all the other related threads and tried adding identity drop-allocation=true before the sink as suggested elsewhere, but it doesn’t work.

The debug logs for the above pipeline are as followinsare as follows:

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstV4l2Sink:v4l2sink0: sync = false
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)BGRx
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)BGRx
0:00:00.056002880 187671 0xaaaae0d60300 WARN           basetransform gstbasetransform.c:1362:gst_base_transform_setcaps:<nvvconv1> transform could not transform video/x-raw(memory:NVMM), width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)BGRx in anything we support
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)BGRx
0:00:00.056356320 187671 0xaaaae0d60300 WARN           basetransform gstbasetransform.c:1362:gst_base_transform_setcaps:<nvvconv1> transform could not transform video/x-raw(memory:NVMM), width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)BGRx in anything we support
0:00:00.056376288 187671 0xaaaae0d60300 WARN                GST_PADS gstpad.c:4231:gst_pad_peer_query:<capsfilter1:src> could not send sticky events
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
0:00:00.076043904 187671 0xaaaae0d60300 WARN           basetransform gstbasetransform.c:1362:gst_base_transform_setcaps:<nvvconv1> transform could not transform video/x-raw(memory:NVMM), width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)BGRx in anything we support
0:00:00.100487136 187671 0xaaaae0d60300 WARN           basetransform gstbasetransform.c:1362:gst_base_transform_setcaps:<nvvconv1> transform could not transform video/x-raw(memory:NVMM), width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)BGRx in anything we support
0:00:00.100533312 187671 0xaaaae0d60300 WARN                 basesrc gstbasesrc.c:3072:gst_base_src_loop:<videotestsrc0> error: Internal data stream error.
0:00:00.100543424 187671 0xaaaae0d60300 WARN                 basesrc gstbasesrc.c:3072:gst_base_src_loop:<videotestsrc0> error: streaming stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
0:00:00.101006496 187671 0xaaaae0d60300 WARN           basetransform gstbasetransform.c:1362:gst_base_transform_setcaps:<nvvconv1> transform could not transform video/x-raw(memory:NVMM), width=(int)2880, height=(int)1860, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)BGRx in anything we support
Freeing pipeline ...

Any help would be appreciated!

henry.fuller · April 4, 2023, 6:59pm

I was able to get this working after much fiddling by setting the source caps from the second nvvidconv to include the colorimetry=sRGB. Unfortunately there wasn’t a great way to figure this out, the WARN messages from nvvidconv only said “cannot convert into anything we support”.

Honey_Patouceul · April 4, 2023, 9:12pm

What is your v4l2loopback version ? You can see it from dmesg when loading it:

sudo modprobe v4l2loopback
sudo dmesg | tail -1

I’m using dkms build v4l2loopback driver version 0.12.3 with AGX Orin 32GB devkit running R35.1 and this doesn’t fault:

gst-launch-1.0 -v videotestsrc ! "video/x-raw, width=2880, height=1860, framerate=(fraction)30/1, format=UYVY" !  nvvidconv ! "video/x-raw(memory:NVMM),format=BGRx" ! nvvidconv ! "video/x-raw" ! identity drop-allocation=1 ! fpsdisplaysink video-sink="v4l2sink device=/dev/video0"

[EDIT:]
Also note that videotestsrc is not so efficient on arm64 (nor is videoscale), so you may also face a bottleneck with such high pixel rate… You may also try:

gst-launch-1.0 -v videotestsrc ! "video/x-raw,width=320,height=240,framerate=30/1, format=UYVY" !  nvvidconv ! "video/x-raw(memory:NVMM),format=UYVY" ! nvvidconv ! "video/x-raw,format=BGRx,width=2880,height=1860" ! identity drop-allocation=1 ! fpsdisplaysink video-sink="v4l2sink device=/dev/video0"

henry.fuller · April 5, 2023, 6:34pm

Thanks, for the reply. I’m inserting the module directly into the kernel, so its not a simple to get the v4l2loopback version from dmesg, but let me see if I can pull that up. I did try using the identity drop-allocation=true suggestion from a different thread initially and it didn’t seem to make any difference.

Honey_Patouceul · April 5, 2023, 6:43pm

When inserting the module into the kernel, it should issue a kernel message, unless further disabled.
So just use one terminal for monitoring kernel messages:

sudo dmesg --follow

then then from another terminal, insert the module from your app and you would be able to see version displayed at load time in first terminal.

For DMKS build of v4l2loopback, you may see Gstreamer v4l2loopback hw mjpeg stream - #5 by Honey_Patouceul

system · April 19, 2023, 6:44pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.