GStreamer unable to reduce the framerate of USB webcam

[patryk.rogalski]

Hi,

I’m working on the following gst command:

gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=60/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=I420 ! videorate ! video/x-raw,framerate=30/1 ! fakesink sync=false -v

What I’m trying to achive is to capture the image from the /dev/video0 at 60 fps but then before I’ll do anything else with reduce it to for example 30 fps or any other framereate that may not be supported by the webcam. I thought that by putting the videorate I’m going to achieve my goal but this just results with the following error:

Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000753128
Setting pipeline to NULL ...
Freeing pipeline ...

If I change the command just so the framerate for capture and then videorate are the same everything works.

How can I capture at 60/30 fps but then reduce that to something like 10 fps?

[Honey_Patouceul]

For better advice, what gives?

# v4l2-ctl is provided by apt package v4l-utils

# This would output all native modes provided by your camera
v4l2-ctl -d0 --list-formats-ext

You may also try adding a queue or try mmap io-mode for v4l2src, or adding jpegparse:

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=1920, height=1080, framerate=60/1 ! queue ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=I420 ! identity ! videorate ! video/x-raw,framerate=30/1 ! fakesink -v

[patryk.rogalski]

Here is the output from v4l2-ctl v4l2-ctl -d0 --list-formats-ext - Pastebin.com

I’ve slightly modified the command you provided to this:

gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1, format=MJPG ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=I420 ! videorate ! video/x-raw,framerate=15/1 ! fakesink -v

This seems to solve the problem of the framerate, but at the same time it introduces a weird issue where sometimes the command will immediately fail or work for a few minutes (or hours) and then fail with the following error:

Error: Duplicated or bad SOF marker

[Honey_Patouceul]

I have seen issues with AGX Orin trying your case with a simulated MJPG camera with v4l2loopback, while the same was working seamlessly with XavierNX (both running a recent JP5). I had to use identity, one after v4l2src, and another one before videorate:

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! identity drop-allocation=1 ! queue ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=I420 ! identity ! videorate ! video/x-raw,framerate=15/1 ! fpsdisplaysink text-overlay=0 video-sink=fakesink -v

It didn’t work for v4l2src at 60 fps, but it did work at 30 fps. You may give it try.

Or try nvjpegdec:

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! identity drop-allocation=1 ! queue ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! video/x-raw,format=I420 ! identity ! videorate ! video/x-raw,framerate=15/1 ! fpsdisplaysink text-overlay=0 video-sink=fakesink -v

Note that your camera also provides 1080p @10 fps, so for 10 fps you wouldn’t need videorate.
Otherwise, when using videorate, better have an integer division of framerate (ex for 30 fps input, 15 or 10 or 6 or 5 or 3 or 2 or 1 would be ok).

If you have no USB bandwidth concerns, the easiest way may be to get raw video from your camera (format reported as ‘YUYV’ (YUYV 4:2:2) from V4L2, and known as YUY2 in gstreamer):

gst-launch-1.0 v4l2src io-mode=2 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! videorate ! video/x-raw,framerate=15/1 ! fpsdisplaysink text-overlay=0 video-sink=fakesink -v

# Or if you want I420 format rather than YUY2:
gst-launch-1.0 v4l2src io-mode=2 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! videorate ! videoconvert ! video/x-raw,format=I420,framerate=15/1 ! fpsdisplaysink text-overlay=0 video-sink=fakesink -v