To reduce FPS of camera capturing gstreamer pipeline interfaced with Xavier NX

vishnu.prasad · May 19, 2021, 1:33pm

Hi All,

We are using Nvidia Xavier NX device to capture images from e-con see3cam USB camera to capture images continuously. The pipeline suggested by the camera provider is as follows.

gst-launch-1.0 v4l2src do-timestamp=true device=/dev/video0 ! image/jpeg,width=4208,height=3120,framerate=20/1 ! videorate ! capsfilter caps="image/jpeg" name=timelapse_framerate ! queue ! multifilesink post-messages=true location=test-%d.jpg

This will produces images at 20 fps. But we don’t need 20 fps speed.We need lower fps for our requirement. There are two image formats supported on this camera which are MJPG and UYVY.The above pipeline is using MJPG format. If we use UYVY , I think we can reduce the frame rate. Please help us on this. Also any suggestion to save the images in uuid names.

Thanks and Regards

Honey_Patouceul · May 19, 2021, 8:58pm

You may try this for slowing down to 10 fps:

gst-launch-1.0 v4l2src do-timestamp=true device=/dev/video0 ! image/jpeg,width=4208,height=3120,framerate=20/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! videorate ! video/x-raw,framerate=10/1 ! xvimagesink sync=false

Post the modes that your camera provides for better advice (v4l2-ctl is provided by apt package v4l-utils):

v4l2-ctl -d0 --list-formats-ext

vishnu.prasad · May 20, 2021, 5:50am

Thanks for the reply…
I have tried the pipeline you suggested but got the following error.

$ gst-launch-1.0 v4l2src do-timestamp=true device=/dev/video0 ! image/jpeg,width=4208,height=3120,framerate=20/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! videorate ! video/x-raw,framerate=10/1 ! xvimagesink sync=false
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.001832306
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

The modes provided by camera are as follows:

$ v4l2-ctl -d0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 3840x2160
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Size: Discrete 4208x3120
Interval: Discrete 0.111s (9.000 fps)
Interval: Discrete 0.222s (4.500 fps)
Size: Discrete 4096x2160
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Size: Discrete 640x480
Interval: Discrete 0.008s (120.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1280x960
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1440
Interval: Discrete 0.022s (45.000 fps)
Interval: Discrete 0.044s (22.500 fps)
Size: Discrete 2880x2160
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.100s (10.000 fps)

Index : 1
Type : Video Capture
Pixel Format: ‘MJPG’ (compressed)
Name : Motion-JPEG
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 3840x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 4208x3120
Interval: Discrete 0.050s (20.000 fps)
Size: Discrete 4096x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.008s (120.000 fps)
Size: Discrete 1280x960
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1920x1440
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 2880x2160
Interval: Discrete 0.033s (30.000 fps)

Honey_Patouceul · May 20, 2021, 10:57am

Seems your camera can provide raw mode UYVY at 9 fps, so that should match your expectation:

 gst-launch-1.0 v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/1 ! xvimagesink

vishnu.prasad · May 20, 2021, 11:56am

Thanks for the response. This pipeline is working fine for streaming video at 9 fps. How can we capture images at 9fps or 4.5fps and save it into the drive ?

Honey_Patouceul · May 20, 2021, 3:05pm

For capturing at 4.5 fps:

gst-launch-1.0 v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/2 ! xvimagesink

For recording at 9 fps (encoding into h264):

gst-launch-1.0 -e v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test.mp4

Not sure if nvvidconv and nvv4l2h264enc can deal with framerates below 10 fps. If it doesn’t work, you would try omxh264enc encoder:

gst-launch-1.0 -e v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/1 ! videoconvert ! omxh264enc ! h264parse ! qtmux ! filesink location=test.mp4

vishnu.prasad · May 21, 2021, 5:12am

Thanks for the response. The pipeline for streaming at 4.5 fps is working fine. But other 2 pipelines are not working and giving the following output:

For recording at 9 fps (encoding into h264):

$ gst-launch-1.0 -e v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test.mp4
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
H264: Profile = 66, Level = 0 
_ValidateEncodeParams: Invalid encode width 
^Chandling interrupt.
Interrupt: Stopping pipeline ...
EOS on shutdown enabled -- Forcing EOS on the pipeline
^C

with omxh264enc encoder:

$ gst-launch-1.0 -e v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/1 ! videoconvert ! omxh264enc ! h264parse ! qtmux ! filesink location=test.mp4
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
EOS on shutdown enabled -- waiting for EOS after Error
Waiting for EOS...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Interrupt while waiting for EOS - stopping pipeline...
Execution ended after 0:00:11.068804868
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Actually our requirement is to capture each frames at 4.5 fps and save it as images into drive.

Honey_Patouceul · May 21, 2021, 1:12pm

There may be a limitation to maximum 4096 rows or columns with nvv4l2h264enc. It may work using omxh264enc instead:

gst-launch-1.0 -e v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/1 ! nvvidconv ! omxh264enc ! h264parse ! qtmux ! filesink location=test.mp4

However, this would be a H264 encoded video.

If you want to save each raw frame into a separate file, you can do that but be aware that it can quickly fill your filesystem. If saving into the root filesystem and it gets filled, your Jeston may not be able to boot correctly, so I’d strongly advise to either save on another filesystem, or use a limit for the number of files (here using 10 frames max):

mkdir tmpTest
cd tmpTest
gst-launch-1.0 -e v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/1 ! multifilesink location=test_%05d.UYVY max-files=10

If you want to convert each image into JPG, you would use:

gst-launch-1.0 -e v4l2src do-timestamp=true device=/dev/video0 ! video/x-raw,format=UYVY,width=4208,height=3120,framerate=9/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvjpegenc ! multifilesink location=test_%05d.jpg max-files=10

vishnu.prasad · May 21, 2021, 6:24pm

Thank you very much for the response. These pipelines are working fine.We are able to capture images and save them to drive at both 4.5 fps and 9 fps using these pipelines. We are using external sdcard to store captured images so it should not directly affect the root filesystem . Can we save images to random UUID in this pipeline ?

Once again thanks for your help

DaneLLL · May 26, 2021, 8:23am

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi,

Not quite sure what this means. Please share more detail about it.

@Honey_Patouceul we really appreciate your active help. Many thanks.