GStreamer Output with Hardware Decoding/Encoding Sped Up

reddy.vivek.m · October 5, 2020, 3:25pm

I’m trying to get my MJPEG webcam to record at 1080P to my SD card and the result is sped up about 7.5x. It would indicate the pipeline cannot keep up with device. I’m using both hardware decoding/encoding.

gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! nvjpegdec ! 'video/x-raw' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! omxh264enc ! h264parse ! qtmux ! filesink location=test_MJPG_H264enc.mp4 -e

Any ideas on how I can improve the speed? I figure it should be able to handle this…

Honey_Patouceul · October 5, 2020, 7:32pm

You may try :

nvjpegdec outputting I420 into NVMM buffers, not into CPU buffers:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! omxh264enc ! h264parse ! qtmux ! filesink location=test_MJPG_H264enc.mp4 -e

Try nvv4l2decoder mjpeg=1 instead of nvjpeddec:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! nvv4l2decoder mjpeg=1 ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! omxh264enc ! h264parse ! qtmux ! filesink location=test_MJPG_H264enc.mp4 -e

reddy.vivek.m · October 5, 2020, 9:06pm

Running this command:

Honey_Patouceul:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! omxh264enc ! h264parse ! qtmux ! filesink location=test_MJPG_H264enc.mp4 -e

I get the following output and error:

> $ gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! omxh264enc ! h264parse ! qtmux ! filesink location=test_MJPG_H264enc.mp4 -e
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)NULL, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)NULL, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:sink: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)NULL, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)NULL, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:src: caps = image/jpeg, parsed=(boolean)true, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)30/1
> /GstPipeline:pipeline0/GstNvJpegDec:nvjpegdec0.GstPad:sink: caps = image/jpeg, parsed=(boolean)true, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)30/1
> /GstPipeline:pipeline0/GstNvJpegDec:nvjpegdec0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)30/1
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)30/1
> Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
> ===== NVMEDIA: NVENC =====
> NvMMLiteBlockCreate : Block : BlockType = 4 
> H264: Profile = 66, Level = 40 
> /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)30/1
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)30/1
> NvMMLiteVideoEncDoWork: Surface resolution (0 x 0) smaller than encode resolution (1920 x 1080)
> VENC: NvMMLiteVideoEncDoWork: 4283: BlockSide error 0x4
> Event_BlockError from 0BlockAvcEnc : Error code - 4
> Sending error event from 0BlockAvcEncERROR: from element /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0: GStreamer encountered a general supporting library error.
> Additional debug info:
> /dvs/git/dirty/git-master_linux/3rdparty/gst/gst-omx/omx/gstomxvideoenc.c(1331): gst_omx_video_enc_loop (): /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:
> OpenMAX component in error state Bad parameter (0x80001005)
> EOS on shutdown enabled -- waiting for EOS after Error
> Waiting for EOS...
> /GstPipeline:pipeline0/GstQTMux:qtmux0.GstPad:src: caps = video/quicktime, variant=(string)apple
> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/quicktime, variant=(string)apple
> ERROR: from element /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0: GStreamer encountered a general supporting library error.
> Additional debug info:
> /dvs/git/dirty/git-master_linux/3rdparty/gst/gst-omx/omx/gstomxvideoenc.c(2346): gst_omx_video_enc_handle_frame (): /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:
> OpenMAX component in error state Bad parameter (0x80001005)
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
> Additional debug info:
> gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> streaming stopped, reason error (-5)

Running this command:

Honey_Patouceul:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! nvv4l2decoder mjpeg=1 ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! omxh264enc ! h264parse ! qtmux ! filesink location=test_MJPG_H264enc.mp4 -e

I get the following output and error, but it hangs versus being terminated:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! nvv4l2decoder mjpeg=1 ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! omxh264enc ! h264parse ! qtmux ! filesink location=test_MJPG_H264enc.mp4 -e
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)NULL, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)NULL, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:sink: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)NULL, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)NULL, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:src: caps = image/jpeg, parsed=(boolean)true, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)30/1
NvMMLiteOpen : Block : BlockType = 277 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 277 
/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = image/jpeg, parsed=(boolean)true, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)30/1
/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
EOS on shutdown enabled -- waiting for EOS after Error
Waiting for EOS...
/GstPipeline:pipeline0/GstQTMux:qtmux0.GstPad:src: caps = video/quicktime, variant=(string)apple
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/quicktime, variant=(string)apple

All that being said, if I add io-mode=2 to the input, the original command works.

Topic		Replies	Views
Gstreamer pipeline streaming stopped Jetson TX2 gstreamer	2	2796	January 26, 2022
Incosistent gstreamer v4l2src "internal data stream error" Jetson Nano camera , gstreamer	3	11306	October 15, 2021
Help improving gstreamer pipeline on Xavier AGX Jetson AGX Xavier gstreamer	12	1399	October 18, 2021
Run gstreamer ,got error Jetson AGX Orin gstreamer	7	565	June 21, 2023
Problems with gstreamer video decoding pipeline Jetson AGX Xavier	11	5283	October 18, 2021
Gstreamer Pipeline issue : PosixMemMap:84 mmap failed Jetson Orin Nano gstreamer	15	675	April 25, 2024
Gstreamer use MJPEG codec Jetson Nano opencv , gstreamer	16	15746	October 15, 2021
Gstreamer and v4l2loopback Jetson Nano gstreamer	12	5651	October 15, 2021
H.264 decoding failure Jetson Nano gstreamer	12	3674	October 18, 2021
Simple Gstream USB Camera Live View Jetson AGX Orin gstreamer	12	2524	July 25, 2023

GStreamer Output with Hardware Decoding/Encoding Sped Up

Related topics