USB Camera MJPEG to h265

I am using the Jetson Nano to capture video from a UVC camera using the v4l2src plugin, but when using the pipeline below, the video is only around 700k and does not contain any video. I have read the ACCELERATED GSTREAMER USER GUIDE, but it does not contain much information for sending jpeg streams to omxh265enc.

Any advice would be greatly received.

gst-launch-1.0 -vvv v4l2src device=/dev/video0 do-timestamp=true ! image/jpeg, width=3264, height=2448, framerate=15/1 ! jpegparse ! nvjpegdec ! 'video/x-raw' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! omxh265enc ! matroskamux ! filesink location=h265_example.mkv

Hi,
You may break down jpeg decoding and h265 encoding. The following pipeline can check if jpeg decoding is working:

gst-launch-1.0 -vvv v4l2src device=/dev/video0 do-timestamp=true ! image/jpeg, width=3264, height=2448, framerate=15/1 ! jpegparse ! nvjpegdec ! 'video/x-raw' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! nvoverlaysink

When running the provided command the display shows a single frame and then the terminal shows the following messages:

Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
There may be a timestamping problem, or this computer is too slow.

The hardware accelerated jpeg decoding cannot handle this amount of data? This seems surprising.

Assuming you have no CSI camera connected, and /dev/video0 appears a few seconds after you plug your UVC camera in, check which formats your camera driver provides with:

v4l2-ctl -d /dev/video0 --list-formats-ext

(v4l2-ctl is available from apt package v4l-utils)

Get a supported MJPG mode, for example if seeing this:

Index       : 1
	Type        : Video Capture
	Pixel Format: <b>'MJPG'</b> (compressed)
	Name        : Motion-JPEG
		Size: Discrete <b>640x480</b>
			Interval: Discrete 0.008s (120.000 fps)
			Interval: Discrete 0.011s (90.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s <b>(5.000 fps)</b>

I’d suggest to first try with low resolution and framerate.

You may then try this pipeline:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'image/jpeg, width=640, height=480, framerate=5/1, format=MJPG' ! jpegdec ! xvimagesink

If this doesn’t work, you may post the output of these commands for better advice, and may also try to use io-mode(=1 or 2) option of v4l2src plugin.

hi everyone

I want to stream h264 with uvc camera(mjpg)

so i trying like this
"
nvidia@nvidia:~/source/gst-rtsp-server-1.14.5/examples$ ./test-launch “v4l2src device=/dev/video0 do-timestamp=true ! image/jpeg, width=640, height=480, framerate=30/1 ! jpegparse ! nvjpegdec ! ‘video/x-raw’ ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=I420,width=640,height=480’ ! omxh265enc ! rtph265pay name=pay0 pt=96”
stream ready at rtsp://127.0.0.1:8554/test
No protocol specified
nvbuf_utils: Could not get EGL display connection

(test-launch:28114): GStreamer-CRITICAL **: 18:43:18.220: gst_element_make_from_uri: assertion ‘gst_uri_is_valid (uri)’ failed

(test-launch:28114): GStreamer-CRITICAL **: 18:43:18.222: gst_element_make_from_uri: assertion ‘gst_uri_is_valid (uri)’ failed

(test-launch:28114): GStreamer-CRITICAL **: 18:43:18.228: gst_element_link_pads_filtered: assertion ‘GST_IS_BIN (parent)’ failed

(test-launch:28114): GStreamer-CRITICAL **: 18:43:38.248: gst_element_make_from_uri: assertion ‘gst_uri_is_valid (uri)’ failed

(test-launch:28114): GStreamer-CRITICAL **: 18:43:38.249: gst_element_make_from_uri: assertion ‘gst_uri_is_valid (uri)’ failed

(test-launch:28114): GStreamer-CRITICAL **: 18:43:38.249: gst_element_link_pads_filtered: assertion ‘GST_IS_BIN (parent)’ failed

"


nvidia@nvidia:~/source/gst-rtsp-server-1.14.5/examples$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2
Size: Discrete 2592x1944
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.267s (3.750 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Size: Discrete 1280x960
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.008s (120.000 fps)
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.008s (120.000 fps)
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'MJPG' (compressed)
    Name        : Motion-JPEG
            Size: Discrete 1920x1080
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
            Size: Discrete 1280x720
                    Interval: Discrete 0.022s (45.000 fps)
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)
            Size: Discrete 640x480
                    Interval: Discrete 0.033s (30.000 fps)
                    Interval: Discrete 0.067s (15.000 fps)

What should it review ?

You may first try to access camera before further encoding/streaming.

For YUYV422, you may try video format YUY2:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480, framerate=30/1, format=YUY2 ! videoconvert ! xvimagesink

For MJPG, you would try jpeg format MJPG:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=640, height=480, framerate=30/1, format=MJPG ! jpegdec ! xvimagesink
#if it fails try to add jpegparse
gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=640, height=480, framerate=30/1, format=MJPG ! jpegparse ! jpegdec ! xvimagesink

Does one work ?

If it works, you would try to encode and write into file for checking that encoding works:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480, framerate=30/1, format=YUY2 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! omxh265enc ! matroskamux ! filesink location=test_YUY2_H265enc.mkv
gst-launch-1.0 -v v4l2src device=/dev/video0 ! image/jpeg, width=640, height=480, framerate=30/1, format=MJPG ! jpegdec ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! omxh265enc ! matroskamux ! filesink location=test_MJPG_H265enc.mkv

If it’s fine, you would try to stream with test-launch:

test-launch "v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480, framerate=30/1, format=YUY2 ! nvvidconv ! video/x-raw(memory:NVMM), format=NV12 ! omxh265enc ! rtph265pay name=pay0 pt=96 config-interval=1 "
test-launch "v4l2src device=/dev/video0 ! image/jpeg, width=640, height=480, framerate=30/1, format=MJPG ! jpegdec ! nvvidconv ! video/x-raw(memory:NVMM), format=NV12 ! omxh265enc ! rtph265pay name=pay0 pt=96 config-interval=1 "

1.2.3.4 steps all command works !

Thanks Honey_Patouceul