Stream 4k webcam - GStreamer

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hi, I have Kodak PIXPRO SP360 4k camera connected to the Jetson Nano via USB cable.

I want to be able to see that video over browser, either with RTSP stream, Webrtc or something else.
It doesn’t matter how it works in terms of technology, as long as it works. So if you have any ideas or suggestions be free to share them.

I’m currently trying to run the basic setup.

./test-launch "nvarguscamerasrc ! video/x-raw(memory:NVMM), format=NV12, width=1920, height=1080, framerate=30/1 ! nvvidconv ! video/x-raw, width=640, height=480, format=NV12, framerate=30/1 ! omxh265enc ! rtph265pay name=pay0 pt=96 config-interval=1"

and

gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/test ! queue ! decodebin ! videoconvert ! xvimagesink

and I’m getting the error saying

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://127.0.0.1:8554/test
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0: Could not get/set settings from/on resource.
Additional debug info:
gstrtspsrc.c(6999): gst_rtspsrc_setup_streams_start (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
SDP contains no streams
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

(test-launch:22440): GLib-GObject-WARNING **: 11:36:46.018: invalid cast from 'GstRtpH265Pay' to 'GstBin'

(test-launch:22440): GStreamer-CRITICAL **: 11:36:46.018: gst_bin_get_by_name: assertion 'GST_IS_BIN (bin)' failed

(test-launch:22440): GLib-GObject-WARNING **: 11:36:46.018: invalid cast from 'GstRtpH265Pay' to 'GstBin'

(test-launch:22440): GStreamer-CRITICAL **: 11:36:46.018: gst_bin_get_by_name: assertion 'GST_IS_BIN (bin)' failed

(test-launch:22440): GLib-GObject-WARNING **: 11:36:46.018: invalid cast from 'GstRtpH265Pay' to 'GstBin'

(test-launch:22440): GStreamer-CRITICAL **: 11:36:46.018: gst_bin_get_by_name: assertion 'GST_IS_BIN (bin)' failed

I have also tried an option that worked for me on PC but I can’t get it to work on Jetson. The setup goes as follows.

Download Streameye from https://github.com/ccrisan/streameye and run:

netcat -l 8700 | ./streameye -p 1337

To send the webcam stream I run:

gst-launch-1.0 v4l2src device=/dev/video0 ! decodebin ! videoconvert ! videoscale ! videorate ! jpegenc quality=30 ! tcpclientsink host=127.0.0.1 port=8700

After this I get:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3064): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:03.944998186
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
1 Like
2

Hi,
We support USB cameras through uvc drivers. Please check FAQ
Q: I have a USB camera. How can I lauch it on Jetson Nano?

3

Hi, I don’t have a problem displaying webcam with Gstreamer or VLC.
I need to be able to access the video via browser from another PC.

4

Also, this may be important. When I run the commands from the post you’ve linked I get:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
5

Hi,
Linking ‘v4l2src ! decodebin’ looks not right. You should get supported formats through ‘v4l2-ctl’ and configure it into v4l2src.

6

I don’t understand why wouldn’t it work. It works on Ubuntu 18.04 laptop, with the same camera and same pipeline.

When I run

v4l2-ctl -d /dev/video1 --list-formats-ext

I get:

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Discrete 3840x2160
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 2880x2880
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 2048x2048
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1440x1440
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.200s (5.000 fps)
1 Like
7

Hi,
Please try following commands:

$ sudo jetson_clocks
$ gst-launch-1.0 v4l2src device=/dev/video1 io-mode=2 ! image/jpeg,width=1280,height=720,framerate=30/1 ! nvjpegdec ! video/x-raw ! xvimagesink

There is a known memleak on r32.2.1. Please also apply the prebuilt lib:
https://elinux.org/L4T_Jetson/r32.2.1_patch
[MMAPI][DS4.0] Memory leak in MJPEG decoding

8

So this is “working”, it’s displaying 4k video in 5fps (that’s supported), but it goes to the green screen after some time. I think that will be fixed once I do the patch.

I can’t use this with the streameye, it gives me the following

2019-11-29 10:44:48: ERROR: input: jpeg size too large, discarding buffer

when I run:

gst-launch-1.0 v4l2src device=/dev/video1 io-mode=2 ! image/jpeg,width=640,height=360,framerate=5/1 ! nvjpegdec ! video/x-raw ! tcpclientsink host=127.0.0.1 port=8700

How can I use your pipeline with ./test-launch from gst-server examples in order to get RTSP that I can access from another device?

9

When I run:

./test-launch " v4l2src device=/dev/video1 io-mode=2 ! image/jpeg,width=1280,height=720,framerate=30/1 ! nvjpegdec ! video/x-raw ! xvimagesink name=pay0 pt=96 config-interval=1"

And try to capture RTSP stream with

gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/test ! queue ! decodebin ! videoconvert ! xvimagesink

I get the following error in first terminal:

** (test-launch:32689): CRITICAL **: 11:10:48.227: gst_rtsp_media_create_stream: assertion 'GST_IS_PAD (pad)' failed

(test-launch:32689): GStreamer-CRITICAL **: 11:10:48.227: gst_object_unref: assertion 'object != NULL' failed

And this in the one where I ran pipeline to capture rtsp:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://127.0.0.1:8554/test
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0: Could not get/set settings from/on resource.
Additional debug info:
gstrtspsrc.c(6999): gst_rtspsrc_setup_streams_start (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
SDP contains no streams
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
10

Hi,
Please refer to examples of using test-launch:
https://devtalk.nvidia.com/default/topic/1018689/jetson-tx2/vlc-playing-gstreamer-flow/post/5187270/#5187270
https://devtalk.nvidia.com/default/topic/1043770/jetson-tx2/problems-minimizing-latency-and-maximizing-quality-for-rtsp-and-mpeg-ts-/post/5295828/#5295828

11

I’ve tried it, you can see it in the comment above. I’m not sure which part of this pipeline needs to go with ./test-launch.

$ gst-launch-1.0 v4l2src device=/dev/video1 io-mode=2 ! image/jpeg,width=1280,height=720,framerate=30/1 ! nvjpegdec ! video/x-raw ! xvimagesink

Those examples are more or less just the basic examples, I can’t seem to combine them.

12

Hi,
In general, the pipeline is ‘source + encode + rtph26Xpay’. You may try

"v4l2src device=/dev/video1 io-mode=2 ! image/jpeg,width=1280,height=720,framerate=30/1 ! nvjpegdec ! video/x-raw ! omxh265enc ! rtph265pay name=pay0 pt=96"
13

Hi,

When I run that and try to read the stream using

gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/test ! queue ! decodebin ! videoconvert ! xvimagesink

I get the following. In the window where I started the stream I get:

stream ready at rtsp://127.0.0.1:8554/test
Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8 
NVMEDIA: H265 : Profile : 1

And in the one where I tried to read it I get:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://127.0.0.1:8554/test
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 279 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 279 
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0:
not negotiated

And it just keep printing the warning.

14

You may try VLC player as the example does. Would be better if you can run example successfully as a reference.

15

I am trying that at the moment, and it doesn’t work. I am running the same pipeline and

vlc rtsp://127.0.0.1:8554/test

and I’m getting:

VLC media player 3.0.8 Vetinari (revision 3.0.8-0-gf350b6b5a7)
[00000055b0f2c600] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Created new TCP socket 38 for connection
Bus error (core dumped)

stream ready at rtsp://127.0.0.1:8554/test
0:00:04.075103919 29935   0x55867b4590 WARN                     omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/jetsontx2/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
0:00:04.091633187 29935   0x7fa8005720 FIXME           videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<nvjpegdec0> Sub-class should implement drain()
0:00:04.096427771 29935   0x7fa8005720 WARN          v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:05.566157767 29935   0x7fa8005720 FIXME           videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<nvjpegdec0> Sub-class should implement drain()
0:00:05.569537944 29935   0x7fa8005720 FIXME           videoencoder gstvideoencoder.c:661:gst_video_encoder_setcaps:<omxh265enc-omxh265enc0> GstVideoEncoder::reset() is deprecated
Framerate set to : 5 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8 
0:00:05.571147168 29935   0x7fa8005720 WARN             omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<omxh265enc-omxh265enc0> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
NVMEDIA: H265 : Profile : 1 
0:00:05.633032247 29935   0x55867b4590 FIXME              rtspmedia rtsp-media.c:3841:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:05.634079772 29935   0x55867b4590 FIXME              rtspmedia rtsp-media.c:3841:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:05.634117756 29935   0x55867b4590 WARN               rtspmedia rtsp-media.c:3867:gst_rtsp_media_suspend: media 0x7fb00a0280 was not prepared
0:00:05.638013840 29935   0x55867b4590 FIXME             rtspclient rtsp-client.c:1657:handle_play_request:<GstRTSPClient@0x55869a1110> Add support for seek style (null)
0:00:05.638167505 29935   0x55867b4590 FIXME              rtspmedia rtsp-media.c:2437:gst_rtsp_media_seek_full:<GstRTSPMedia@0x7fb00a0280> Handle going back to 0 for none live not seekable streams.
0:00:06.427259002 29935   0x55867b4590 WARN               rtspmedia rtsp-media.c:4156:gst_rtsp_media_set_state: media 0x7fb00a0280 was not prepared
16

Hi,
There is a know memleak. Please apply the patch.
https://elinux.org/L4T_Jetson/r32.2.1_patch
[MMAPI][DS4.0] Memory leak in MJPEG decoding

With patch being applied, we can launch E-Con See3CAM_CU135 in executing the command:

$ sudo jetson_clocks
$ ./test-launch "v4l2src device=/dev/video1 io-mode=2 ! image/jpeg,width=3840,height=2160,framerate=30/1 ! nvjpegdec ! video/x-raw ! omxh265enc ! rtph265pay name=pay0 pt=96"
17

Ok, I’ll try this as soon as I do the patch.
On that topic, I can’t find any instructions on the forum on how to apply these patches.

18

Hi,
Please overwrite libnvjpeg.so

$ sudo cp -i libnvjpeg.so /usr/lib/aarch64-linux-gnu/tegra
[sudo] password for nvidia:
cp: overwrite '/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so'? y
19

It still doesn’t work.
I’m trying:

./test-launch "v4l2src device=/dev/video0 io-mode=2 ! image/jpeg,width=3840,height=2160,framerate=5/1 ! nvjpegdec ! video/x-raw ! omxh265enc ! rtph265pay name=pay0 pt=96"

I am able to run

fprobe -v quiet -print_format json -show_streams rtsp://127.0.0.1:8554/test

And I get:

{
    "streams": [
        {
            "index": 0,
            "codec_name": "hevc",
            "codec_long_name": "H.265 / HEVC (High Efficiency Video Coding)",
            "profile": "Main",
            "codec_type": "video",
            "codec_time_base": "0/1",
            "codec_tag_string": "[0][0][0][0]",
            "codec_tag": "0x0000",
            "width": 3840,
            "height": 2160,
            "coded_width": 3840,
            "coded_height": 2176,
            "has_b_frames": 0,
            "sample_aspect_ratio": "0:1",
            "display_aspect_ratio": "0:1",
            "pix_fmt": "yuv420p",
            "level": 120,
            "color_range": "tv",
            "refs": 1,
            "r_frame_rate": "5/1",
            "avg_frame_rate": "0/0",
            "time_base": "1/90000",
            "start_pts": 27361,
            "start_time": "0.304011",
            "disposition": {
                "default": 0,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0
            }
        }
    ]
}

But when I try to capture it with VLC using:

vlc rtsp://127.0.0.1:8554/test

I get:

VLC media player 3.0.8 Vetinari (revision 3.0.8-0-gf350b6b5a7)
[0000005576fed600] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Created new TCP socket 38 for connection
Bus error (core dumped)
20

Hi,
Seems to be an issue in vlc installed on Jetson Nano. We use VLC player on x86 PC.

On Nano, please try uridecodebin:

$ gst-launch-1.0 uridecodebin uri=rtsp://127.0.0.1:8554/test ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1920,height=1080' ! nvoverlaysink

If your TV supports 4K resolution, you can remove nvvidconv which does downscale to 1080p.

$ gst-launch-1.0 uridecodebin uri=rtsp://127.0.0.1:8554/test ! nvoverlaysink