Using gst-launch-1.0 for webcam live streaming between Ubuntu PC (host) and Nvidia Jetson TX2(target)

Pratosha · January 3, 2018, 5:50am

Hello,

I have connected the Ubuntu PC and the Jetson TX2 board with ethernet cable.
A USB webcam is connected to the host PC. I am trying to establish live video streaming from the host PC to the target(Jetson TX2).
The command used on the host PC is as follows:

$ gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480,framerate=30/1 ! vp8enc ! rtpvp8pay ! udpsink host=192.168.43.159 port=5000

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
Redistribute latency…
/GstPipeline:pipeline0/GstVP8Enc:vp8enc0.GstPad:src: caps = “video/x-vp8,\ profile=(string)0,\ streamheader=(buffer)<\ 4f56503830010100028001e00000010000010000001e00000001\ >,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstRtpVP8Pay:rtpvp8pay0.GstPad:src: caps = “application/x-rtp,\ media=(string)video,\ clock-rate=(int)90000,\ encoding-name=(string)VP8,\ payload=(int)96,\ ssrc=(uint)987696047,\ timestamp-offset=(uint)1312859554,\ seqnum-offset=(uint)22401,\ a-framerate=(string)30”
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = “application/x-rtp,\ media=(string)video,\ clock-rate=(int)90000,\ encoding-name=(string)VP8,\ payload=(int)96,\ ssrc=(uint)987696047,\ timestamp-offset=(uint)1312859554,\ seqnum-offset=(uint)22401,\ a-framerate=(string)30”
/GstPipeline:pipeline0/GstRtpVP8Pay:rtpvp8pay0.GstPad:sink: caps = “video/x-vp8,\ profile=(string)0,\ streamheader=(buffer)<\ 4f56503830010100028001e00000010000010000001e00000001\ >,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstVP8Enc:vp8enc0.GstPad:sink: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstRtpVP8Pay:rtpvp8pay0: timestamp = 1313041915
/GstPipeline:pipeline0/GstRtpVP8Pay:rtpvp8pay0: seqnum = 22401

On the Jetson TX2 board,the command used is as follows:

$ gst-launch-1.0 udpsrc port=5000 caps = “application/x-rtp,\ media=(string)video,\ clock-rate=(int)90000,\ encoding-name=(string)VP8,\ payload=(int)96,\ ssrc=(uint)987696047,\ timestamp-offset=(uint)1312859554,\ seqnum-offset=(uint)992,\ a-framerate=(string)30” ! rtpvp8depay ! vp8dec ! nvvidconv ! xvimagesink

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock

The cursor keeps blinking after this with no gst-launch-1.0 video output console window.

$ gst-launch-1.0 udpsrc port=5000 caps = “application/x-rtp,\ media=(string)video,\ clock-rate=(int)90000,\ encoding-name=(string)VP8,\ payload=(int)96,\ ssrc=(uint)987696047,\ timestamp-offset=(uint)1312859554,\ seqnum-offset=(uint)22401,\ a-framerate=(string)30” ! rtpvp8depay ! vp8dec ! xvimagesink

It is the same output on the usage of this command as well.
Is there any changes to be made for the video output to be displayed?

Kindly help.

Thanks in advance.

DaneLLL · January 3, 2018, 8:16am

Hi,
You may try omxvp8dec on TX2:

rtpvp8depay ! <b>omxvp8dec ! nvoverlaysink</b>

A post about udp h264 streaming for your reference:
https://devtalk.nvidia.com/default/topic/1027423/jetson-tx2/gstreamer-issue-on-tx2/post/5225972/#5225972

Pratosha · January 3, 2018, 10:24am

Hi DaneLLL,

Thanks for your suggestion.

I tried replacing the filters in the pipeline with rtpvp8depay ! omxvp8dec ! nvoverlaysink, but I am still facing the same issue.

Thanks

DaneLLL · January 4, 2018, 2:05am

Hi,
Please check if you can get solution here:
http://gstreamer-devel.966125.n4.nabble.com/template/NamlServlet.jtp?macro=search_page&node=966125&query=vp8enc+rtsp

If you can have your server/client pipelines work fine with vp8enc and vp8dec, it should also work by using omxvp8dec

Pratosha · January 4, 2018, 6:29am

Hello,

Yes DaneLLL , the link shared was of much use.
I figured out encoding and decoding issues while using vp8 . Hence switched over to avenc_mpeg4 and avdec_mpeg4.

I tried to execute the commands for encoding and decoding video thats recorded from the webcam on both Jetson Tx2 and Ubuntu PC.
The following commands below were successfully executed.

COMMAND TO RUN CAMERA ON TX2 AND UBUNTU PC
$gst-launch-1.0 v4l2src device=“/dev/video1” ! ‘video/x-raw, format=(string)I420,width=(int)640, height=(int)480’ ! xvimagesink

COMMAND TO RUN A LIVE CAMERA AND ENCODE IT AND SAVE IT AS A .mp4 file(TX2 AND UBUNTU PC)
$ gst-launch-1.0 v4l2src device=“/dev/video1” ! ‘video/x-raw, format=(string)I420, width=(int)640, height=(int)480’ ! avenc_mpeg4 ! qtmux ! filesink location=test33.mp4

COMMAND TO DECODE THE .mp4 file saved.

TX2 -
$gst-launch-1.0 filesrc location = test33.mp4 ! qtdemux name=demux demux.video_0 ! queue ! mpeg4videoparse ! avdec_mpeg4 ! nveglglessink -e

UBUNTU-
$gst-launch-1.0 filesrc location = test33.mp4 ! qtdemux name=demux demux.video_0 ! queue ! mpeg4videoparse ! avdec_mpeg4 ! autovideosink -e

Now I connected the Ubuntu PC and TX2 via the ethernet cable.
UBUNTU PC - 192.168.43.159
TX2 - 192.168.43.160

I then executed the following commands:
The command used on the host PC is as follows:
$gst-launch-1.0 -v v4l2src device=“/dev/video2” ! videoconvert ! videoscale ! ‘video/x-raw ,format=(string)I420, width=(int)640, height=(int)480’ ! avenc_mpeg4 ! rtpmp4vpay ! udpsink host=192.168.43.159 port=5001

** (gst-launch-1.0:7841): CRITICAL **: gst_ffmpeg_cfg_set_property: assertion ‘qdata->size == sizeof (gint64)’ failed
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:src: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/avenc_mpeg4:avenc_mpeg4-0.GstPad:sink: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = “video/x-raw,\ format=(string)I420,\ width=(int)640,\ height=(int)480,\ pixel-aspect-ratio=(fraction)1/1,\ interlace-mode=(string)progressive,\ framerate=(fraction)30/1”
/GstPipeline:pipeline0/avenc_mpeg4:avenc_mpeg4-0.GstPad:src: caps = “video/mpeg,\ mpegversion=(int)4,\ systemstream=(boolean)false,\ profile=(string)simple,\ width=(int)640,\ height=(int)480,\ framerate=(fraction)30/1,\ pixel-aspect-ratio=(fraction)1/1”
/GstPipeline:pipeline0/GstRtpMP4VPay:rtpmp4vpay0.GstPad:sink: caps = “video/mpeg,\ mpegversion=(int)4,\ systemstream=(boolean)false,\ profile=(string)simple,\ width=(int)640,\ height=(int)480,\ framerate=(fraction)30/1,\ pixel-aspect-ratio=(fraction)1/1”
/GstPipeline:pipeline0/GstRtpMP4VPay:rtpmp4vpay0.GstPad:src: caps = “application/x-rtp,\ media=(string)video,\ payload=(int)96,\ encoding-name=(string)MP4V-ES,\ ssrc=(uint)1644981825,\ timestamp-offset=(uint)239444051,\ seqnum-offset=(uint)23738,\ a-framerate=(string)30”
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = “application/x-rtp,\ media=(string)video,\ payload=(int)96,\ encoding-name=(string)MP4V-ES,\ ssrc=(uint)1644981825,\ timestamp-offset=(uint)239444051,\ seqnum-offset=(uint)23738,\ a-framerate=(string)30”
/GstPipeline:pipeline0/GstRtpMP4VPay:rtpmp4vpay0.GstPad:src: caps = “application/x-rtp,\ media=(string)video,\ clock-rate=(int)90000,\ encoding-name=(string)MP4V-ES,\ profile-level-id=(string)1,\ config=(string)000001b001000001b58913000001000000012000c48d8800f514043c1443000001b24c61766335362e36302e313030,\ payload=(int)96,\ seqnum-offset=(uint)23738,\ timestamp-offset=(uint)239444051,\ ssrc=(uint)1644981825,\ a-framerate=(string)30”
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = “application/x-rtp,\ media=(string)video,\ clock-rate=(int)90000,\ encoding-name=(string)MP4V-ES,\ profile-level-id=(string)1,\ config=(string)000001b001000001b58913000001000000012000c48d8800f514043c1443000001b24c61766335362e36302e313030,\ payload=(int)96,\ seqnum-offset=(uint)23738,\ timestamp-offset=(uint)239444051,\ ssrc=(uint)1644981825,\ a-framerate=(string)30”
/GstPipeline:pipeline0/GstRtpMP4VPay:rtpmp4vpay0: timestamp = 239464203
/GstPipeline:pipeline0/GstRtpMP4VPay:rtpmp4vpay0: seqnum = 23738

On the Jetson TX2 board,the command used is as follows:

$gst-launch-1.0 udpsrc port=5001 caps =“application/x-rtp,\ media=(string)video,\ clock-rate=(int)90000,\ encoding-name=(string)MP4V-ES,\ profile-level-id=(string)1,\ config=(string)000001b001000001b58913000001000000012000c48d8800f514043c1443000001b24c61766335362e36302e313030,\ payload=(int)96,\ seqnum-offset=(uint)23738,\ timestamp-offset=(uint)239444051,\ ssrc=(uint)1644981825,\ a-framerate=(string)30” ! rtpmp4vdepay ! avdec_mpeg4 ! nveglglessink -e

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Got context from element ‘eglglessink0’: gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING …
New clock: GstSystemClock

It is the same output on the usage of this command as well.The cursor keeps blinking after this with no gst-launch-1.0 video output console window.

Kindly help and correct me if I am going wrong with the usage of commands while sending the livestream from PC to TX2.

Thanks

Pratosha · January 4, 2018, 6:30am

.

DaneLLL · January 4, 2018, 6:34am

Hi Pratosha,
For avdec_mpeg4, you can use xvimagesink

Pratosha · January 4, 2018, 6:44am

Thanks for your response DaneLLL.
I did replace nveglglessink with xvimagesink.

No change in the output, cursor still blinks.

DaneLLL · January 4, 2018, 12:19pm

Hi,
Have you tried the h264 streaming? We have verified this case.

And have you tried the local playback?
There are examples in https://developer.nvidia.com/embedded/dlc/l4t-accelerated-gstreamer-guide-28-1

Pratosha · January 5, 2018, 11:09am

Hey,

Thanks for your response DaneLLL.
Yes the suggestion of working with h264 did work.

I referred to the link below for usage of commands:

I am able to establish the connection now but with a delay of 3 to 4 seconds.
The commands used are as follows:

UBUNTU TO TX2

UBUNTU:

CLIENT IP = 172.19.81.237
gst-launch-1.0 v4l2src device =/dev/video0 ! ‘video/x-raw, width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! x264enc bitrate=4000000 ! ‘video/x-h264, stream-format=(string)byte-stream’ ! h264parse ! rtph264pay mtu=1400 ! udpsink host=$CLIENT_IP port=5000 sync=false async=false

TX2:

gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! omxh264dec ! nvoverlaysink sync=false async=false -e

If there is any method to reduce the delay caused would be of great help :)

Thanks,
Pratosha