How to display 4 RTP Streams from Jetson Xavier NX to VLC Mosaic on another Windows PC?

Hello All,

Hope you are doing well.

Referencing the video on youtube about setting up vlc mosaic and we had a few issues around setting up the same

Youtube video linkVideo Mosaic With VLC - YouTube

Issue - Showing 4 RTP video feeds from 2 Jetsons on vlc using Mosaic grid

Scenario -

We have two Nvidia Jetson Xavier NX. Both of the jetsons are connected with 2 Intel Realsense D415’s each

Jetson 1 IP - 192.168.0.101
Jetson 2 IP - 192.168.0.102
Personal Computer IP address 192.168.0.100

Now, using detectnet utility on Jetson, we could turn on the video feed from the cameras and detect human beings using an object detection model [SSD-MobileNet]

The personal computer has the latest vlc installed.

The command that we have used to send the video from both the jetsons to the pc looks something like this

Jetson 1 - detectnet /dev/video2 --input-width=300 --input-height=300 --headless rtp://PERSONAL-COMPUTER-IP:10100
Jetson 1 - detectnet /dev/video5 --input-width=300 --input-height=300 --headless rtp://PERSONAL-COMPUTER-IP:10200

Jetson 2 - detectnet /dev/video2 --input-width=300 --input-height=300 --headless rtp://PERSONAL-COMPUTER-IP:10300
Jetson 2 - detectnet /dev/video5 --input-width=300 --input-height=300 --headless rtp://PERSONAL-COMPUTER-IP:10400

Considering vlc’s capability to read the .sdp file, we can successfully see one of the videos in vlc from the personal computer using the .sdp content below

c=IN IP4 192.168.0.100
m=video 10100 RTP/AVP 96
a=rtpmap:96 H264/90000

Changing the 10*00 port gives output for all the video streams received at the personal computer IP

Opening 4 vlc’s at the same time won’t be the practical solution, so we have considered a mosaic grid for achieving the same.

The expected grid is 2*2 grid where all 4 videos will be played

Referencing the .vlm file in the above Youtube video, this is how the .vlm file for our expected configuration looks like

new channel1 broadcast enabled
setup channel1 input “rtp://192.168.0.100:10100
setup channel1 output #duplicate{dst=mosaic-bridge{id=1,width=960,height=540},select=video}

new channel2 broadcast enabled
setup channel2 input “rtp://192.168.0.100:10200
setup channel2 output #duplicate{dst=mosaic-bridge{id=2,width=960,height=540},select=video}

new channel3 broadcast enabled
setup channel3 input “rtp://192.168.0.100:10300
setup channel3 output #duplicate{dst=mosaic-bridge{id=3,width=960,height=540},select=video}

new channel4 broadcast enabled
setup channel4 input “rtp://192.168.0.100:10400
setup channel4 output #duplicate{dst=mosaic-bridge{id=4,width=960,height=540},select=video}

new mosaic broadcast enabled
setup mosaic input “file:///C:/Users/foo/bar/bg.jpg”
setup mosaic option image-duration=-1
setup mosaic output #transcode{sfilter=mosaic{width=1920,height=1080,cols=2,rows=2,order=“1,2,3,4”,keep-aspect-ratio=enabled,keep-picture=1,mosaic-align=5},vcodec=mp4v}:bridge-in:display

#setup mosaic output #transcode{sfilter=mosaic{width=1920,height=1080,cols=2,rows=2,order=“1,2,3,4”,keep-aspect-ratio=enabled,keep-picture=1,mosaic-align=5},vcodec=mp4v}:bridge-in:rtp{mux=ts,dst=192.168.0.255,port=10000,sap,name=testing}

control mosaic play
control channel1 play
control channel2 play
control channel3 play
control channel4 play

and this is the command that we put in personal computer PowerShell referencing the .vlm

vlc --vlm-conf demo.vlm

When we execute this command, vlc media player opens but nothing is played. Also, we tried to do this

But still, no stream is loaded. The personal computer firewall is disabled [Windows 10]

What can we do to achieve the expected scenario? Please let us know if we could provide any more information.

Hi,
This is an advanced function and we don’t have much experience about it. Would suggest go to vlc forum: https://forum.videolan.org/

Generally we run gstreamer commands on host PC. Can decode the streams into separate display windows, or composite into a video plane through compositor plugin.

Hi @DaneLLL,

In case of gstreamer how to display multiple streams from both Jetsons with RTP protocol in a single 4*4 grid on different machine other than Jetson using compositor plugin?

Can you please also share examples commands related to gstreamer for displaying in a single 4*4 grid with reference to above scenario?

Hi,
Please refer to the gstreamer command:

gst-launch-1.0 compositor name=cp sink_1::xpos=640 sink_2::ypos=360 name=cp sink_3::xpos=640 sink_3::ypos=360 ! xvimagesink rtspsrc location=IPADDR_0 ! rtph264depay ! h264parse ! avdec_h264 ! videoscale ! video/x-raw,width=640,height=360 ! cp. rtspsrc location=IPADDR_1 ! rtph264depay ! h264parse ! avdec_h264 ! videoscale ! video/x-raw,width=640,height=360 ! cp. rtspsrc location=IPADDR_2 ! rtph264depay ! h264parse ! avdec_h264 ! videoscale ! video/x-raw,width=640,height=360 ! cp. rtspsrc location=IPADDR_3 ! rtph264depay ! h264parse ! avdec_h264 ! videoscale ! video/x-raw,width=640,height=360 ! cp.

It displays 4 sources in 2x2 grid.

Hi @DaneLLL

We have tested the gstreamer command with above reference

gst-launch-1.0 compositor name=cp sink_1::xpos=640 sink_2::ypos=360 name=cp sink_3::xpos=640 sink_3::ypos=360 ! xvimagesink rtspsrc location=rtp://192.168.0.101:10300 ! rtph264depay ! h264parse ! avdec_h264 ! videoscale ! video/x-raw,width=640,height=360 ! cp. rtspsrc location=rtp://192.168.0.101:10400 ! rtph264depay ! h264parse ! avdec_h264 ! videoscale ! video/x-raw,width=640,height=360 ! cp.

But we were getting this output i.e No valid RTSP URL was provided

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (open) Opening Stream
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc1: Resource not found.
Additional debug info:
gstrtspsrc.c(7460): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc1:
No valid RTSP URL was provided
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

How to use RTP protocol URL such as rtp://192.168.0.105:1234 with GStreamer Compositor plugin?

Hi,
The error shows URI is invalid. Suggest you check the URIs are good by running

$ gst-launch-1.0 rtspsrc location=__VALID_URI__ ! rtph264depay ! h264parse ! avdec_h264 ! xvimagesink

Hello @DaneLLL

We were displaying RTP streams by referencing the below link:

Here it is mentioned that it only supports RTP when we want to send the stream to the remote system.

How to display 4 RTP streams in the 4*4 grid using the Gstremer Compositor plugin?

Hi,
You would need to have a valid URI to run this command successfully:

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.