Need help to display a video stream with Gstreamer

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
,
1

Hi, I have a camera sending video stream to my NX devkit through multicast channel. I extracted the format of the video stream as following:

Analyzing udp://224.100.10.10:3001
Opening in BLOCKING MODE 
Done discovering udp://224.100.10.10:3001

Topology:
  container: video/mpegts, systemstream=(boolean)true, packetsize=(int)188
    video: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)baseline, level=(string)3
      Tags:
        video codec: H.264
      
      Codec:
        video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)baseline, level=(string)3
      Additional info:
        None
      Stream ID: d373b570d63985f2f90c9655bb22aa695cf476072a2c255d0fd6a9501fa65c1d:1/00000011
      Width: 640
      Height: 480
      Depth: 24
      Frame rate: 30/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

Properties:
  Duration: 99:99:99.999999999
  Seekable: no
  Live: yes
  Tags: 
      video codec: H.264

I want to use gst-lunch-1.0 to display the video, and tried different pipelines, all failed.

gst-launch-1.0 udpsrc port=3001 auto-multicast=true multicast-group="224.100.10.10" ! "application/x-rtp,encoding-name=H264,paylod=96" ! rtph264depay ! avdec_h264 ! autovideosink

gst-launch-1.0 udpsrc port=3001 auto-multicast=true multicast-group="224.100.10.10" ! tsparse ! autovideosink

Anyone can provide a help ? Appreciated.

2

It looks to be MPEG2TS over RTP, so you would try something like:

gst-launch-1.0 -v udpsrc multicast-group=224.100.10.10 port=3001 ! application/x-rtp,media=video,encoding-name=MP2T,clock-rate=90000, payload=33 ! rtpjitterbuffer latency=300 ! rtpmp2tdepay ! decodebin ! autovideosink

# or even simpler
gst-launch-1.0 -v uridecodebin uri=udp://224.100.10.10:3001 ! autovideosink
3

@Honey_Patouceul Thank you very much for the help. I tried both commands. Both of them run no error. But no video display window either. Did I miss anything?

procerus@procerus-desktop:~$ gst-launch-1.0 -v uridecodebin uri=udp://224.100.10.10:3001 ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: source = "\(GstUDPSrc\)\ source"
Setting pipeline to PLAYING ...
New clock: GstSystemClock


procerus@procerus-desktop:~$ gst-launch-1.0 -v udpsrc multicast-group=224.100.10.10 port=3001 ! application/x-rtp,media=video,encoding-name=MP2T,clock-rate=90000, payload=33 ! rtpjitterbuffer latency=300 ! rtpmp2tdepay ! decodebin ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

I also tried to start the command first and them start the camera. The results were same.

4

@Honey_Patouceul, I got some more information. The device with the camera was not reachable through the default gateway. After fixed the routing table, the gst-launch command can pick up the udp packets but resulted in decode errors. I am uploading the terminal output. What should I do next?
Thanks.
gst-output-200 (14.8 KB)

5

Looking at your logs, it seems it didn’t connect… Have the camera connected first, and if possible check on another host on same LAN that you can read the camera.

It may take some time to set up, depending on how frequently the source sends SPS/PPS and intra frames.
Also, I am assuming that your are locally logged in, with a monitor attached to Jetson (remote would need a different pipeline).

My assumption of MP2T over RTP may not be correct, re-looking at your logs, just use uridecodebin for now.

6

@Honey_Patouceul Thank you for your advice. I got video display with uridecodebin, but slow and bad quality. The device is a NX devkit and a monitor is directly connected to its HDMI port. The terminal output is captured and I am uploading it.
gst-output-uridecodebin (7.7 KB)

What should I do to tune the video quality?
Appreciated.

7

Seems it failed to keep sync…You may try buffering:

gst-launch-1.0 -v uridecodebin uri=udp://224.100.10.10:3001 use-buffering=true ! autovideosink

# Or try disabling sync
gst-launch-1.0 -v uridecodebin uri=udp://224.100.10.10:3001 ! autovideosink sync=false

You can also boost clocks with jetson_clocks, but this should be no problem for a NX.
I’d rather suspect a network loss with UDP. If your camera supports RTP it might be worth trying it.

8

I used the sync=false and use-buffering=true options, separately and together. I got similar results. To simplify the problem I changed the video stream to a unicast address. I also tried to use VLC to play the stream. On one Windows system, the VLC plays the video normally. On the NX, the VLC crashed:

procerus@procerus-desktop:~$ vlc
VLC media player 3.0.8 Vetinari (revision 3.0.8-0-gf350b6b5a7)
[00000055a4303580] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
[00000055a4306c80] main playlist: playlist is empty
[0000007f6c001110] ts demux error: can't attach SI SDT table handleron already in used pid 17 (Not DVB ?)
Bus error (core dumped)

PID 17 looks like a kernel thread handling soft interrupt…

procerus@procerus-desktop:~$ ps -ef |grep 17
root        17     2  0 16:19 ?        00:00:00 [ksoftirqd/1]

This happens even after a fresh boot.
The CPU load was low during the gst display. I am uploading the terminal output from the new runs.
gst-output-uridecodebin (7.7 KB)
uribin-out-noSync (7.6 KB)

Any other options should I try?
Thanks a lot.

9

For using VLC on Jetson, you would try this.

Without sync, I don’t see errors. Low quality may be due to network loss.

You may also try using ffmpeg that has ability for guessing and see if it gives further details (assuming you have a GUI running):

ffmpeg -hide_banner -loglevel debug  -i udp://224.100.10.10:3001 -f xv display
10

@Honey_Patouceul After removing the libomxil_plugin the vlc does not crash anymore but no video display either. ffmmpeg does not display the video either, but generating much output. I captured the output and upload it for your reference.

Thanks a lot.
ffmpeg-output (25.4 KB)

11

For VLC, I don’t think it won’t further help. Just shared the link for general usage.

For FFMPEG, do you have a X-based GUI running locally (logging in with local keyboard and mouse and display) ? No VNC or remote ?

You may also better detail your case (L4T release, camera model, network such as wired ethernet LAN or WiFi…).

I might not be able to further help. Someone more skilled may better answer.

12

I do have the X-based GUI running locally. The NX is directly connected to the camera with Ethernet cable. We have different cameras. All of the cameras use the same control board so the outputted video streams are in same format/encode. FOr the NX, I am using JetPack 4.6, L4T 32.6.1.
Thanks.

13

Can you try only one camera ?

14

Only one camera can be attached at one time. Sorry for the confusion.

16

Hi,
Please try

gst-launch-1.0 uridecodebin uri=udp://224.100.10.10:3001 ! nvoverlaysink sync=0

And check if you can see good vide preview. And can set this environment:

$ export GST_DEBUG=FACTORY:4

To list what elements are picked by uridecodebin.

18

@DaneLLL
Following your advice, I still can not get better video output. The gst-launch output is attached. Thanks.
gst-out.txt (5.9 KB)

20

Hi,
Probably the network is not stable so the stream is not well received. You may try UDP streaming for comparison:
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

We can run this smoothly and see good video preview. Please check if it works well in your environment.

22

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.