Gstreamer sreaming video over wifi network frame droping and freezing issues

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
1

Hi, i am trying to stream video over the wifi network using gstreamer on the jetson nano but i am getting frame dropping issue on client side(android). the streaming frame stuck and freezing frequently , i am using this pipeline

jetson nano - {
import gi
gi.require_version(‘Gst’,‘1.0’)
gi.require_version(‘GstVideo’,‘1.0’)
gi.require_version(‘GstRtspServer’,‘1.0’)
from gi.repository import GObject, Gst, GstVideo, GstRtspServer

Gst.init(None)

mainloop = GObject.MainLoop()
server = GstRtspServer.RTSPServer()
mounts = server.get_mount_points()

factory = GstRtspServer.RTSPMediaFactory()

factory.set_launch(‘( v4l2src device=/dev/video1 ! video/x-h264, width=1920, height=1080, framerate=30/1, format=H264 ! nvv4l2decoder ! nvvidconv ! video/x-raw(memory:NVMM) ! nv4l2h264enc preset-level=1 MeasureEncoderLatency=1 maxperf-enable=true insert-vui=true insert-sps-pps=1 bitrate=10000000 control-rate=1 vbv-size=734000 ! h264parse ! rtph264pay name=pay0 pt=96 )’)

mounts.add_factory(“/video”, factory)
server.attach(None)

print(“stream ready at rtsp://127.0.0.1:8554/video”)
mainloop.run()

}
client side - gst-launch-1.0 rtspsrc location=rtsp://192.168.31.111:8554/video latency=200 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink sync=false

what is the best way to stream video from jetson to android without frame dropping and low latency?

2

For wifi use case, you may try disabling udp-multicast, or try tcp transport.
On receiver side:

gst-launch-1.0 rtspsrc location=rtsp://192.168.31.111:8554/video latency=200 protocols=udp ! queue ...

gst-launch-1.0 rtspsrc location=rtsp://192.168.31.111:8554/video latency=200 protocols=tcp ! queue ...
3

thank you for the reply, i tried tcp transport it minimalized frop dropping and not reduced , can you please make the pipeline on server side

4

On server side, you may first try just relaying without decode/encode:

factory.set_launch('( v4l2src device=/dev/video1 ! video/x-h264, width=1920, height=1080, framerate=30/1 ! h264parse ! rtph264pay name=pay0 )')
5

thanks for made pipeline! now can i streaming with low latency but i am getting frame glitches like this

Screenshot (183)
Screenshot (183)1920×1080 108 KB

Screenshot (185)
Screenshot (185)1920×1080 119 KB

6

Seems an I frame has been incompletely read… Did you got that with TCP transport ?
If yes, you may try this on sender side:

# Adding a queue
factory.set_launch('( v4l2src device=/dev/video1 ! video/x-h264, width=1920, height=1080, framerate=30/1 ! queue ! h264parse ! rtph264pay name=pay0 )')

# Trying io-mode mmap :
factory.set_launch('( v4l2src device=/dev/video1 io-mode=2 ! video/x-h264, width=1920, height=1080, framerate=30/1 ! h264parse ! rtph264pay name=pay0 )')

Be also sure to use a USB3 port (not through a USB2 hub). You may check with lsusb -t.

Or try tcpserversink:

factory.set_launch('( v4l2src device=/dev/video1 ! video/x-h264, width=1920, height=1080, framerate=30/1 ! queue ! h264parse ! mpegtsmux ! tcpserversink port=50001

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-h264, width=1920, height=1080, framerate=30/1 ! queue ! h264parse ! mpegtsmux ! tcpserversink port=50001

That you may receive with:

gst-launch-1.0 tcpclientsrc host=<jetson_IP> port=50001 ! queue ! tsdemux ! h264parse ! avdec_h264 ! autovideosink

# Or simply:
gst-play-1.0 tcp://<Jeston_IP>:50001
7

thanks for sharing , i tried all of commands but getting same thing happend :(

8

Sorry, I may not able to help much further.
You may try configuring the wifi network to another channel (for example, if using channel 4, try channel 1 or 7. Better use 5GHz).
You may use a wifi analyzer (you can easily find free ones for Linux) for checking signal.

  • Does it happen when receiver is localhost (the streaming jetson itself) ?
  • Does it happen with wired ethernet ?
  • Does it happen only when Jeston is far from Wifi receiver ?

Someone more skilled may better advise.

9

Hi, i am using 11th channel with 5GHZ

  • Does it happen when receiver is localhost (the streaming jetson itself) ?
    no i am trying to receiving on windows 11
  • Does it happen with wired ethernet ?
    No jetson and windows are connected in the wifi network
  • Does it happen only when Jeston is far from Wifi receiver ?
    both of in same place only

i am getting these log in windows cmd when using autovideosink
“…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video: A lot of buffers are being dropped.
Additional debug info:
…/libs/gst/base/gstbasesink.c(3145): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSink:autovideosink0-actual-sink-d3d11video:
There may be a timestamping problem, or this computer is too slow.
0:10:10.5 / 99:99:99.”

10

Sorry, re-reading this I realize I made an error. The server pipeline should not be given to rtsp server but directly to gst-launch. Try adding timestamp:

gst-launch-1.0 v4l2src device=/dev/video1 do-timestamp=true ! video/x-h264, width=1920, height=1080, framerate=30/1 ! queue ! h264parse ! mpegtsmux ! tcpserversink port=50001

You may also try flvmux/flvdemux or matroskamux/matroskademux instead of mpegtsmux/tsdemux. These would use H264 avc stream-format instead of byte-stream.

11

thanks for new pipeline, i getting problem in implementing tcpserver in the mobile app side, i would be helpful changes in rtspserver, i added timestamp in the rtsp server, i got improvement but sometimes getting frame skipping

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.