Streaming's latency

Hi,
I have 2 pipelines to display video streamed from a camera like below:

  • Server (on Jetson):
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw' ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=NV12" ! nvv4l2h265enc  control-rate=1 insert-sps-pps=1 idrinterval=15 maxperf-enable=1 bitrate=4000000 ! rtph265pay ! udpsink host={CLIENT_IP} port=8554 sync=false
  • Client (on laptop):
gst-launch-1.0 udpsrc port=8554 ! application/x-rtp, media=video, clock-rate=90000, encoding-name="H265", payload=96 ! rtph265depay ! avdec_h265 ! xvimagesink sync=false

I see that the latency is almost 300ms. I measure by displaying a timer video and using my phone to record and capture the frame that shows the gap between the video and the stream.
Server and Client are connected via LAN.
Q: Is this latency normal ? Could I reduce it ? For more information, my laptop doesnt have GPU so I cant use NVIDIA decoder.
Thank you for your help.

Hi,
The latency may be from network and source. Please try this setup:
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

It network bandwidth is enough, you should see neglectable latency in the setup. So the setup can show latency in network. And then please try with v4l2src plugin to identify latency from the camera source.

1 Like

Thank you for your support.
I’m quite sure the network’s bandwidth is enough. It is ~93Mbps, which is measured by iperf.
I’ll find a way to use a monitor for my Jetson and see the latency when using videotestsrc.
The idea is to compare two latency values measured by using videotestsrc and v4l2src to identify if my latency issue comes from the camera source, isn’t it?
Are there solutions to help reduce the latency if the issue is from my camera?

Hi,
If your camera is connected to CSI port(s), you can enable the engines at maximum clock:
Jetson/l4t/Camera BringUp - eLinux.org

For further enhancement, you may need to consult vendor of the camera.

1 Like

For more information, I test execution time of some pipelines like below:

Send buffer using UDP:

  • Using videotestsrc:
gst-launch-1.0 videotestsrc is-live=1 do-timestamp=true num-buffers=300 ! video/x-raw,width=1920,height=1080 ! timeoverlay valignment=4 halignment=1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1920,height=1080' ! nvv4l2h264enc insert-sps-pps=1 idrinterval=15 ! h264parse ! rtph264pay ! udpsink host=CLIENT_IP port=5000 sync=0

image

  • Using v4l2src:
gst-launch-1.0 v4l2src device=/dev/video0 do-timestamp=true num-buffers=300 ! 'video/x-raw' ! timeoverlay valignment=4 halignment=1 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2h264enc insert-sps-pps=1 idrinterval=15 ! h264parse ! rtph264pay ! udpsink host=CLIENT_IP port=5000 sync=0

image

Using v4l2src has ~350ms execution time longer than using videotestsrc.

But if just simplify the pipeline using fakesink:

  • Using videotestsrc:
gst-launch-1.0 videotestsrc is-live=1 do-timestamp=true num-buffers=300 ! 'video/x-raw, width=1920, height=1080, framerate=30/1' ! fakesink

image

  • Using v4l2src:
gst-launch-1.0 v4l2src device=/dev/video0 do-timestamp=true num-buffers=300 ! 'video/x-raw, format=UYVY, width=1920, height=1080, framerate=30/1' ! fakesink

image

Using videotestsrc has ~5s execution time longer than using v4l2src.

Does this circumstance make sense ?

Thank you, I will try to enable the engines at maximum clock as you said.

Hi,
Maybe the v4l2 source is in 60fps. You can use fpsdisplaysink to show frame rate:

... ! ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

Thank you.
You’re right that the v4l2 source is in 60fps, although I set “framerate=30/1” for v4l2src.
After setting “framerate=60/1” for videotestsrc, its execution time is less than v4l2src’s. In the pipeline with fakesink, the gap is too small, it is about 40ms. But in the udpsink one, v4l2src’s exe time is 3.5s longer than videotestsrc’s.

Hi @DaneLLL, can I enable the engines at maximum clock by referring to this Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL ?
And how to return the default settings ?

Hi,
The steps enable VIC engine at maximum clock. You can enable it for a try. And can run sudo jetson_clocks to run CPU cores at maximum clock.

Would also suggest consult with vendor of the camera

Hi,
I could reboot Jetson to return default settings for both VIC and jetson_clocks, couldn’t I?

Hi,
Yes, it falls back to default mode after rebooting.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.