RTSP stream delay in deepstream-app

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 6.0.1
• JetPack Version (valid for Jetson only) 4.6.1
• TensorRT Version 8.2.1+cuda10.2
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I have a question about deepstream-app with rtsp stream input.

When I use an rtsp server and source an mp4 file, it takes some time for the stream to pass through the decoder bin in deepstream-app.
The amount of time varies run to run, sometimes over 20 sec.

Here is an example of the terminal output.
out.txt (3.2 KB)
deepstream_source_bin.c is modifed to count and print the number of frames at rtsp_src and decodebin every 1 sec.
Because I use a video with 5 fps, the log shows there is about 17 sec delay.

Here is data to reproduce the issue.
240418_data.zip (19.7 MB)
Please note that rtsp server is built on another PC by mediamtx and that I use a sample video “sample_1080p_h264.mp4” with the fps converted to 5.

Is this an expected behavior of deepstream-app?
I would appreciate it if you could check.

1 Like
3

It may be that data has not been consumed since then, resulting in no data from the decoderbin. But the rtsp can buffer data, so there will be a gap.
Could you try to adjust the value of the latency for source group and see if it makes a differece?

4

Thank you for the reply!
I’ve tried Solution 11 in the following link as well as your suggestion, but the initial stream delay still happens.
(If “latency” is too large, the stream will not be received after some time.)
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_troubleshooting.html#:~:text=starts%20dropping%20frames.-,Solution%2011%3A,-For%20RTSP%20streaming

5

The solution is for the stream to be smooth. It will increase the latency. So you need to do the opposite to decrease the latency. But it will cause the stream to be unsmooth.

6

Thank you, but lowering “latency” does not change the result.

7

Could you try to use the fakesink to see if there’s any improvement?
Also, do not set drop-on-latency to false according to the solution 11.

8

Sorry for the late reply.
Fakesink has been used throughout in my experiment.
Adjusting “latency” has been done also in the case of “drop-on-latency” set to TRUE.

9

I tried that on my side with DeepStream 6.4 and ffplay Build rtsp server.

+++ Received frame count rtsp_src:19 decodebin:3
+++ Received frame count rtsp_src:38 decodebin:3
+++ Received frame count rtsp_src:52 decodebin:51
+++ Received frame count rtsp_src:81 decodebin:80
**PERF:  47.59 (47.26)
+++ Received frame count rtsp_src:112 decodebin:111
+++ Received frame count rtsp_src:141 decodebin:140
+++ Received frame count rtsp_src:170 decodebin:169
+++ Received frame count rtsp_src:201 decodebin:200
+++ Received frame count rtsp_src:231 decodebin:230
**PERF:  29.91 (35.44)
+++ Received frame count rtsp_src:261 decodebin:260
+++ Received frame count rtsp_src:290 decodebin:289
+++ Received frame count rtsp_src:320 decodebin:319
+++ Received frame count rtsp_src:351 decodebin:350

It seems to have something to do with how the server generates the stream. Do you have a real rtsp source and try that?

10

Thank you for checking!
If an rtsp source is a real camera, the delay is much reduced as follows

+++ Received frame count rtsp_src:0 decodebin:0
** INFO: <bus_callback:194>: Pipeline ready
**PERF: FPS 0 (Avg)
**PERF: 0.00 (0.00)
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 279
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 279
+++ Received frame count rtsp_src:6 decodebin:0
** INFO: <bus_callback:180>: Pipeline running
+++ Received frame count rtsp_src:16 decodebin:13
+++ Received frame count rtsp_src:26 decodebin:23
+++ Received frame count rtsp_src:36 decodebin:33
+++ Received frame count rtsp_src:46 decodebin:43
**PERF: 10.89 (10.76)
+++ Received frame count rtsp_src:56 decodebin:53
+++ Received frame count rtsp_src:66 decodebin:63
+++ Received frame count rtsp_src:76 decodebin:73
+++ Received frame count rtsp_src:86 decodebin:83
+++ Received frame count rtsp_src:96 decodebin:93
**PERF: 10.00 (10.35)

As for your log showing almost no delay, did you use ffplay with an mp4 file or a real rtsp source?

11

Both. I use the ffplay with our sample_720.mp4 sample video and a real camera rtsp source. It both works well.

One reason I can think of for the delay in this place might be the I frame interval of your rtsp source is too big. When the source is connected, a large number of frames are not I-frames. The decoder needs to drop the Non-I frame.

12

I haven’t been able to reproduce your result for “sample_720p.mp4” on my side.
But I found that using “sample_1080p_h264.mp4” with the I frame interval set to be small results in almost no delay.
Your suggested reason seems to be correct! Thank you very much for your help!

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.