The delay increases as the output video over time

Hi

Using jetpack 4.6’s “jetson-inferance” posenet.py
I want to process the input video 4ch.

  • Environment
    ・ Input video is delivered by RTSP, and the frame rate is 15fps.
  • Command
posenet.py --input-codec = h264 --network = resnet18-body rtsp: //192.1.10.152:12345

However, if you increase the resolution of the input video delivered by RTSP, the delay of the output video may or may not increase over time.
ex1): Delay does not increase for resolution 640x480
ex2): For a resolution of 1920x1280, the delay increases over time.

Why?

Hi,

To narrow down the delay comes from, could you test the two experiments with the below sample as well?

Thanks.

Hello Asta LLL

I did two tests with “video-viewer.py”.
The result was the same.
Since the result is the same for “video-viewer.py”, the processing speed of “videoSource.Capture ()” of “video-viewer.py” is slower than the frame rate delivered by rtsp, and the frame is “videoSource.Capture ()”. I think it’s in the buffer, what about?

If you are running this as 4x independent posenet.py processes, if you want to keep 1920x1080 resolution you may want to look to using DeepStream which is more optimized for handling multiple streams. As you have found, you could also try reducing the RTSP resolution, which often is okay since the DNN models run at a low resolution anyways (IIRC the pose estimation models are 300x300)

If you haven’t already, you could also try starting posenet with --input-rtsp-latency=0
It may also help adding rtpjitterbuffer element to the RTSP pipeline, I’m not entirely sure: https://github.com/dusty-nv/jetson-utils/blob/56d57f6713a5afe9029ab768ec5d13c285a52dfb/codec/gstDecoder.cpp#L528

Hi dusty_nv

Thanks for your comment
I will try “rtpjitterbuffer” as well.