Delay due to nvinfer

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

When I set the interval to 0 on my deepstream pipeline, I start to face issues, where the stream starts to face delays which increases with time.
But on the other hand when i set it to a higher number for eg. 100, I see no such delay.

Is there any optimisations techniques or parameters I should focus on tuning?

I am running the script on:
NVIDIA Jetson Xavier NX - AVerMedia NX215 - Jetpack 5.1

And I am using an engine file for the model.

2

What kind of model are you running?

3

we are using a model trained from yolov5

4

This is the config file we are using

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
labelfile-path=path to label.txt
model-engine-file=path to engine file
parse-bbox-func-name=NvDsInferParseCustomYoloV5
custom-lib-path=/home/nvidia/triton-deepstream/yolo_deepstream/deepstream_yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
maintain-aspect-ratio=1
symmetric-padding=1
infer-dims=3;640;640
batch-size=2
network-mode=2
num-detected-classes=1
interval=100
gie-unique-id=1

#Use the config params below for NMS clustering mode
[class-attrs-all]
topk=300
nms-iou-threshold=0.5
pre-cluster-threshold=0.1
5

Hey, @DaneLLL or @junshengy is there something I am missing?

6

It simply means your inference FPS is lower than stream FPS. There’s not much you can do about it other than increasing intervals or dropping frames (using drop-frame-interval)

7

Are there any starting value you might recommend, because interval is already set to 100

8

interval shouldn’t be that high. Make it 2 or 3. drop-frame-interval to 5 which would make a 30FPS stream into 6FPS, only keeping every 5th frame

10 
sudo pip3 install jetson-stats
jtop

1.Please check the GPU usage through jtop first.

2.It seems that you are not using int8 models, which can reduce GPU usage

Refer to the FAQ below to measure latency of every element. When measuring, please use local files to avoid the impact of network latency.

11

I tested different settings on my DeepStream pipeline and monitored system performance using jtop:

  1. Drop Frame: Off, Interval: Off

    • Temperature: GPU reached 70°C, CPU reached around 65°C, with a slight increase over time.
    • Delay: After running for 5 minutes, the delay was around 2 minute which increases with time.

  2. Drop Frame: On, Interval: Off

    • Temperature: GPU at 69°C, CPU at 65°C.
    • Delay: After running for 5 minutes, the delay was around 1 minute which increases with time.

  3. Drop Frame: On, Interval: On

    • Temperature: Both GPU and CPU stayed below 50°C.
    • Delay: Reduced to a constant 5–6 seconds delay.

Drop Frame Interval = 4
Interval = 4

P.S.- I have 2 sources that are sending me data continuously and both of them are rtsp streams.

12

Is it a nano model, small model or bigger?

13

GPU temperature is not important, please check if GPU usage is too high without network impact.
However, even if the GPU usage is very high, the delay will not reach 2 minutes.

If I use playbin vs deepstream pipeline on the same device, are the latencies similar?

 gst-launch-1.0 playbin uri="