How to use custom yolov4-tiny in deepstream6 with rtsp output?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU)
Jetson Orin
• DeepStream Version
6.2
• JetPack Version (valid for Jetson only)
5.0.2
• TensorRT Version
8.4.1-1+cuda11.4

Hello, I have a yolov4-tiny custom model that I would like to use with Deepstream 6 in python. So far I have followed this repo on github:

It is a very helpful repo and I succesfully converted my yolov4-tiny model into a .engine file and ran it in Deepstream using the command:

deepstream-app -c deepstream_app_config.txt

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=3
#uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
uri=file://../../../testVideos/3.mp4
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
type=2
sync=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[tests]
file-loop=0

config_infer_primary.txt below:

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
custom-network-config=yolov4-tiny-416.cfg
model-file=yolov4-tiny-416.weights
model-engine-file=model_b1_gpu0_fp32.engine
#int8-calib-file=calib.table
labelfile-path=labels.txt
batch-size=1
network-mode=0
num-detected-classes=1
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=0
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

The performance of it is 60FPS .

I need to write the app in python, so I used the config_infer_primary.txt file to run the model with the Deepstream sample python apps.
In deepstream_test_1.py the performance is 30FPS
in deepstream-imagedata-multistream-redaction the performance is 30FPS
I need a higher FPS for the RTSP stream.
How can I achieve higher FPS using my custom yolov4-tiny model in Jetson Orin?

4

DeepStream is just a SDK but not an application. To measure performance with DeppStream, please refer to Performance — DeepStream 6.1.1 Release documentation (nvidia.com)

deepstream_test_1.py and deepstream-imagedata-multistream-redaction use different pipeline and configurations with deepstream-app, they are impossible to have the same FPS as deepstream-app. It is no sense to compare different pipelines’ performance. To handle the video with bigger resolution will take more time than handling the smaller resolution videos. Nobody can tell you what is the speed if we don’t know how many things need to be handled.

Take deepstream_test_1.py as example, there is nvosd and nveglglessink in the pipeline, if you change the pipeline as the same as deepstream-app(without nvdsosd, replace nveglglesssink and nvegltransform with fakesink and sync=0) and use the same source(the same resolution and format), the performance may be similar.

1 Like
5

okay got it. So how could I modify deepstream-imagedata-multistream-redaction to achieve similar performance?

It uses udpsink to transmit the RTP packets over IP network.
By setting the sink property to False ‘sink.set_property(‘sync’, 0)’ I can achieve better performance but the FPS varies and goes up and down.

**PERF:  {'stream0': 63.91, 'stream1': 63.91, 'stream2': 63.91, 'stream3': 63.91} 

**PERF:  {'stream0': 49.93, 'stream1': 49.93, 'stream2': 49.93, 'stream3': 49.93} 

**PERF:  {'stream0': 35.96, 'stream1': 35.96, 'stream2': 35.96, 'stream3': 35.96} 

**PERF:  {'stream0': 53.93, 'stream1': 53.93, 'stream2': 53.93, 'stream3': 53.93} 

**PERF:  {'stream0': 61.92, 'stream1': 61.92, 'stream2': 61.92, 'stream3': 61.92} 

**PERF:  {'stream0': 43.95, 'stream1': 43.95, 'stream2': 43.95, 'stream3': 43.95}
6

No way. The network performance can not be changed by application code.

1 Like
7

It is caused by the network performance.

1 Like
8

Got it Thanks! I added two more secondary detectors to the pipeline.

print("Linking elements in the Pipeline \n")
    streammux.link(pgie)
    pgie.link(sgie1)
    sgie1.link(sgie2)
    sgie2.link(nvvidconv1)

pgie and sgie1 work just fine but when I add the third one I get the following error:

WARNING: Num classes mismatch. Configured: 37, detected by network: 1

I have tested the third detector by its own and works just fine, I dont get that error and it can detect all the objects. The error is coming from ‘nvdsparsebbox_Yolo.cpp’ I seems like it is reading the same number of classes of the first model. How do I fix this?

9

I fixed it by following the following instructions:

10

Glad to hear that! Thank you for sharing!

1 Like
11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.