Run sqie if necessary

Hi! I’m using deepstream for license plate detection.

First of all, I run pgie(resnet10) for car detection and then run yolov3_tiny as sgie for LP detect.
How can i run sgie if only car is in roi?

Hi,

You can update the roi-top-offset and roi-bottom-offset value in the configure file.
Please check this document for the usage in detail:
https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.01.html#wwpID0E0OFB0HA

Thanks.

Hi, AastaLLL!
Thanks for reply!
I’ve tried to set different roi-bottom-offset and roi-top-offset, but it doesn’t work.
For example, I have the video resolution is 2448x2048 px and when i set roi-bottom-offset is 2440, the sgie is still detecting license plate in all regions of image

this is the yolov3_tiny configure file:
[property]
gpu-id=0
process-mode=2
net-scale-factor=1
#0=RGB, 1=BGR
model-color-format=0
custom-network-config=./tiny/yolov3-tiny.cfg
model-file=./tiny/yolov3-tiny.weights
#model-engine-file=./tiny/model_b1_fp32.engine
model-engine-file=model_b16_fp16.engine
labelfile-path=./tiny/labels.txt
#0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
batch-size=16
num-detected-classes=1
gie-unique-id=2
is-classifier=0
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV3Tiny
custom-lib-path=tiny/libnvdsinfer_custom_impl_Yolo.so
#classifier-async-mode=1
#roi-top-offset=2000
roi-bottom-offset=2000

and I want to run sgie only once for each object_id in pgie.

Hi,

It should be declared in the [class-attrs-<class-id>].
For example:

[class-attrs-2]
roi-bottomp-offset=400

You can find some example in /opt/nvidia/deepstream/deepstream-[ver]/samples/configs/deepstream-app/config_infer_primary.txt.

Thanks.

Thanks!
One more question, can I run sgie only once for each object_id ?
For exaple, when I run original back_to_back_detector on TX2 and Busler camera, it’s very slowly( ~1 fps). As a sgie with yolov3_tiny it works faster, but this isn’t enough too.

Hi,

YES. You can use the interval configure.
This parameter indicates the frequency to apply inference.

For example:

[property]
...
interval=5

This imply do the inference every 5 frames.
To set this number into a large value should meet your requirement.

hanks.

I want to create an lp detector that works in real time, but the traffic of the cars is very intensive.
My pipleline pylonsrc-> nvvideoconvert-> capsfilter-> nvstreammux-> pgie-> nvtracker-> sgie-> nvvideoconvert-> nvdsosd-> queue-> sink. Perhaps there are some ways to speed up the pipeline?
I also have a working example, in which the car detects in DS, and lp detects only once for each car in a Python script.
I want to move on the detection that happens in Python into the DS pipeline. Is there such a possibility?

Hi,

Yes.
In deepstream, you can choose the frequency to apply the sgie (the interval value mentioned above).
It indicates that you will apply the inference just once in a given period.

So suppose each car captured by the camera around 100 frames in average, to set the interval to 100 or larger should meet your requirement.

Thanks.

Hi, @AastaLLL!
Thank you a lot for answers!

I’ve tried everything you said but the pipeline is still slow.
I’m set interval=2 and roi-top-offset=250 to pgie, interval=100 to sgie.

But I still have a few questions:

  1. All the same, is there a way to run sgie only once for each object_id in pgie detection?

  2. I noticed that the coordinates in roi-bottom-offset and roi-top-offset need to be multiplied by 2. Because with a camera extension of 2500x2000, with a roi-top-offset=250 in pgie, cars are not detected at the top quarter of the frame.

  3. If I set roi-bottom-offset or roi-top-offset is sgie runs anyway not even in the roi? And Is It filtred by coordinates then? Does it start depending on the coordinates of the pgie detection?

  4. Can I use classifier-async-mode for sgie detector? Or Is it for classification only?

  5. How can I write gst-plugin for yolo inference on my own?

  6. If I add queue in the pipeline, will it improve capacity? If so, where is it better to be added?

any update?

Hi,

Thanks for your question.

I’m going to check this with our internal team.
Will update to you later.

Hi,

  1. sgie always infers on each object only once, why do you think its being done multiple times ?

  2. If you want to filter out car in specific ROI region then you can use roi-top-offset and roi-bottom-offset at the pgie for the car class. This offset is with respect to the output resolution of the streammux and not the camera resolution.

  3. See answer in 2.

  4. It is for classification only

  5. May i know why you want to write a new plugin when nvinfer plugin already supports yolo inference for yolov2/v3/tiny models ?

  6. You can try adding a queue in between all the adjacent elements and see if it helps. Although in your case, since you are using multiple detectors on TX2, i suspect if your GPU is saturated. You can check this via tegrastats tool. If your GPU is saturated, i would suggest increasing the intervals of the detectors to an acceptable value. You can also try using a overlaysink instead of an eglsink since its lightweight.