Deepstream 6.3 can i use one detector to detect car, one detector to detect plate and detector (ocr) to get plate number?

[zhoutianyou00]

Please provide complete information as applicable to your setup.

**• GPU A10
**• deepstream 6.3
**• TensorRT Version 8.6
**• NVIDIA GPU Driver Version 530
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

deepstream pipeline :
streammux->nvvideoconvert->nvinfer(pgie)->nvinfer(sgie plate detect)->nvinfer(sgie ocr)->streamdemux

sgie operator on pgie, ocr operator on sgie(plate )

problem:
car number is putted in user meta.

I know ocr run, it out put car number, but I can’t get car number in custom gst buffer.

pgie config:
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=…/weights/yolo_nas_s_coco.onnx
model-engine-file=…/weights/yolo_nas_s_coco_fp16.engine

int8-calib-file=calib.table

labelfile-path=…/…/labels/coco_labels.txt
batch-size=16
network-mode=2
num-detected-classes=80
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=1
symmetric-padding=0

force-implicit-batch-dim=1

workspace-size=1000

parse-bbox-func-name=NvDsInferParseYoloE

parse-bbox-func-name=NvDsInferParseYoloECuda

custom-lib-path=…/nvdsinfer_custom_impl/libnvdsinfer_custom_impl.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

sgie detect car plate config:
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#model-color-format=0
model-engine-file=…/weights/yolov5s_lp.engine
network-mode=2
#network-input-order=0
#symmetric-padding=1
#maintain-aspect-ratio=1
labelfile-path=…/…/labels/lp_labels.txt
#force-implicit-batch-dim=1
batch-size=16
num-detected-classes=1
interval=0
gie-unique-id=2
operate-on-gie-id=1
process-mode=2
network-type=0
cluster-mode=4
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=…/nvdsinfer_custom_impl/libnvdsinfer_custom_impl.so

sgie detector car number config:

[property]
gpu-id=0

net-scale-factor=0.0039215697906911373

net-scale-factor=0.00784313725490196
offsets=127.5;127.5;127.5
model-color-format=0
model-engine-file=…/weights/paddleocr_v3_lp_fp16.engine
labelfile-path=…/…/labels/ocr_labels.txt
network-mode=2
network-input-order=0
symmetric-padding=1
maintain-aspect-ratio=1
#force-implicit-batch-dim=1
batch-size=1
num-detected-classes=1
interval=0
gie-unique-id=3
operate-on-gie-id=2
process-mode=2
#network-type=0
network-type=0
cluster-mode=4
parse-bbox-func-name=NvDsInferParsePaddleocr
custom-lib-path=…/nvdsinfer_custom_impl/libnvdsinfer_custom_impl.so
[class-attrs-all]
pre-cluster-threshold=0.0

[Fiona.Chen]

@zhoutianyou00

I know ocr run, it out put car number, but I can’t get car number in custom gst buffer.

You can attach the car number to the object meta label just as the sample deepstream_lpr_app/nvinfer_custom_lpr_parser at master · NVIDIA-AI-IOT/deepstream_lpr_app (github.com)

Or you can customize a new object user meta to contain the car number. There is a sample of object user meta customization: deepstream_tao_apps/apps/tao_others/deepstream-emotion-app/emotion_impl/emotion_impl.cpp at master · NVIDIA-AI-IOT/deepstream_tao_apps (github.com)

[zhoutianyou00]

Yes，I do it. I can get pgie object info, sgie (plate) object info in python.
I can’t get ocr object detect info in python, it don’t output ocr obj detect info.

But if I add “output-tensor-meta=1” to ocr config, can get tensor data.

[Fiona.Chen]

@zhoutianyou00

I can’t get ocr object detect info in python, it don’t output ocr obj detect info.

Please check your code which adding the customized meta data to the NvDsMeta.

[Fiona.Chen]

Please refer to deepstream_python_apps/bindings/CUSTOMUSERMETAGUIDE.md at master · NVIDIA-AI-IOT/deepstream_python_apps (github.com)
and deepstream_python_apps/apps/deepstream-custom-binding-test at master · NVIDIA-AI-IOT/deepstream_python_apps (github.com)

[zhoutianyou00]

I add code to deepstream,in “attach_metadata_detector” function,

It work ok for face 5 key point.

It work for ocr plate number single.

[Fiona.Chen]

It is no difference to DeepStream what the data is inside the user meta. You need to check your implementation. Or you can refer to the link I send to you to implement the customizes user meta in c++ and implement the binding to use it. And please name your own user meta with a specific name but not always use “NVDS_GST_CUSTOM_META”, this name has been used by our sample.

[zhoutianyou00]

I wan’t to know if can sgie “operate-on-gie-id” sgie which is “operate-on-gie-id” on pgie?
all the pgie and sgie is detector.

[Fiona.Chen]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

@zhoutianyou00

I wan’t to know if can sgie “operate-on-gie-id” sgie which is “operate-on-gie-id” on pgie?

Any, depends on the relationship between your models. For your case, the plate detect sgie should operate on the car detect pgie, the ocr sgie should operate on the plate detect sgie.