get secondary inference model output in deepstream_test_2

hi,

i want to do cascaded inference according to test app 2 from deepstream.

i have a yolo vehicle detector as primary inference(which is working fine in deepstream) and then i want to feed the cropped vehicles to the secondary model to get the license plates.

i have two questions regarding the above scenario.

1- does the pipeline automatically crop the output objects from the primary inference and feed it as input to the secondary inference model?

2- where do i find secondary model output?

my goal is to get cropped license plates that i can display with openCV or save locally.

thanks

1- does the pipeline automatically crop the output objects from the primary inference and feed it as input to the secondary inference model?

Yes.

2- where do i find secondary model output?

In metadata. gstnvinfer.cpp → attach_metadata_classifier()
Here has nvinfer process: https://devtalk.nvidia.com/default/topic/1061492/deepstream-sdk/deepstream-sdk-faq/

my goal is to get cropped license plates that i can display with openCV or save locally.
You can refer to dsexample plugin sample. → sources/gst-plugins/gst-dsexample

thanks for you response

my secondary model is a yolo detector not a classifier. will the attach_metadata_classifier() include cropped license plate and other meta data just like the primary model meta?

Here is back-to-back detector sample: GitHub - NVIDIA-AI-IOT/deepstream_reference_apps: Samples for TensorRT/Deepstream for Tesla & Jetson

1 Like

this was very helpful! i was able to adapt above sample to my yolo models.

now i have a tracker for the vehicle detector(primary) in my pipeline and i am wondering weather a correlation between the license plate and its respective vehicle through tracker ID is made automatically.

i have also added dsexample plugin to my piple line, but i have two issues regarding that.

1- on the display i always have two bboxes called obj0 and Obj1

2- i can only save full frames with opencv, but i want to access nvbufsurface of secondary model and save cropped license plates as well.

thanks

any updates on this?

thanks

correlation between the license plate and its respective vehicle through tracker ID

Here is metadata structure. I think you can get tracker ID from objectMeta which license plate belongs.
https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream_Plugin_Manual%2Fdeepstream_plugin_metadata.03.2.html%23wwconnect_header

NvDsObjectMeta → object_id is the tracking ID

save cropped license plates as well

https://devtalk.nvidia.com/default/topic/1061492/deepstream-sdk/deepstream-sdk-faq/ → 2. Sample of customizing gst-dsexample: