Can we use both object detection and semantic segmentation simultaneously in deepstream

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

env:deepstream6.3
q:primary model is detect,second model is segmentation,Can we use both object detection and semantic segmentation simultaneously

Do you mean you want to get bboxes and the segment masks for each objects detected by the detector?

yes,segment config is
image
but not effective.
the pipeline is

What do you mean by “no effective”?

segvisual does not support NvDsInferSegmentationMeta in the object meta. So you can’t see the mask display in this way. The segment model can output class map to identify all classes in one image. It is a waste to use segment model to just segment one object by one object.

It is better to use the instance segmentation model to get the bboxes and segment masks together. NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream (github.com)

The segmentation result displayed in the segmentation src pad probe function is incorrect,don’t really care about the display effect,but the separate segmentation pipeline results are correct.My segmentation model is semantic segmentation。
NvDsInferSegmentationMeta cannot retrieve from frame_ User_ Meta_ Obtained from the list。

Since you have set the segment model nvinfer configuration as SGIE, please get the NvDsInferSegmentationMeta user meta from object meta instead of from frame meta.

thank you so much !!!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.