# Using yolo-pose and yolo-seg together in one python app

**URL:** <https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856>\
**Category:** DeepStream SDK\
**Created:** [August 19, 2024, 12:10pm UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856 "2024-08-19T12:10:44Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![md44](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@md44](https://forums.developer.nvidia.com/u/md44)\
**Post date:** [August 19, 2024, 12:10pm UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/1 "2024-08-19T12:10:44Z")

</div>

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU)** GPU  
**• DeepStream Version** 7.0  
**• JetPack Version (valid for Jetson only)**  
**• TensorRT Version**  
**• NVIDIA GPU Driver Version (valid for GPU only)**  
**• Issue Type( questions, new requirements, bugs)**  
**• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)**  
**• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)**

Hi,

I would like to use both yolo-seg and yolo-pose as stages before the tracker, is it possible?  
what exactly I need to set in the config files in order to do that?  
how do I access the mask in the prob function?  
I see that both pose and seg are using obj\_meta.mask\_params, so how to make each use its own?

thanks

---

<div class="post-metadata">

**Author:** ![junshengy](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/junshengy/32/251332_2.png) [@junshengy](https://forums.developer.nvidia.com/u/junshengy)\
**Post date:** [August 19, 2024, 12:39pm UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/2 "2024-08-19T12:39:26Z")

</div>

I think it is possible, you can refer to this sample.

> <https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/blob/master/back-to-back-detectors/back_to_back_detectors.c>

Just configure it as back-to-back instance segmentation.

---

<div class="post-metadata">

**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\
**Post date:** [August 20, 2024, 1:40am UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/4 "2024-08-20T01:40:53Z")

</div>

> [@md44](#):
>
> I would like to use both yolo-seg and yolo-pose as stages before the tracker, is it possible?

nvtracker can only track objects with bboxes. If the yolo-seg or yolo-pose can output bboxes to nvtracker, you can put them before nvtracker.

> [@md44](#):
>
> I see that both pose and seg are using obj\_meta.mask\_params, so how to make each use its own?

Where did you see such code?

---

<div class="post-metadata">

**Author:** ![md44](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@md44](https://forums.developer.nvidia.com/u/md44)\
**Post date:** [August 20, 2024, 8:17am UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/5 "2024-08-20T08:17:52Z")

</div>

for segmentation here:

> <https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/216ce3a08a61566cb3abcffdc0ddae30a57eb6b0/apps/deepstream-segmask/deepstream_segmask.py#L132C22-L132C36>

for pose esimation here:

> <https://github.com/marcoslucianops/DeepStream-Yolo-Pose/blob/9cf44737d2effcaecea599d122bb2f553493a17d/deepstream.py#L99C1-L99C53>

---

<div class="post-metadata">

**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\
**Post date:** [August 20, 2024, 9:11am UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/6 "2024-08-20T09:11:15Z")

</div>

> [@md44](#):
>
> how do I access the mask in the prob function?

DeepStream uses [Metadata in DeepStream](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_metadata.html) to store and transfer information through the elements after nvstreammux plugin in the pipeline. You can get the meta data in the pad probe functions. There are lots of [DeepStream sample applications](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_C_Sample_Apps.html), almost every sample application has the sample code of getting the batch meta, frame meta, object meta,…

The python samples are also available: [NVIDIA-AI-IOT/deepstream\_python\_apps: DeepStream SDK Python bindings and sample applications (github.com)](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps)

---

<div class="post-metadata">

**Author:** ![md44](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@md44](https://forums.developer.nvidia.com/u/md44)\
**Post date:** [August 20, 2024, 9:58am UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/7 "2024-08-20T09:58:54Z")

</div>

Thank you for your reply, the problem is as I showed in my last reply it seems that both pose estimator and segmenter get their data from the same object: obj\_meta.mask\_params.get\_mask\_array(), since I need both the pose estimator and segmenter output how do I get the correct data for each model?

---

<div class="post-metadata">

**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\
**Post date:** [August 21, 2024, 5:38am UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/8 "2024-08-21T05:38:19Z")

</div>

According to the sample you refers to. Both of the two models are instance segmentation models while the [NvDsObjectMeta](https://docs.nvidia.com/metropolis/deepstream/dev-guide/sdk-api/struct__NvDsObjectMeta.html) supports only one mask array. The suggestion is to change one of the model to the “unknown type” and set the mask array in the customized user meta [NvDsUserMeta](https://docs.nvidia.com/metropolis/deepstream/dev-guide/sdk-api/struct__NvDsUserMeta.html) in the [NvDsObjectMeta](https://docs.nvidia.com/metropolis/deepstream/dev-guide/sdk-api/struct__NvDsObjectMeta.html). Then the two mask arrays are all available.

---

<div class="post-metadata">

**Author:** ![md44](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@md44](https://forums.developer.nvidia.com/u/md44)\
**Post date:** [August 21, 2024, 6:11am UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/9 "2024-08-21T06:11:19Z")

</div>

Thank you, it works.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [September 4, 2024, 6:11am UTC](https://forums.developer.nvidia.com/t/using-yolo-pose-and-yolo-seg-together-in-one-python-app/303856/10 "2024-09-04T06:11:49Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.