Using yolo-seg and yolo-pose with the same tracker and access both in the probe

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 7.0
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) 535.161.08
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi,

I would like to run both yolo-seg and yolo-pose with the same tracker, and use their results in the probe, I managed to make it work, I set the seg config gie-unique-id=1 and pose config gie-unique-id=2
in the probe I have this code:
if obj_meta.unique_component_id == 1:
process_segments()
if obj_meta.unique_component_id == 2:
process_pose()

however for some reason I see for the same frame and for the same track id, either pose information or segment information, I never see both, from time to time they might interchange, so If I look at tracker data I see the mask for several frames and than I see the skeleton for several frames, I never see both mask and skeleton on the same frame.
I tried to set the link between the two in the pipeline so that the first is segment and the second is pose and vice versa (the tracker is right after) and it didn’t solve the issue. If I use each model alone they work fine (accept that the pose return less results)

thanks.

3

Can you share some image or video to show the issue?

4

as you can see wherever there is a mask there is no skeleton and vice versa

5

How you store the seg and pose data in the DS meta?

6

I only took yolo-seg and yolo-pose from their respective repositories,

where exactly the store in the meta happens?

7

See nvds_add_display_meta_to_frame in the code
Thanks

8

You shared two repo for seg and pose. How you run the two model together? Can you share the reproduce of your issue?