DeepStream metamux breaks parent-child relationship in back-to-back detectors

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform GPU
• DeepStream Version 8.0
• NVIDIA GPU : RTX 5090
Pipeline Description

I am using back-to-back detectors in NVIDIA DeepStream SDK:

Without metamux (works as expected)

nvstreammux 
  -> pgie (car detection) 
  -> sgie (license plate detection) 
  -> fakesink (callback here)
  • I can access obj_meta.parent correctly for SGIE objects.

With metamux (issue)

nvstreammux 
  -> pgie (car detection) 
  -> sgie (license plate detection) 
  -> metamux (callback at src pad)
  -> fakesink
  • obj_meta.parent is always None when probing at metamux src.

Code Snippet

if obj_meta.unique_component_id == 4:  # SGIE (LPD)
    print("Found LPD")
    if not obj_meta.parent:
        print("NO parent")
    else:
        print("FOUND parent")

🔍 Observation

  • If I attach probe at sink_1 of metamux, I can still get the parent.

  • But at src of metamux, parent info is lost.

Questions

  1. Does metamux remove or not preserve parent relationships in NvDsObjectMeta?

  2. Is this expected behavior of metamux when merging metadata?

  3. Is there a way to preserve parent info when using metamux?

  4. If not, what is the recommended way to run parallel models (like DeepStream parallel inference sample) while still keeping parent-child relationships?

🎯 Goal

I want to:

  • Run multiple models in parallel (using metamux)

  • Still preserve parent-child mapping between PGIE and SGIE outputs

3

If you’re certain you need parallel inference, please refer to this sample. Could you share your current pipeline? I can then help you determine if parallel inference is necessary.

Parallel inference requires that the batch metata of each branch be independent; otherwise, metamux will not work as expected.

Metamux copies batchmeta data, which may result in the loss of parent-child mapping. This is the expected behavior.

In such scenarios, we only recommend saving relevant information in the sgie source pad probe function, rather than retrieving it after Metamux.

4

@junshengy thank you for your support
Here is my pipeline

graphviz (1)
graphviz (1)13590×2574 2.76 MB

SCR-20260425-ibwt
SCR-20260425-ibwt2354×864 163 KB

I’m trying to implement custom GStreamer elements to handle rule-based logic (e.g., person entering a zone, missing PPE, fall detection) using the output from the DeepStream metamux.

The metamux aggregates metadata from multiple models, including keypoints, detection, detection-to-classification, and detection-to-detection pipelines. This metadata is then passed downstream for rule processing.

However, I’m currently facing an issue with the detection-to-detection case: the metadata coming from metamux does not preserve the parent object information, so I’m unable to map secondary detections back to their corresponding primary objects.

Do you have any recommendations on how to handle this? Also, could you help review whether there might be any issues in my pipeline design?

6

This parallel inference pipeline should be working correctly. Regarding the issue of parent object information not being preserved, this might require using a global variable mapped to the object ID; this is caused by batch meta-copying.

7

@junshengy
I found the metamux source code in DeepStream 9. Would it be possible to modify it so that the parent-child metadata relationship is preserved after passing through metamux and then rebuild the plugin?

8

You can modify metamux as you like.
The metacopy should be implemented in libnvds_meta.so, which is not currently open source.

9

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.