Issues with tee branching

I am working on running the inference pipeline in parallel.

Primary branch
pgie → tee breaker
1st branch
breaker → preprocess (to increase the bounding boxes dimension at the probe level) → sgie 1 → sink
2nd branch
breaker → sgie 2 → sink

List of questions:

  1. As I understand the tee should not copy the changes happened in the one branch on the other branch. But, as I increase the bounding boxes in the 1st branch, thats being effecting on the second branch (2nd branch). Is it possible to keep the obj_meta independent for both branches.

  2. Instead of changing the bounding box in the probe function before passing it to the sgie-1, can I use a preprocess element or some other element before the sgie which will impose only on the 1st-branch.

Thank you,

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 6.2
• JetPack Version (valid for Jetson only) 5.1.2
• TensorRT Version 8.5.2
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

It will “copy”. Actually, the GstBuffer in one branch maps to the same memory of the other branch, when there is any change in one branch, the change is reflected in the other branch.

You need to copy the GstBuffer in at least one branch to separate the GstBuffers in the two branches.

“Internal data stream error” with tee and nvstreammux - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums

Thanks for the quick response,

I am not getting any errors, but the data gets modified in one branch, and changes happen in the other branch as well.

As you suggested, I created a gstBuffer copy and updated all necessary changes, along with unrefing the newly created GstBuffer in Python. Is this what we have to do?

I would appreciate it if you could provide an example of how to do it.

Thank you,

That is because of the GstBuffers in the two branches point to the same memory.

Please refer to Parallel branching in deepstream 6.4 - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums

I checked the link you provided. As I understand, the example or explanation is all about creating multiple branch by cloning the GstBuffer.

Here, in my case, I want to copy the GstBuffer. How can we do ?

No. The sample is to make the GstBuffers in different branches be separated. After the “nvvideoconvert”, you can change anything in the GstBuffer in one branch, the other branch will not be impacted.

You can apply the nvpreprocess after the nvvideoconvert. But it is not recommended to modify the dimension of the bboxes in nvpreprocess. The proper way should be adding new user meta to contain the new bboxes(E.G. to contain the 3D bbox coordinates). And the interface between nvpreprocess and nvinfer is the output tensor data. I don’t understand why you change the bbox dimension, how will you use the changed bboxes?
If you only want to add more bboxes in one branch, the nvpreprocess is not a good choice. A simple probe function after the nvvideoconvert is enough.

Please make sure you are familiar with GStreamer before you start with DeepStream SDK.

Unfortunately nvvideoconvert didn’t help to create a metadata copy or break the link between two branches @Fiona.Chen

I need to enlarge the bounding box in 1st branch for different functionality as the sgie-1 model is expecting bigger bounding box compared with another sgie-2 model in another branch.

Any other suggestions, please?