Where to put the nvmultistreamtiler in a pipeline?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson AGX Xavier
• DeepStream Version: 5.0
• JetPack Version (valid for Jetson only): 4.4
• TensorRT Version: 7.1.3
• Issue Type( questions, new requirements, bugs):questions

In my application, I have 2 camera sources to capture objects from 2 different angles. As I understand, nvmultistreamtiler merges frames in a buffer into a single frame, it also merges the frame-metadatas in a buffer into a single frame-metadata. My questions are:

  1. Do we put nvmultistreamtiler before or after nvtracker / nvdsosd?
  2. I need to keep track of how many detection of each class from each source, but nvmultistreamtiler merge frames & their metadatas from different sources into a single frame/metadata, thus making it impossible to distinguish which detection comes from which source. To solve this, I think I can push nvmultistreamtiler further downstream but does it make sense to put nvmultistreamtiler after nvtracker & nvdsosd (i.e., nvtracker -> nvdsosd -> nvmultistreamtiler) ?

The typical sample of the pipeline is deepstream-nvdsanalytics-test, it supports multiple sources.

  1. nvtracker should be right after nvinfer(PGIE, detector), nvmultistreamtiler should be used after nvinfer, nvtracker and nvdsanalytics and before nvdsosd.
  2. Even with multiple sources, the detector(nvinfer), tarcker, and analytics can identify which object belongs to which source. That is the reason we use nvstreammux to start the deepstream pipeline. https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvstreammux.html

Hey, is it possible to somehow run the tracker after tiling the batches ?
nvinfer batch-4 -> nvmultistreamtiler -> nvtracker so the nvtracker would think its a full frame rather than 4 different frames.

Hi aditdoshi333,

Please help to open a new topic with more details of your requirement.