Split input stream into multiple ROIs

[kind_of_blue]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson Nano 3GB
• DeepStream Version: 6.0
• JetPack Version (valid for Jetson only)
• TensorRT Version: 8.2.1
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs): Creating Deepstream pipeline
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

So my issue is that I have a single source in my pipeline and I want to only infer certain regions of it with the same PGIE and Tracker and then visualize them back on the same output (display/rtsp stream - mostly for debugging). I also need a probe to fetch data about each detected object inside each region for counting them and sending statistics to the external server which is why I’m using Python API for Deepstream.

From what I’ve researched so far is that I need a “tee” element to duplicate input stream and then use “nvvideoconvert” or “videocrop” to cut my ROIs and then infer them on separate branches - kinda like here.

But I can’t get it to work with Python and I’m kinda lost what’s the best approach here and the additional issue is that I haven’t really seen anyone doing this kinda thing in Python. But this is the pipeline that I think should work here (for 2 ROIs):

                                       |-> NVVIDEOCONVERT [with src-crop] -> PGIE -> TRACKER ->|
URIDECODE_BIN -> NVSTREAMMUX -> TEE  ->|                                                       |-> NVMULTISTREAMTILER -> NVOSD -> NV3D-SINK
                                       |-> NVVIDEOCONVERT [with src-crop] -> PGIE -> TRACKER ->|

The resulting OSD doesn’t have to be those regions put back in their place with visualized detections but they have to be at least placed next to each other using for example Tiler. I can’t seem to get it to work in Python and wanted to ask if this is the right approach for this kind of thing. Would really appreciate some tips or maybe links with similar issues resolved using Python API so I could implement this in my own project.

[yuweiw]

We don’t have python demo for deepstream_parallel_inference_app. Could you refer to the gst-nvdsanalytics and see if this plugin can meet your needs?

[kind_of_blue]

Thanks for the fast reply!

This plugin is really helpful and could actually solve my issue with counting objects in certain regions but there is another problem because I have a FullHD stream (possibly 4K in the future) and objects that I’m detecting are rather small. So if I’m scaling 1920x1080 to 640x640 - which is an input for my YoloV5n detector - it has some problems detecting those objects. That’s why I implemented splitting fetched frames by those ROIs and then running inference on each ROI separately which worked like a charm.

I really want to use Deepstream in my project but can’t really find a way for it to work, cause the only way I can think of is splitting this input stream using tee, performing inference+tracking on each of those separately, and then putting them back together for further analysis. That’s okay if you do not have an example for it (but it could be really helpful) and I’m rather looking for a confirmation that I can be done this way and there is no better way to do this.

[yuweiw]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

For your case, we suggest you use preprocess to divide your image into different ROI areas to infer. You can refer the link below:
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvdspreprocess.html