Accessing the preprocess ROIs information after inferencing

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jeston AGX Orin
• DeepStream Version 6.1
• JetPack Version (valid for Jetson only) 5.0.2
• TensorRT Version 8.4.1.5
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I want to use python Deepstream pipeline to do the following?
1- cropping the video frame into small crops with overlap between the crops
2- run segmentation
3- teil the crops again considering the overlap between them and the same for the resulting mask

I did some research and I found out that I can use nvdspreprocess plugin to select ROIs which in my case I could assume they are crops. After inferencing, I was thinking that I could use nvmultistreamtiler plug-in and add a probe to get informed of the metadata generated where I can call gst_buffer_get_nvds_batch_meta() but what I couldn’t find (or know) how to get the selected ROIs ? I was always getting the complete frame. The same if I add a probe to nvinfer. is there a way to get the selected ROIs ? or do I need to create a custom plug-in for pre and postprocessing?

3
  1. what is the media pipeline?
  2. do you want to know which object belongs to which roi? currently there is no roi information in object meta. there is roi preprocess_batchmeta with id NVDS_PREPROCESS_BATCH_META in batchmeta, you can compare the object’s coordinate with roi’s, then you can know which object belongs to which roi.
4

Hi, thanks for the fast replay I am using rtsp video stream to do segmentation. Do u know if there is somewhere an example of using NVDS_PREPROCESS_BATCH_META ?
and also if I have the ROIs overlapped then how segmentation mask is created? (the result of which ROI is selected?)

BR,

6

nvinfer plugin opensource, please refer to nvinfer’s gst_nvinfer_process_tensor_input in deeptream sdk, if nvinfer’s input-tensor-meta is 1, nvinfer will call this function to get the tensor preprocessed by nvdspreporcess plugin.

is the topic preprocess the same question?

7

Hi Fanzh,

is the topic preprocess the same question?

yes, it is the same. so to understand correctly, the overlap is not supported currently?
is there any other way to for example crop the main frame to multi crops and do inference on them

8

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

nvpreproccess plugin dose not forbid to use overlapped roi, please refer to nvdspreprocess. nvosd is reponsbile for drawing mask using segmenation mask data, nvosd will draw mask by turn.

pelase refer to deepstream sample opt\nvidia\deepstream\deepstream-6.2\sources\apps\sample_apps\deepstream-preprocess-test.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.