Prevent certain frames from being inferred on

Hello,
I have a situation where I apply certain checks on a frame, and inference should happen only on the frames that pass those checks. I wanted to know what would be the best way to go about doing this.

One way would be to create a new NvBufSurface and copy the frames the pass the checks to this surface, but I feel like this copy process can be avoided altogether. Another option is to remove the NvDsFrameMeta of the frames that fail the checks, but after having a look at the nvinfer element implementation, it seems that gst_nvinfer_process_full_frame just loops through all the available frames in the buffer, regardless of what NvDsFrameMeta is present. gst_nvinfer_process_objects however, seems to do this since it loops through the NvDsFrameMetaList in NvDsBatchMeta.
I couldn’t find anything about deleting a surfaceList in nvbufsurface api as well.

Is there any way to do this without copying frames unnecessarily?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): GPU
• DeepStream Version: 6.0.1
• NVIDIA GPU Driver Version (valid for GPU only): 510.54

Inference interval in nvinfer can prevent inference. Is it possible to change this parameter for your case?

hey kesong,
Nope, I cannot use inference interval, I apply few opencv operations on the frame for my checks

Can you share your pipeline? Where are you add the opencv operations?

Hey kesong,
The opencv operations are in a custom gstreamer element. Currently, I’m mapping the nvbufsurface (on unified memory), apply my checks using opencv, and if the checks pass, copy that frame to new NvBufSurface (again unified memory).

the pipeline is the same as a regular detection pipeline, just with this custom element before nvinfer.

Is it possible to return input buffer without any output buffer if can’t pass the check?

Hey kesong,
I do drop the buffer if none of the frames pass the check. But what if out of, say, 4 frames in the batch, only 2 pass the check? then the output buffer needs to have only those two frames. and inference should only occur on those 2 frames.

If there’s any alternate way to achieve the same functionality that would help too.

nvdspreprocess can set ROI to define which source need inference. maybe you can have a try in nvdspreprocess.

Gst-nvdspreprocess (Alpha) — DeepStream 6.1 Release documentation (nvidia.com)