Prevent certain frames from being inferred on

I have a situation where I apply certain checks on a frame, and inference should happen only on the frames that pass those checks. I wanted to know what would be the best way to go about doing this.

One way would be to create a new NvBufSurface and copy the frames the pass the checks to this surface, but I feel like this copy process can be avoided altogether. Another option is to remove the NvDsFrameMeta of the frames that fail the checks, but after having a look at the nvinfer element implementation, it seems that gst_nvinfer_process_full_frame just loops through all the available frames in the buffer, regardless of what NvDsFrameMeta is present. gst_nvinfer_process_objects however, seems to do this since it loops through the NvDsFrameMetaList in NvDsBatchMeta.
I couldn’t find anything about deleting a surfaceList in nvbufsurface api as well.

Is there any way to do this without copying frames unnecessarily?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): GPU
• DeepStream Version: 6.0.1
• NVIDIA GPU Driver Version (valid for GPU only): 510.54

Inference interval in nvinfer can prevent inference. Is it possible to change this parameter for your case?

hey kesong,
Nope, I cannot use inference interval, I apply few opencv operations on the frame for my checks

Can you share your pipeline? Where are you add the opencv operations?

Hey kesong,
The opencv operations are in a custom gstreamer element. Currently, I’m mapping the nvbufsurface (on unified memory), apply my checks using opencv, and if the checks pass, copy that frame to new NvBufSurface (again unified memory).

the pipeline is the same as a regular detection pipeline, just with this custom element before nvinfer.

Is it possible to return input buffer without any output buffer if can’t pass the check?

Hey kesong,
I do drop the buffer if none of the frames pass the check. But what if out of, say, 4 frames in the batch, only 2 pass the check? then the output buffer needs to have only those two frames. and inference should only occur on those 2 frames.

If there’s any alternate way to achieve the same functionality that would help too.

nvdspreprocess can set ROI to define which source need inference. maybe you can have a try in nvdspreprocess.

Gst-nvdspreprocess (Alpha) — DeepStream 6.1 Release documentation (

Hey kesong,
thanks for you suggestion.
I had a look at the nvdspreprocess documentation link you sent and have a doubt. The custom library exposes two interface functions, custom_transform and custom_tensor_function. custom_transform’s default implementation provides scale & conversion, so the logic I require doesn’t go there. That leaves custom_tensor_function, where I have to fill NvDsPreProcessCustomBuf with the rgb data of all the frames that pass my checks? I’m not sure how to set the ROIs, the documentation mentions setting the ROIs in the config file but that does not fit my use case (since it is possible that none of the frames in the batch pass the check, and there can be times when all pass the checks, etc). The documentation isn’t very clear and the deepstream-3d-action-recognition sample app seems to be overkill for my use case.

Current implementation is set ROI in the configure file. Seems you need to change the ROI based on “check pass”

how would I go about doing that? do I need to modify nvdspreprocess?

Yes. nvdsperprocess is open source plugin.

ok will look into it.
thanks for your help kesong

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.