Hello,
I have a situation where I apply certain checks on a frame, and inference should happen only on the frames that pass those checks. I wanted to know what would be the best way to go about doing this.
One way would be to create a new NvBufSurface and copy the frames the pass the checks to this surface, but I feel like this copy process can be avoided altogether. Another option is to remove the NvDsFrameMeta of the frames that fail the checks, but after having a look at the nvinfer element implementation, it seems that gst_nvinfer_process_full_frame just loops through all the available frames in the buffer, regardless of what NvDsFrameMeta is present. gst_nvinfer_process_objects however, seems to do this since it loops through the NvDsFrameMetaList in NvDsBatchMeta.
I couldn’t find anything about deleting a surfaceList in nvbufsurface api as well.
Is there any way to do this without copying frames unnecessarily?
Please provide complete information as applicable to your setup.
Hey kesong,
The opencv operations are in a custom gstreamer element. Currently, I’m mapping the nvbufsurface (on unified memory), apply my checks using opencv, and if the checks pass, copy that frame to new NvBufSurface (again unified memory).
the pipeline is the same as a regular detection pipeline, just with this custom element before nvinfer.
Hey kesong,
I do drop the buffer if none of the frames pass the check. But what if out of, say, 4 frames in the batch, only 2 pass the check? then the output buffer needs to have only those two frames. and inference should only occur on those 2 frames.
If there’s any alternate way to achieve the same functionality that would help too.
Hey kesong,
thanks for you suggestion.
I had a look at the nvdspreprocess documentation link you sent and have a doubt. The custom library exposes two interface functions, custom_transform and custom_tensor_function. custom_transform’s default implementation provides scale & conversion, so the logic I require doesn’t go there. That leaves custom_tensor_function, where I have to fill NvDsPreProcessCustomBuf with the rgb data of all the frames that pass my checks? I’m not sure how to set the ROIs, the documentation mentions setting the ROIs in the config file but that does not fit my use case (since it is possible that none of the frames in the batch pass the check, and there can be times when all pass the checks, etc). The documentation isn’t very clear and the deepstream-3d-action-recognition sample app seems to be overkill for my use case.