In [Letterbox and Rectification probe] we use VPI’s Remap to rectify the image for stereo generation and undistortion as well as run a small cuda kernel to make letterbox the image with gray padding.
Currently in the probe there is the following sring of conversions:
NvBufSurface frame → cpu memory → VPIImage (with cuda backend) → VPI Remap(rectify the image) → cpu memory → device memory(gpu) → custom letterbox kernel → cpu(in gstbuffer) → nvvideoconvert(to NvBufSurface)
I have an idea how to use NvBufSurface for the letterbox but I don’t know how to do the vpi-remap using NvBufSurface memory.
One guess I have is to try VPI’s EGL Interoperability to try to pass NvSurface memory to VPI.
Is that a correct asumption? Any suggestions and tip would very helpful.
And can you point me to detailed documentation how to use NvBufSurface?
I have successfully implemented image rectification using an input image NvBufSurface with EGL interoperability, as described in your documentation here: EGL Interoperability. I was able to obtain an EGLImage from NvBufSurface using the NvBufSurfaceMapEglImage function.
Now, I need to convert the processed buffer back into a NvBufSurface to pass it downstream in DeepStream. However, I have a few concerns and would like your guidance:
Handling Multiple Buffers and Batches: Given that NvInfer works with batches, I am concerned about managing multiple buffers. Will DeepStream automatically manage (delete or recycle) the NvBufSurface buffers after they are processed, or do I need to manually manage the buffer’s lifecycle? Is there an internal structure within DeepStream that handles the reference counting (refs/unrefs) for these buffers? How can I know when a certain buffer is available, so it can be reused.