Hi,
There are samples for mapping NvBuffer to cv::gpu::gpuMat in gstreamer and jetson_multimedia_api. Please take a look at
Nano not using GPU with gstreamer/python. Slow FPS, dropped frames - #8 by DaneLLL
LibArgus EGLStream to nvivafilter - #14 by DaneLLL
With the approach you can apply CUDA filters to the buffers directly. Please check and give it a try.