Hi,
Please refer to this sample:
Nano not using GPU with gstreamer/python. Slow FPS, dropped frames - #8 by DaneLLL
You can map NvBuffer to cv::gpu::gpuMat for doing processing. Please take a look at the sample and see if you can apply it to your usecase.