An application linked to opencv would only receive frames into CPU allocated cv::Mat using opencv videoio. This is not very fast and would not work for high resolutions/framerates. Same applies to CPU Mat processing.
You would indeed use GPU processing from NVMM memory in gstreamer in such case.
You can access NVMM buffers with gstreamer plugin nvivafilter. It is intended to perform CUDA operations on NVMM hosted frames, so you can use it with opencv CUDA. You would have to output RGBA frames from this plugin. You may have a look to this example.
Also note that you can directly access from gstreamer buffer if your application builds the gstreamer pipeline.