I would like to retrieve a buffer from a Gstreamer pipeline and do some processing on it using CUDA functions on gpu. We tried several approaches on our Jetson TX2:
gstreamer custom plugin: we downloaded the standard gstreamer template that processes images using the so called “chain function”, which retrieves the input buffer as GstBuffer, elaborates it and pushes the output in the source pad. The main problem is that the data pointer of the buffer is being allocated on the cpu memory and then it is not directly accessible by CUDA.
An alternative would be to use the NVMM memory obtained using a caps filter followed by videoconvert plugin. However, CUDA still cannot read the input buffer. As a result, a memory copy from host to device is necessary even if the TX2 supports unified memory protocol.
NVIVAFILTER plugin: after some research I found out that the only solution for direct gpu memory allocation using gstreamer is to use NVIVAFILTER that calls a .so library.
Although I managed to use CUDA functions on the input data, NVIVAFILTER accepts only one input buffer at a time and this is a problem in case of image stitching or stereo acquisition because they both need multiple inputs.
tegra_multimedia_api: another possibility is to use tegra_multimedia_api functions, that is mentioned in this question that we made:
But this does not allow to use gstreamer parallelism because it is a separated library.
To sum up, my question is: is there a gstreamer plugin/tool that allows to process multiple inputs directly allocated in CUDA memory and to output multiple buffers too?