Need information on deep tunneled gst video buffer 'video/x-raw(memory:NVMM)'

Hi,

Does anybody have information on deep tunneled gst video buffer, i.e. ‘video/x-raw(memory:NVMM)’ used in nvcamerasrc and other NVIDIA plugins? I am trying to process the video buffer using Visionworks libraries, but don’t want to pull it up to host land and push down again to device. Ideally I’d like to pass the deep tunneled buffer pointer directly to vxCreateImageFromHandle().

Thanks,

  • Biyong

Hi biyong,

Sorry for the late response.

We need detailed information about the exact use-case to do more investigation, would you please provide it?

Thanks

Hi kayccc,

I do OpenVX based image processing using Visionworks library, in which I need to access video frame buffer from camera source and pass it to Visionworks library calls. For example, in order to create a source image for the video frame, it is needed to pass the address of the frame buffer along with access information such as pitch size etc. to vxCreateImageFromHandle(). Without insight of the ‘video/x-raw(memory:NVMM)’, the only way to do this is to transfer the content from ‘video/x-raw(memory:NVMM)’ memory to user land and copy it again to a cuda based memory and then pass it on to Visionworks calls. Ideally, if we could get buffer address and information that is suitable for creating vx_imagepatch_addressing_t needed for vxCreateImageFromHandle() from ‘video/x-raw(memory:NVMM)’ directly, it would save a bunch of memory copies. This is critical for us to keep up with the framerate requirement.

Thanks,

  • Biyong

Hi biyong,

The “video/x-raw(memory:NVMM)” gst camera buffer is converted to CUDA buffer in VisionWorks (as a part of NVXIO) framework when nvxio::FrameSource::fetch is called.
You can refer to sample application of nvgst camera catpture: ./nvgstcamera_capture/main_nvgstcamera_capture.cpp to know the usage.

Thanks

Hi Kaycc,

Thanks for the update. It took me a while to get back on this. What I found is that in ./nvgstcamera_capture/main_nvgstcamera_capture.cpp, the nvcamerasrc seems to be hooked up to nvvideosink, and the frame fetch is done later through cuEGLStreamConsumerAcquireFrame method. Are you saying that EGL Stream is being used in nvcamerasrc? Is there any more information regarding Nvidia EGL Stream that you can point me to if that is the case?

Thanks,

  • Biyong

Hi biyong

Are you saying that EGL Stream is being used in nvcamerasrc?
There are various modes supported by nvgstcapture application like csi, v4l2 and egl stream. The nvvideosink is like a EGLStream producer to gen EGLStream buffer for EGLStream consumer.

Is there any more information regarding Nvidia EGL Stream that you can point me to if that is the case?
The more information is on following link:
https://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_stream.txt