I am writing a camera src based on libArgus on TX2 with tegra_multimedia_api 28.3
I modified the gstVideoEncode example to use 4x nveglstreamsrc to pass the media into our gstreamer plugin.
This plugin hands the data over to a CUDA kernel which processes the input of these 4 streams.
The first question is what data is produced by nveglstreamsrc ?
Do i get a ref to an EGLStream or does nveglstreamsrc already Acquires EGLImages and passes these down the pipeline?
My first assumption was that I get EGLImages from the src. So I am trying to do the following before starting the kernel. According to samples/common/algortihm/cuda/NvCudaProc.cpp
The only exception seams to be cuda_add_library … SHARED which is required by gstreamer.
We are also manually setting the rpath to work with gstreamer.
I assume it is related to the rpath or some other linker option. Please help me out how to properly link libArgus libEGL cudaEGL and so on with cmake for shared libs.
I managed to get it working with this ugly workaround:
target_link_libraries( ${PROJECT_NAME}
${GST_LIBRARIES}
${EGL_LIBRARIES}
${CUDA_LIBRARIES}
${ARGUS_LIBRARIES}
EGL
argus
cuda
cudart
)
target_link_directories( ${PROJECT_NAME}
PUBLIC /usr/local/cuda/lib64
PUBLIC /usr/lib/aarch64-linux-gnu
PUBLIC /usr/lib/aarch64-linux-gnu/tegra
)
by using find_package the variable EGL_LIBRARIES and so on should actually be populated correctly and point to the actual lib. The contents are:
Thank you for pointing this out. In this case I have some additional questions.
Do I understand this correctly?
nveglstreamsrc only passes a ref to an EGLStream. I have to Acquire Frames myself in our cuda-powered element.
cudaEGLStreamConsumerAcquireFrame
The EGLStream is not buffered. So whatever I acquire, is what the camera currently produces.
We are going to sync the camera captures in hardware.
If I manage the acquisition of frames myself in our cuda-powered gst element I can make sure that all the frames are produced at the same time?!
What happens if a camera did not produce a new frame since the last Acquire?
Does nveglstreamsrc pass a new buffer on every new frame?
What is actually the content of the buffers passed by nveglstreamsrc?
I did not find any reference/doc to this element!! Please point me to the doc.
I need to use Interoperability to enable Argus > CUDA zero-copy
How do I do that when using nveglstreamsrc.
I hope to clarify the situation to get a better picture of the architecture and to better integrate it into our architecture.
Hi,
We suggest yo refer to 09_camera_jpeg_capture and 10_camera_recording. You can do processing by calling:
// Create EGLImage from dmabuf fd
ctx->egl_image = NvEGLImageFromFd(ctx->egl_display, buffer->planes[0].fd);
if (ctx->egl_image == NULL)
{
fprintf(stderr, "Error while mapping dmabuf fd (0x%X) to EGLImage\n",
buffer->planes[0].fd);
return false;
}
// Running algo process with EGLImage via GPU multi cores
HandleEGLImage(&ctx->egl_image);
// Destroy EGLImage
NvDestroyEGLImage(ctx->egl_display, ctx->egl_image);
ctx->egl_image = NULL;
Implementation of HandleEGLImage()
/**
* Performs CUDA Operations on egl image.
*
* @param image : EGL image
*/
static void
Handle_EGLImage(EGLImageKHR image)
{
CUresult status;
CUeglFrame eglFrame;
CUgraphicsResource pResource = NULL;
cudaFree(0);
status = cuGraphicsEGLRegisterImage(&pResource, image,
CU_GRAPHICS_MAP_RESOURCE_FLAGS_NONE);
if (status != CUDA_SUCCESS)
{
printf("cuGraphicsEGLRegisterImage failed: %d, cuda process stop\n",
status);
return;
}
status = cuGraphicsResourceGetMappedEglFrame(&eglFrame, pResource, 0, 0);
if (status != CUDA_SUCCESS)
{
printf("cuGraphicsSubResourceGetMappedArray failed\n");
}
status = cuCtxSynchronize();
if (status != CUDA_SUCCESS)
{
printf("cuCtxSynchronize failed\n");
}
if (eglFrame.frameType == CU_EGL_FRAME_TYPE_PITCH)
{
//Rect label in plan Y, you can replace this with any cuda algorithms.
addLabels((CUdeviceptr) eglFrame.frame.pPitch[0], eglFrame.pitch);
}
status = cuCtxSynchronize();
if (status != CUDA_SUCCESS)
{
printf("cuCtxSynchronize failed after memcpy\n");
}
status = cuGraphicsUnregisterResource(pResource);
if (status != CUDA_SUCCESS)
{
printf("cuGraphicsEGLUnRegisterResource failed: %d\n", status);
}
}