In v4l2cuda example,how to add renderer to display the data in Unified Memory converted by GPU?

Hi Nvidia:
In tegra_multimedia_api/sample/v4l2cuda, we use command ./capture-cuda -d /dev/video0 -u -z to get video data from usb camera and convert the video data with GPU,and the memory we use is the Unified Memory.
In order to display the converted data with renderer,I add renderer in program, at the same time I modified NvEglRenderer.cpp file in tegra_multimedia_api/sample/common/classes and NvEglRenderer.h file in tegra_multimedia_api/include.

In tegra_multimedia_api/sample/v4l2cuda capture.cpp file modified as below:

In capture.cpp file add global variable renderer;
#include “NvEglRender.h”
NvEglRenderer *renderer = NULL;

In main function add function display_initial() and display_destroyed(),
static bool dispaly_initial()
{
renderer = NvEglRenderer::createEglRenderer(“renderer0”,640,480, 100,100);
if(!renderer)
printf(“Failed to create EGL render\n”);
renderer->setFPS(30);
return true;
}

static bool dispaly_destroyed()
{
if(renderer != NULL)
delete renderer;
return true;
}

modified 
static void process_image(void *p)
{
    printf("CUDA format conversion on frame %p\n",p);
    gpuConvertYUYVtoRGB((unsined char*)p,cuda_out_buffer,width,height);

    printf("cuda_out_buffer value:%p\n",cuda_out_buffer);
    [b]renderer->render((EGLClientBuffer) cuda_out_buffer);[/b]
 }

 In file [b]NvEglRenderer.h[/b]
 modified int [b]render(int fd)[/b] as int [b]render(EGLClientBuffer buffer)[/b];
 add privite variable  [b]EGLClientBuffer egl_buffer[/b];
 delete privite variable [b]int render_fd[/b];


 In file NvEglRenderer.cpp
 in function NvEglRenderer::NvEglRenderer(const char *name, uint32_t width, uint32_t height,  uint32_t x_offset, uint32_t y_offset) : NvElement(name,valid_fields)
 [b]add egl_buffer = NULL;[/b]
 
 modified function int NvEglRenderer::render(int fd) as
 int NvEglRenderer::render(NvEglClientBuffer buffer)
 {
      [b]this->egl_buffer = buffer;[/b]
      pthread_mutex_lock(&render_lock);
      pthread_cond_broadcast(&render_cond);
      COMP_DEBUG_MSG("Rendered egl_buffer="<<buffer):
      pthread_cond_wait(&render_cond,&render_lock);
      pthread_mutex_unlock(&render_lock);
      return 0;
 }

 modified function int NvEglRenderer::renderInternal() as 
 int NvEglRenderer::renderInternal()
 {
       EGLImageKHR hEglImage;
       bool frame_is_late = false;

       EGLSyncKHR egl_sync;
       int iErr;
       [b]EGLint eglImageAttributes[]={EGL_WIDTH,640,EGL_HEIGHT,480};
       hEglImage = eglCreateImageKHR(egl_display,EGL_NO_CONTEXT,EGL_GL_TEXTURE_2D_KHR,egl_buffer,eglImageAttributes);[/b]
       if(!hEglImage)
       {
            COMP_ERROR_MSG("could not get EglImage from buffer. Not rendering");
            return -1;
       }
       ......
       ......
 }
 run my program,"could not get EglImage from buffer. Not rendering" printed many time!
 is there any wrongs or is there any else to do?

 Expected your reply !  Thanks a lot !

Hi feng,
It is not supported. You must allocate NvBuffer.

Hi DaneLLL
Renderer can only operate NvBuffer? any other buffer can not be operated by renderer?

NvEglRenderer only operates NvBuffer.
There is another renderer for Argus at tegra_multimedia_api\argus\samples\utils\PreviewConsumer.cpp

If you cannot use our implementation, you need to implement your own by yourself.

Hi DaneLLL:
Thank you for your reply!
CPU can both access to NvBuffer and Unified Memory, converted camera video data saved in NvBuffer is indicated by buffer_fd, and converted camera video data saved in Unified Memory is indicated by buffer start address,the only difference in whole process of display converted camera video data is the EGLImageKHR hEglImage in EGL, when buffer is NvBuffer,use tegra_multimedia_api NvEGLImageFromFd to get hEglImage,and when buffer is Unified Memory,EGL Function eglCreateImageKHR can be used to get hEglImage, it feels that there is no problem in principle.why NvEglRenderer can only operates NvBuffer? it is limited by EGL or any other reason?
Expected your reply ! Thanks a lot !

NvEglRenderer is implemented along with NvBuffer. Not sure why your customization does not work. Other users may share their experience.

Hi,Feng
We use ./capture-cuda -d /dev/video0 -u -z but it always mentioned “select timeout”.Our board is NX.
Could you please provide any suggestion?

Thanks&BRs.

Hi 932315769,

Please help to open a new topic. Thanks