How to render an EGLImageKHR without EGL display?

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

I’m looking at the following code from NVIDIA that renders an image from a file descriptor called render_fd

EGLImageKHR hEglImage;
    bool frame_is_late = false;

    EGLSyncKHR egl_sync;
    int iErr;
    hEglImage = NvEGLImageFromFd(egl_display, render_fd);
    if (!hEglImage)
    {
        COMP_ERROR_MSG("Could not get EglImage from fd. Not rendering");
        return -1;
    }

    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture_id);
    glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, hEglImage);
    glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

    iErr = glGetError();
    if (iErr != GL_NO_ERROR)
    {
        COMP_ERROR_MSG("glDrawArrays arrays failed:" << iErr);
        return -1;
    }
    egl_sync = eglCreateSyncKHR(egl_display, EGL_SYNC_FENCE_KHR, NULL);
    if (egl_sync == EGL_NO_SYNC_KHR)
    {
        COMP_ERROR_MSG("eglCreateSyncKHR() failed");
        return -1;
    }
    eglSwapBuffers(egl_display, egl_surface);
    if (eglGetError() != EGL_SUCCESS)
    {
        COMP_ERROR_MSG("Got Error in eglSwapBuffers " << eglGetError());
        return -1;
    }
    if (eglClientWaitSyncKHR (egl_display, egl_sync,
                EGL_SYNC_FLUSH_COMMANDS_BIT_KHR, EGL_FOREVER_KHR) == EGL_FALSE)
    {
        COMP_ERROR_MSG("eglClientWaitSyncKHR failed!");
    }

    if (eglDestroySyncKHR(egl_display, egl_sync) != EGL_TRUE)
    {
        COMP_ERROR_MSG("eglDestroySyncKHR failed!");
    }
    NvDestroyEGLImage(egl_display, hEglImage);

Here is the fragment shader:

#extension GL_OES_EGL_image_external : require
    precision mediump float;
    varying vec2 interp_tc; 
    uniform samplerExternalOES tex;
    void main() {
        gl_FragColor = texture2D(tex, interp_tc);
    };

The problem is that I want to render independently from an display (or should I sar EGL display?). I might want to display it now but in the future I want to not use a display at all. I want to render to something and plug this something to another opengl program that renders it.

I found the documentation of NvEGLImageFromFd:

EGLImageKHR NvEGLImageFromFd(EGLDisplay display, int dmabuf_fd)

Creates an EGLImage instance from dmabuf-fd.

Parameters
[in] display EGLDisplay object used during the creation of EGLImage. If NULL, the nvbuf_utils API uses its own EGLDisplay instance.
[in] dmabuf_fd DMABUF FD of buffer from which EGLImage to be created.

It looks like I can pass NULL to it, so I tried the following code:

EGLImageKHR hEglImage;

		int iErr;
		hEglImage = NvEGLImageFromFd(NULL, decodedNvFrame->nvBuffer->planes[0].fd);
		if (!hEglImage)
			printf("Could not get EglImage from fd. Not rendering\n");
		
		glBindVertexArray(vertexArrayObject);
                glActiveTexture(GL_TEXTURE0);
		glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture_id);
		glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, hEglImage);
		glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

		iErr = glGetError();
		if (iErr != GL_NO_ERROR)
			printf("glDrawArrays arrays failed:%i\n", iErr);

		NvDestroyEGLImage(NULL, hEglImage);

Even though I see no errors, I also see no output in the screen. Is there something I’m missing? Are those eglConfig, eglSurfaces, etc, needed to render to a screen? If so, how to render to a buffer and then render to a screen?

2

Hi,
If you have to render to a real device, you need to have EGLDisplay and set ‘export DISPLAY=:1(or 0)’

It is expected no output on screen if EGLDisplay is NULL.

3

then what is the NULL option for? What if I want to render to a frame buffer?

4

Hi,
We have a usecase of CUDA postprocessing without rendering:

static bool
cuda_postprocess(context_t *ctx, int fd)
{
    if (ctx->enable_cuda)
    {
        // Create EGLImage from dmabuf fd
        ctx->egl_image = NvEGLImageFromFd(ctx->egl_display, fd);
        if (ctx->egl_image == NULL)
            ERROR_RETURN("Failed to map dmabuf fd (0x%X) to EGLImage",
                    ctx->render_dmabuf_fd);

        // Running algo process with EGLImage via GPU multi cores
        HandleEGLImage(&ctx->egl_image);

        // Destroy EGLImage
        NvDestroyEGLImage(ctx->egl_display, ctx->egl_image);
        ctx->egl_image = NULL;
    }

    return true;
}

In this case, EGLDisplay can be NULL.

5

There’s something I don’t understand. Why do I need to have an eglDisplay just to receive an image? I shouldn’t need a display for that, I should need a display to render the image.

I’m using GTK and I’m having a hard time trying to render something in the screen.

In the code

EGLImageKHR hEglImage;

    hEglImage = NvEGLImageFromFd(eglDisplay, decodedNvFrame->nvBuffer->planes[0].fd);
    if (!hEglImage)
        printf("Could not get EglImage from fd. Not rendering\n");
    glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, frameBufferTexture, 0);     
    glBindTexture(GL_TEXTURE_2D, externalTexture);
    glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, hEglImage);

shouldn’t my texture externalTexture have the image from NvEGLImageFromFd, independently if I pass a display or not? Even if I pass NULL, shouldn’t I get a texture anyway?

I can’t make

glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, hEglImage)

fill a texture so I can render with my shader. I think it has to do with this egl display thing.

6

No matter what I try, I cannot make glEGLImageTargetTexture2DOES fill the texture, perhaps because NvEGLImageFromFd won’t work.

In NvEglRenderer.cpp, even though it uses a certain display at eglMakeCurrent, it uses EGL_NO_DISPLAY in the NvEGLImageFromFd call.

When I use the egl configurations from NvEglRenderer.cpp, I can render to a frame buffer, but I want to not depend on X11. I followed https://devblogs.nvidia.com/egl-eye-opengl-visualization-without-x-server/ but I can’t make glEGLImageTargetTexture2DOES work

I also adapted https://github.com/svenpilz/egl_offscreen_opengl/blob/master/egl_opengl_test.cpp and it won’t work also

The only way it works is when I use the X11 display as in NvEglRenderer.cpp

Could someone please tell me how to make glEGLImageTargetTexture2DOES fill a texture target without an X11 screen?

7

Hi,
NvEglRenderer.cpp depends on X11.
Some other users may have experience of rendering EGLImage without X11.