Lowest-latency method to display an image in real-time

[BareMetalCoder]

Yet another post on my latency deep dive.
Our application produces an output image using CUDA from several synchronized camera feeds. The resulting image is in GPU memory.

I would like to display the image on-screen as fast as possible.

I currently use OpenGL/GLUT in a C/C++/CUDA program to display the image by first copying it to a texture and then rendering the texture on a GL_QUAD that fills the screen, followed by glutSwapBuffers(). I’m the first to admit that my OpenGL chops are a bit out of date.

Is there a faster way to render to the screen on the Jetson AGX Xavier? I.e. can I somehow write to the screen’s framebuffer directly?
I’m not necessarily bound to OpenGL.

Thanks!

[dusty_nv]

Hi @BareMetalCoder, you can find various CUDA<->OpenGL interoperability samples under /usr/local/cuda/samples/2_Graphics

There is also API documentation available here:

• https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__INTEROP.html#group__CUDART__INTEROP
• https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__OPENGL.html#group__CUDART__OPENGL

CUDA<->OpenGL interoperability enables you to eliminate extra CPU copy of the data. For reference, this texture class from jetson-utils also does it: https://github.com/dusty-nv/jetson-utils/blob/master/display/glTexture.h

@BareMetalCoder

can I somehow write to the screen’s framebuffer directly?

You might want to look into the Direct Rendering Manager (DRM) from the Jetson Multimedia API - https://docs.nvidia.com/jetson/l4t-multimedia/group__direct__rendering__manager.html

[BareMetalCoder]

Hi Dusty,

Thanks for the info. I’m already using the CUDA interop to copy from my cuda buffers to textures directly – our data never touches CPU memory.

The DRM looks promising, though. Should I be looking at samples/common/classes/NvDrmRenderer.cpp and samples/08_video_dec_drm/video_dec_drm.cpp under the jetson_multimedia_api examples?

[dusty_nv]

@BareMetalCoder

The DRM looks promising, though. Should I be looking at samples/common/classes/NvDrmRenderer.cpp and samples/08_video_dec_drm/video_dec_drm.cpp under the jetson_multimedia_api examples?

I’m not personally familiar with DRM, but it appears the header is found at /usr/src/jetson_multimedia_api/include/NvDrmRenderer.h and the 08_video_dec_drm sample uses it.

The Jetson Multimedia API samples also show the use of EGL for display. I’m not sure how the performance of that compares to OpenGL, but may also be something for you to look into.

[BareMetalCoder]

Thanks, I’m poking through your texture class and there’s more there for me to learn than I thought.
Your examples are always very helpful, well written and appreciated!