I am experimenting with the Mapbox GL SDK, and their renderer creates its own GLES3.1 context through EGL and renders to its own FBO. I need to embed this renderer into our app which uses Desktop GL 4.6 for rendering, on desktop Linux x86_64 systems with high-end nVidia GPUs.
I am looking for ways to quickly copy (remaining on GPU) a texture (FBO color attachment) from the Mapbox GLES context to a texture of the same format in our GL 4.6 context, so that I can use it as a layer in our render.
First I tried using CUDA Graphics Interop to access both textures and do the copy, but CUDA on Linux x86_64 does not seem to support registering images from a GLES context (presumably that only works on mobile platforms like Tegra/Jetson).
I have been experimenting with EGLImage, and made some progress, but I’d like to know whether this is a viable solution, as frankly I hadn’t heard of it before today, and I fear that it, too, may be a mobile/GLES-only solution, despite what the EGL extension doc implies.
Please advise, thanks!
Simon Eves
OmniSci, Inc.
Here is my attempt at aliasing a GLES FBO color attachment texture to a GL texture using EGLImage.
The mbgl::BackendScope lines make the GLES context current. The GLFW calls do the same for the GL context, obviously.
I had to manually fetch the function pointer for glEGLImageTargetTexture2DOES as it didn’t seem to be bound in my app (I guess the GL functions I’m calling come through GLFW/glad).
Everything seems to execute correctly with valid-looking results (handles, pointers, no GL or EGL error codes) but when I try to render with the resulting texture on the GL side, I just get black.
void MapboxApplication::_UpdateMapboxAliasTexture()
{
// get Mapbox FBO color attachment texture
GLuint hMapboxColorTexture = GL_NONE;
{
mbgl::BackendScope guard { *_pFrontend->getBackend() };
hMapboxColorTexture = _pFrontend->getFramebufferColorTextureGLName();
}
// size or Mapbox texture changed?
if (hMapboxColorTexture != _hMapboxColorTexture || _appContext.windowWidth != _uiWidth || _appContext.windowHeight != _uiHeight) {
// destroy any existing EGLImage
// @CONCERN will EGL get in a tizzy if Mapbox deleted the
// previous texture already due to a renderer resize?
if (_hMapboxEGLImage) {
mbgl::BackendScope guard { *_pFrontend->getBackend() };
EGL_CHECK_ERROR(eglDestroyImage(eglGetCurrentDisplay(), (EGLImage)_hMapboxEGLImage));
_hMapboxEGLImage = nullptr;
}
// destroy any existing alias texture
if (_hMapboxAliasTexture) {
glfwMakeContextCurrent(_pWindow);
GL_CHECK_ERROR(glDeleteTextures(1, &_hMapboxAliasTexture));
_hMapboxAliasTexture = GL_NONE;
glfwMakeContextCurrent(nullptr);
}
// new Mapbox texture valid?
if (hMapboxColorTexture) {
// make new EGL image
{
mbgl::BackendScope guard { *_pFrontend->getBackend() };
const EGLAttrib attrs[] = {
EGL_GL_TEXTURE_LEVEL, 0,
EGL_IMAGE_PRESERVED, EGL_FALSE,
EGL_NONE
};
auto ecb = reinterpret_cast<EGLClientBuffer>(static_cast<size_t>(hMapboxColorTexture));
_hMapboxEGLImage = EGL_CHECK_ERROR(eglCreateImage(eglGetCurrentDisplay(), eglGetCurrentContext(), EGL_GL_TEXTURE_2D, ecb, attrs));
CHECK(_hMapboxEGLImage);
}
// lazy fetch of function pointer for glEGLImageTargetTexture2DOES
if (glEGLImageTargetTexture2DOES == nullptr) {
glfwMakeContextCurrent(_pWindow);
glEGLImageTargetTexture2DOES = (PFNGLEGLIMAGETARGETTEXTURE2DOESPROC)glfwGetProcAddress("glEGLImageTargetTexture2DOES");
CHECK(glEGLImageTargetTexture2DOES);
glfwMakeContextCurrent(nullptr);
}
// make new alias texture
{
glfwMakeContextCurrent(_pWindow);
GL_CHECK_ERROR(glGenTextures(1, &_hMapboxAliasTexture));
CHECK(_hMapboxAliasTexture != GL_NONE);
GL_CHECK_ERROR(glBindTexture(GL_TEXTURE_2D, _hMapboxAliasTexture));
GL_CHECK_ERROR(glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST));
GL_CHECK_ERROR(glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR));
GL_CHECK_ERROR(glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, _hMapboxEGLImage));
GL_CHECK_ERROR(glBindTexture(GL_TEXTURE_2D, GL_NONE));
glfwMakeContextCurrent(nullptr);
}
}
// store new values
_hMapboxColorTexture = hMapboxColorTexture;
_uiWidth = _appContext.windowWidth;
_uiHeight = _appContext.windowHeight;
}
}
Having just read what it actually does, I switched EGL_IMAGE_PRESERVED from EGL_FALSE to EGL_TRUE, but it made no difference.
Again, this is running on Linux x86_64 (Ubuntu 18.04) with an RTX2080 and driver 410.73.
The GL context is 4.6 created with GLFW/EGL, and the GLES context is 2.0 created with EGL by Mapbox’s code, although it’s all in the same process.
I ended up removing Mapbox from the equation and writing a simpler test app that constructs a second GL or GLES context with EGL, renders in that to an FBO, and then uses the EGLImage technique to pull the color attachment back to the main GLFW GL context for rendering.
This worked, certainly when the second context was also GL. I tried making the second context be GLES, but I don’t think I was doing that correctly, as EGL was reporting it as ES 1.1, even though it was happily rendering with a fragment shader. In fact, I think it was actually still GL, as I tried using a GL4.5+ texture creation function and that still worked.
I then changed tack to using the External Objects extensions in GLES and Vulkan to pull the Mapbox texture out to Vulkan for rendering on our side, and that worked fine (GLES and EGL were hidden away in the Mapbox static lib, and my app had no dependencies on either, only Vulkan for rendering, and GLFW for windowing and event handling).
I’d still like to know if the EGLImage technique is viable between a GLES context and a GL context on a desktop Linux x86_64 platform, as we may still need this to work with GL in the short term.
Hoping for an answer or an opinion soon. Thanks.
It’s the driver forum so you can’t expect an experienced left alone educated answer here but I’ll give you an opinion.
Looks that EGLImage is a really misterious extension. In theory, it should provide a way to share data between apis. The way you walked crossing glEGLImageTargetTexture2D(OES) is always mentioned to be taken. In theory. In practice, EGLImage AFAICT is only being used so far as sharing data between contexts.
The contacts for more information are given in the Khronos specifications.
The specific implemention is surely interesting for more groups regarding va-api and zerocopy in general so thank you for documenting your investigations.
BTW the gl/gameworks forums consist mainly of questions, not answers.
Addendum: maybe ask the Angle people if they use it or have some experience with it.
Thanks for your feedback. I realize this is the driver forum, but I was kinda hoping that a driver person would respond on the basic question of whether I should expect it to work between GL and GLES on Linux x64 with the current nVidia drivers.
I have proven since my original post that it works between GL and GL, as I had expected, but I’d like to know if GLES-to-GL is also expected to work before I spend time re-writing my test app to try it.
The question is kinda moot, though, because our app side is going to be Vulkan not GL, and I already proved that works through the External Object extensions instead.
I don’t believe EGLImage is defined to do anything in OpenGL contexts. The “OES” extension implies the related functions only operate in GLES contexts, so glEGLImageTargetTexture2DOES() may be a no-op. The OES_EGL_image extension spec only defines interactions with GLES specifications. Calling functions from unsupported extensions produces undefined behavior. That said, it might just work too. I wasn’t able to convince myself for sure it won’t. For it to have a chance of working, you’d have to be using EGL for both the GLES and GL 4.6 contexts though. Is GLFW using GLX or EGL?
One way you might get this to work is using an X pixmap as a poor-man’s interop mechanism: Create an EGLImage from the pixmap in the EGL/GLES context, and a GLX pixmap or another EGLImage from the pixmap in the GL 4.6 context. If using GLX, use GLX_EXT_texture_from_pixmap to read from it. If you’re using EGL for the 4.6 context, you might run into the same “EGLImage doesn’t work in desktop GL” issue though.
There are two other options, and they’re probably better, but they may be more involved:
-Create your images in Vulkan and import them to both GLES and GL using Vulkan<->GL/GLES interop, which is known to work in both OpenGL 4.6 and GLES 2/3/etc using either EGL or GLX. You don’t need to do anything in Vulkan other than allocate the textures & their memory, then export them for use in GL. This is the most flexible solution.
-If you’re using EGL to manage both the GLES and GL contexts, use this new extension to bind the the EGLImage to immutable textures in GL & GLES, which is explicitly supported in OpenGL 4.x:
https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_EGL_image_storage.txt
The latter is only supported by very recent drivers, so as always, be sure to check the extension string in all contexts for support before using it. Presence of the function as returned by eglGetProcAddress/glXGetProcAddress does not imply support, and in fact, recent versions of our driver will never return NULL from these functions.
Hi James,
The glEGLImageTargetTexture2DOES workflow definitely works from GL to GL, despite that function being part of an ES extension. We have used it before in our product for copying images between contexts on different GPUs, although that was short-lived and was replaced with a CUDA-based approach.
I was indeed using EGL for the GLFW-created GL context and Mapbox uses EGL to create its ES context. I am building Mapbox into a static lib, so its linking with EGL and ES should already be done. I was then linking EGL separately into my app so I could call the functions, the GL functions already being provided by GLFW and Glad.
When I tried to simplify and remove Mapbox from the equation, I think something was getting confused in the link because the GL functions that needed to go to the GLES context would also have been linked to the same symbol set as those intended for the GL context. Same for EGL, although that’s probably OK for that. Anyway, even though I was trying to create a separate ES context, it was being reported as ES 1.1 (unlike the real Mapbox one which reports correctly as 3.2), and there was no rejection or errors throw on using fragment shaders or GL 4.5+ texture functions (I tried that deliberately to try to prove or disprove the “ES-ness” of that contest). Since it didn’t fail, I decided that the code structure was flawed and was never going to be helpful.
I’ll try that new extension, for sure, although we probably won’t need GL-to-ES operation anyway since we’re going to be writing our side against Vulkan anyway, and that works fine using the External Object stuff.
Thanks for getting back to me, anyway! :)
OK, having now followed your link to that EXT_EGL_image_storage spec, I realize that I already found and tried that, and couldn’t get it to work. I think I then determined that it wasn’t actually supported by the driver I am using (currently 410.78).
That deduction was based on manual extension search, code which I have since abandoned, but certainly glxinfo on the command line does not report that one as supported.
This was after I had initially failed to get the glEGLImageTargetTexture2DOES route to work too. Then I went back and realized my mistake in that, and it sprang into life, but still only for GL-to-GL, as I said above.
Never mind. Vulkan to the rescue. It’s all good.
Note glxinfo will never report that extension as supported, because GLX doesn’t support EGLImages, regardless of GLES Vs GL. You’d have to check eglinfo. This relates to my question about GLFW’s choice of window system bindings in your usage.
Ah. Oops. OK, I installed eglinfo and that doesn’t report it either. The function pointer fetch still worked (as you said that it would) and calling it did not crash, but did nothing.
All moot. We’re good. Thank you! :)