Dear Experts,
I’m a little surprised to not find any OpenGL ES extensions for YCrCb (aka YUV) textures on the Nano. Am I missing something?
I have code from iOS that decodes JPEGs into YCrCb textures using an Apple extension; this saves quite a lot of texture memory compared to decoding to RGB, with no loss of image quality as the JPEGs are YCrCb internally anyway.
(Would I be better asking this in an OpenGL forum?)
Thanks, Phil.
Hi,
After installation through SDKManager, you would see samples in
/usr/src/nvidia/graphics_demos
Please check if the desired format is defined in
/usr/src/nvidia/graphics_demos/include/EGL/eglext.h
Hi,
One EGL extension that is present is EGL_NV_stream_consumer_gltexture_yuv. Reading the description of that gives me some idea of how, I think, we’re expected to handle YUV textures. That extension allows a YUV EGL stream to be consumed by up to three OpenGL textures: one texture with YUV mapped to RGB, two textures with Y in the first and UV in the second, or three textures for each component individually. (That seems to correspond to 4:4:4, 4:2:2 and 4:2:0 subsampling).
I guess that tells me that there isn’t anything specific for decoding this in OpenGL, and I would need to create two or three textures for each of my JPEGs, larger for Y and smaller for U and V, and sample each of them separately.
Thanks, Phil.
Hi,
For reference, please share your release version( $ head -1 /etc/nv_tegra_release ). And are the JPEGs decoded into YUV 422 or YUV 420?
The board is running JetPack 4.4 installed over Debian Buster. The nv_tegra_release says
R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t210ref, EABI: aarch64, DATE: Fri Jun 26 04:38:25 UTC 2020
SInce you ask… As I understand it, JPEG is essentially a 4:2:0 format, i.e. each 2x2 block of pixels has four Y, one U and one V value. My iOS code tells libjpeg to output YCbCr and it gives me Y, U and V values for every pixel, which looks like 4:4:4, i.e. libjpeg has upsampled. I then downsample by discarding alternate U and V values horizontally to get YU YV, which is 4:2:2 since there is no vertical downsampling. This is what the Apple YCrCb texture format wants. The result is 16 bits per pixel, with better quality than the alternative 16 bit choice, RGB 565. If there were a way to also downsample vertically it would be 12 bits per pixel with the same quality.
Anyway, this isn’t particularly important; I’m not currently limited by RAM so I can just use RGB for everything.
Hi,
Not sure but maybe the Apple extension follows
https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_YUV_target.txt
This is an optional extension and we currently do not support it.
For supporting YUV in OpenGL on Jetson Platforms, you can create an EGL Image for YUV buffer and use glEGLImageTargetTexture2DOES() to bind that image to a OpenGL texture with GL_TEXTURE_EXTERNAL_OES Type. The YUV to RGB conversion happens internally in fragment shader.
Very interesting, thanks!