Converting from yuv_420_888 to RGB

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX1
1

I’m working with a TX1 and the Argus software library. I’m noticing that the default pixel format is YCbCr and there doesn’t appear to be an option for RGB. Is there some built-in function that allows me to convert this to RGB?

If there isn’t a built-in way to supply data in RGB, are the conversion coefficients listed somewhere? There exists some matrix operation that would allow the user (me) to convert the YCbCr data into RGB. (The writeJPEG function already does this implicitly). Unfortunately, there are so many different variants of YCbCr (sometimes confused with YUV and YPbPr), that only the Argus creators would know what they used.

My best option right now would be to reverse-engineer the writeJPEG function to determine the conversion coefficients, which is possible, but takes some time and I’m sure others will have this question in the future.

2

Hi EEG,
There are two samples you can refer to
[11_camera_object_identification]
Argus → NV12 → NvVideoConverter → RGBA → openCV → caffe
[backend]
h264 stream → NvVideoDecoder → NV12 → NvVideoConverter → RGBA → CUDA → BGR → TensorRT

You can integrate the two parts into your usecase.

3

Thanks!

Curious, is there any intention of adding native support for RGB in the future?

I.e. is there any plan to add an RGB format to this list in Types.h?

DEFINE_UUID(PixelFormat, PIXEL_FMT_UNKNOWN,       00000000,93d5,11e5,0000,1c,b7,2c,ef,d4,1e);
DEFINE_UUID(PixelFormat, PIXEL_FMT_Y8,            569be14a,93d5,11e5,91bc,1c,b7,2c,ef,d4,1e);
DEFINE_UUID(PixelFormat, PIXEL_FMT_Y16,           56ddb19c,93d5,11e5,8e2c,1c,b7,2c,ef,d4,1e);
DEFINE_UUID(PixelFormat, PIXEL_FMT_YCbCr_420_888, 570c10e6,93d5,11e5,8ff3,1c,b7,2c,ef,d4,1e);
DEFINE_UUID(PixelFormat, PIXEL_FMT_YCbCr_422_888, 573a7940,93d5,11e5,99c2,1c,b7,2c,ef,d4,1e);
DEFINE_UUID(PixelFormat, PIXEL_FMT_YCbCr_444_888, 576043dc,93d5,11e5,8983,1c,b7,2c,ef,d4,1e);
DEFINE_UUID(PixelFormat, PIXEL_FMT_JPEG_BLOB,     578b08c4,93d5,11e5,9686,1c,b7,2c,ef,d4,1e);
DEFINE_UUID(PixelFormat, PIXEL_FMT_RAW16,         57b484d8,93d5,11e5,aeb6,1c,b7,2c,ef,d4,1e);
4

Hi EEG, the HW cannot support the format conversion. We will closely review any new HW function, so you may not see it in near future.

5

Hi,

It is possible to convert PIXEL_FMT_YCbCr_420_888 to RGB ?
I could not convert “mapBuffer” to rgb data.
Perhaps have someone any example, how can I do it correctly or any Ids what is wrong ?

Write to JPEG works.

…

EGLStream::Image *image = iFrame->getImage();
if (!image)
{
setMsg(“Failed to get Image from interface”);
return ;
}

if (isRec) // Recording Image
{
EGLStream::IImageJPEG *iImageJPEG = Argus::interface_castEGLStream::IImageJPEG(image);
if (!iImageJPEG)
{
setMsg(“Failed to get ImageJPEG Interface”);
return ;
}
status = iImageJPEG->writeJPEG(“test.jpg”);
if (status != Argus::STATUS_OK)
{
SET_MSG(DEB_LVL_LOW, "Failed write jpeg);
}
}

// //////////////////////////
//
// Prepare Preview Image
//
// //////////////////////////

EGLStream::IImage *iImageData = Argus::interface_castEGLStream::IImage(image);
if (!iImageData)
{
setMsg(“Failed to get IImage Interface”);
return ;
}

size = iImageData->getBufferSize();

int width = 640;
int height = 480;
int nSize = width * height * 3;

char *rgb = new char[nSize];
memset(rgb, 255, nSize);

yuv_to_rgb((char*)iImageData->mapBuffer(), rgb, width, height); // ! Conversion not work !

…

I have try at less with 11 Versions, but still something doesn’t work and my Preview Images look like bad.
A few decode samples:

// PIXEL_FMT_YCbCr_420_888

// VER 1

int yuv2rgb(int Y, int Cb, int Cr, int* r, int* g, int* b)
{
unsigned int r_tmp, g_tmp, b_tmp;
unsigned char r_t, g_t, b_t;

// ORG

// r_tmp = (Y + 1.402 * (Cr - 128));
// g_tmp = (Y - 0.34414 * (Cb - 128) - 0.71414 * (Cr - 128));
// b_tmp = (Y + 1.772 * (Cb - 128));

// T2

r_tmp = (Y + 1.402 * (Cr));
g_tmp = (Y - 0.34414 * (Cr) - 0.71414 * (Cb));
b_tmp = (Y + 1.772 * (Cb));

// T3

// r_tmp = (Y + 1.402 * (Cr));
// g_tmp = (Y - 0.34414 * (Cr) - 0.71414 * (Cb));
// b_tmp = (Y + 1.772 * (Cb));

// T1

// r_tmp = (Y + 1.370705 * (Cr));
// g_tmp = (Y - 0.698001 * (Cr) - 0.337633 * (Cb));
// b_tmp = (Y + 1.732446 * (Cb));

r_t =  MAX (0, MIN (255, r_tmp));
g_t =  MAX (0, MIN (255, g_tmp));
b_t =  MAX (0, MIN (255, b_tmp));

// r_t = (unsigned char)(Y + 1.403Cr);
// g_t = (unsigned char)(Y - 0.344Cb - 0.714Cr);
// b_t = (unsigned char)(Y + 1.770Cb);

*r = r_t;
*g = g_t;
*b = b_t;

return 0;

}

void yuv_to_rgb(char* yuv, char* rgb, int width, int height)
{
int y, cb, cr;
int r, g, b;
int i = 0;
char* p_y;
char* p_u;
char* p_v;
char* p_rgb;
int chroma_v = 2;
int chroma_h = 4;
int cbcr_width;
int cbcr_height;
int cbcr_size;

cbcr_width  = width * (chroma_h / 4.0);
cbcr_height = height * (chroma_v / 4.0);
cbcr_size   = cbcr_width * cbcr_height;

p_y = yuv;
p_u = p_y + width * height;
p_v = p_u + cbcr_size;
p_rgb = rgb;

for (i = 0; i < cbcr_size; i++)
{
    y  = p_y[0];
    cb = p_u[0];
    cr = p_v[0];
    yuv2rgb(y, cb, cr, &r, &g, &b);
    p_rgb[0] = r;
    p_rgb[1] = g;
    p_rgb[2] = b;

    y  = p_y[1];
    cb = p_u[0];
    cr = p_v[0];
    yuv2rgb(y, cb, cr, &r, &g, &b);
    p_rgb[3] = r;
    p_rgb[4] = g;
    p_rgb[5] = b;

    p_y += 2;
    p_u += 1;
    p_v += 1;
    p_rgb += 6;
}

}

// VER 2

void yuv420_to_rgb24_3(char yuv420, char rgb24, int width, int height)
{
char p_y;
char p_u;
char* p_v;
int R,G,B,Y,U,V;
int x,y;
int nWidth = width>>1;

p_y = yuv420;
p_u = p_y + width * height;
p_v = p_u + width * height / 4;

for (y=0;y<height;y++)
{
    for (x=0;x<width;x++)
    {
        Y = *(p_y + y*width + x);
        U = *(p_u + ((y>>1)*nWidth) + (x>>1));
        V = *(p_v + ((y>>1)*nWidth) + (x>>1));
        /*
        R = Y + 1.402*(V-128);
        G = Y - 0.34414*(U-128) - 0.71414*(V-128);
        B = Y + 1.772*(U-128);
        */

        yuv2rgb(Y, U, V, &R, &G, &B);

        if (R>255)R=255;
        if (R<0)R=0;
        if (G>255)G=255;
        if (G<0)G=0;
        if (B>255)B=255;
        if (B<0)B=0;

        *(rgb24 + (y*width + x)*3) = R; 
        *(rgb24 + (y*width + x)*3 + 1) = G;
        *(rgb24 + (y*width + x)*3 + 2) = B; 
    }
}

}

// VER 3

void yuv_to_rgb_ver6(char* yuv, char* rgb, int width, int height)
{

uint8_t size = width * height;
for (int h = 0; h < height; h++)
{
    for (int w = 0; w < width; w++)
    {
        // VER 1

// uint8_t y = yuv[h * width + w];
// uint8_t u = yuv[(uint)((h * 0.5) * (width * 0.5) + (w * 0.5) + size)];
// uint8_t v = yuv[(uint)((h * 0.5) * (width * 0.5) + (w * 0.5) + size + (size / 4))];

// uint8_t r = 0;
// uint8_t g = 0;
// uint8_t b = 0;

// yuv2rgb_ver6(y, u, v, &r, &g, &b);

// rgb[0] = r;
// rgb[1] = g;
// rgb[2] = b;

// rgb += 3;

        // END VER 1

        // VER2

        int Y = yuv[h * width + w] & 0xFF;
        int U = yuv[(h / 2) * (width / 2) + (w / 2) + size] & 0xFF;
        int V = yuv[(h / 2) * (width / 2) + (w / 2) + (size / 4)] & 0xFF;

        U = U - 128;
        V = V - 128;
        int R,G,B;
        R = (int)(Y + 1.402 * V);
        G = (int)(Y - 0.34414 * U - 0.71414 * V);
        B = (int)(Y + 1.772 * U);

        R = MAX (0, MIN (255, R));
        G = MAX (0, MIN (255, G));
        B = MAX (0, MIN (255, B));

        rgb[h*4*width+w*4  ] = (char) R;
        rgb[h*4*width+w*4+1] = (char) G;
        rgb[h*4*width+w*4+2] = (char) B;
        rgb[h*4*width+w*4+3] = 255;
    }
}

}

etc …

Thx for any help.