Nvvidconv RGB to YUV conversion colors incorrect

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
,
1

Hello,

We have an issue with accelerated video conversion using nvvidconv gstreamer component. When converting RGB video to NV12 and then h264 encoding it, the colors in the video appear too dark. This does not happen when using software color space conversion using videoconvert gstreamer component.
I created a small reproducer that shows the issue, using a gray gradient test pattern:

nvvidconv:
gst-launch-1.0 videotestsrc pattern=gradient ! “video/x-raw, width=64, height=256, format=BGRx” ! nvvidconv ! “video/x-raw,width=64,height=256,format=NV12” ! imagefreeze num-buffers=1 ! filesink location=nvvidconv.raw

videoconvert:
gst-launch-1.0 videotestsrc pattern=gradient ! “video/x-raw, width=64, height=256, format=BGRx” ! videoconvert ! “video/x-raw,width=64,height=256,format=NV12” ! imagefreeze num-buffers=1 ! filesink location=videoconvert.raw

Since the conversion is to limited range, the expectation is that the Y values of the result are linearly increasing from 16 to 235. With videoconvert this is happening, but with nvvidconv some non-linear curve can be observed. It seems like the nvvidconv is adding some sort of gamma correction?

nvvidconv
nvvidconv955×707 36 KB

A few questions:

  • Is a model available of the calculations the nvvidconv does?
  • Can this gamma effect be turned off, so that the conversion is linear?
3

Hi,
Please share which Jetpack version you are using. So that we can check and suggest next.

4

Hi, we are using Jetpack 5.1.2

5

Hi,
Please try

  1. Download source code of nvvidconv plugin from:

Jetson Linux 35.4.1 | NVIDIA Developer
Driver Package (BSP) Sources

  1. Apply the code to gst_nvvconv_get_pix_fmt():
      case GST_VIDEO_FORMAT_NV12:
        *pix_fmt = NVBUF_COLOR_FORMAT_NV12;
        *isurf_count = 2;
        if (info->colorimetry.range == GST_VIDEO_COLOR_RANGE_0_255) {
          if (info->colorimetry.matrix == GST_VIDEO_COLOR_MATRIX_BT709) {
            *pix_fmt = NVBUF_COLOR_FORMAT_NV12_709_ER;
          }
          else
            *pix_fmt = NVBUF_COLOR_FORMAT_NV12_ER;
        }
        else if (info->colorimetry.matrix == GST_VIDEO_COLOR_MATRIX_BT709) {
            *pix_fmt = NVBUF_COLOR_FORMAT_NV12_709;
        }
        else if (info->colorimetry.matrix == GST_VIDEO_COLOR_MATRIX_BT2020) {
            *pix_fmt = NVBUF_COLOR_FORMAT_NV12_2020;
        }
        break;
  1. Rebuild the plugin
6

Thank you. This actually solved another issue I was looking into where nvvidconv would convert to bt601 even though the caps were set to bt709.

Unfortunetely it does not solve the non-linear gamma curve, this still looks exactly the same.

7

Hi,
Both plugins are implemented through NvBufSurfTransform(), so it’s supposed to work identically if formats are identical. A bit strange you still observe the deviation. Are you able to try Jetpack 5.1.4? Would be great if you can help try latest Jetpack 5 version.

8

Hi,
Please try this:

  1. Get source code of nvvidconv of Jetpack 6.1(r36.4):

Jetson Linux | NVIDIA Developer

  1. Compile it on Jetpack 5.1.2
  2. Overwrite the so file:
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvidconv.so

There are certain fixes in later releases. Please apply it to previous version and see if it helps.

9

Hello,

I just compiled the nvvidconv from Jetpack 6.1 and tried it, but it does not solve the issue, the curve still looks the same.

11

Hi,
Please try

  1. Use nvvidconv of r36.4 on Jetpack 5.1.2
  2. Apply attached lib to 5.1.2(please backup original lib before replacement)
$ md5sum libnvvic.so
f30126f43fb4f60b20a187f6908912bf *libnvvic.so

r35_x_TEST_libnvvic.zip (39.8 KB)

12

Thanks, this indeed solves the issue!
Will it be applied in a future jetpack release?

13

Hi,
If you use Xavier, please apply it if you would like to upgrade to later Jetpack 5.1.4. If you use Orin, please use Jetpack 6.1, which version shall include the fix.