How to convert NV12 from nvargus to RGBA/RGB correctly using VIC or VPI?

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
, ,
1

Device / Software Info

$ cat /etc/nv_tegra_release
R36 (release), REVISION: 4.3, GCID: 38968081, BOARD: generic, EABI: aarch64, DATE: Wed Jan  8 01:49:37 UTC 2025
KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia
$ uname -r
5.15.148-tegra
$ jetson_release
Software part of jetson-stats 4.3.2 - (c) 2024, Raffaello Bonghi
Model: NVIDIA Jetson Orin NX Engineering Reference Developer Kit Super V2Board - Jetpack 6.2 \[L4T 36.4.3\]
NV Power Mode\[0\]: MAXN_SUPER
Serial Number: \[XXX Show with: jetson_release -s XXX\]
Hardware:
 - P-Number: p3767-0000
 - Module: NVIDIA Jetson Orin NX (16GB ram)
Platform:
 - Distribution: Ubuntu 22.04 Jammy Jellyfish
 - Release: 5.15.148-tegra
Libraries:
 - CUDA: 12.6.68
 - cuDNN: 9.3.0.75
 - TensorRT: 10.7.0.23
 - VPI: 3.2.4
 - OpenCV: 4.8.0 - with CUDA: NO

Problem Description

I capture camera frames from nvargus as NV12
Then I do two downstream operations:

  1. Encode JPEG directly from NV12
  2. Convert NV12 to RGBA (or RGB) using VIC or VPI

The JPEG output looks correct, but the RGBA/RGB output is consistently yellow-ish (color cast).

image
image1399×992 242 KB

Because JPEG is correct, I suspect the source NV12 data itself is fine, and the issue may be in color conversion (matrix/range/color format interpretation).

I tried changing the NV12 buffer format from:

  • NVBUF_COLOR_FORMAT_NV12
    to
  • NVBUF_COLOR_FORMAT_NV12_709_ER
  • NVBUF_COLOR_FORMAT_NV12_709

but the yellow cast remains.
I also tried CUDA kernel conversion, and still got yellow-ish RGB output.

Question

What could be the likely root cause here?

Is there any known issue or required setting when converting Argus NV12 to RGBA/RGB via VIC or VPI on JetPack 6.2 / L4T 36.4.3?

Reference Test (GStreamer)

With GStreamer, the output looks good:

gst-launch-1.0 nvarguscamerasrc num-buffers=1 ! \\
'video/x-raw(memory:NVMM),width=2064,height=1552,format=NV12' ! \\
tee name=t \\
t. ! queue ! nvvidconv ! video/x-raw,format=NV12 ! filesink location=nv12.yuv \\
t. ! queue ! nvvidconv ! video/x-raw,format=RGBA ! filesink location=rgba.raw

So the issue seems specific to my conversion path in code rather than camera source data itself.

2

*** Please note that this reply is generated by LLM automatically ***

The issue you’re experiencing with the color cast when converting NV12 to RGBA or RGB using VIC or VPI on JetPack 6.2 (L4T 34.43) is likely due to the color format interpretation or the color conversion matrix. Here are some possible causes and solutions:

  1. Color Format Interpretation: The NV12 format is a YUV 4:2:0 format, where the Y (luminance) plane is separate from the UV (chrominance) plane. When converting NV12 to RGBA or RGB, the color format interpretation might be incorrect, leading to a color cast. Make sure that the color format interpretation is correct in your conversion code.

  2. Color Conversion Matrix: The color conversion matrix is used to convert the YUV color space to the RGB color space. If the color conversion matrix is incorrect, it can lead to a color cast. You can try using a different color conversion matrix or adjust the existing one to see if it resolves the issue.

  3. VIC or VPI Configuration: VIC (Video Image Composer) and VPI (Vision Programming Interface) are NVIDIA’s proprietary APIs for image and video processing. There might be specific configuration or settings required for VIC or VPI to correctly convert NV12 to RGBA or RGB. Check the VIC and VPI documentation to see if there are any specific settings or configurations required for your use case.

  4. JetPack Version: You’re using JetPack 6.2 (L4T 34.43), which might have some known issues or limitations with VIC or VPI. Check the NVIDIA forums or documentation to see if there are any known issues or workarounds for your specific version of JetPack.

  5. GStreamer Output: Since the GStreamer output looks good, it’s likely that the issue is specific to your conversion path in code rather than the camera source data itself. This suggests that the problem is with the VIC or VPI conversion, rather than the camera or the NV12 data.

To troubleshoot the issue, you can try the following:

  • Verify that the NV12 data is correct by displaying it using a tool like nvgstcapture or gst-launch-1.0.
  • Check the VIC and VPI documentation to see if there are any specific settings or configurations required for your use case.
  • Try using a different color conversion matrix or adjust the existing one to see if it resolves the issue.
  • Compare your conversion code with the GStreamer pipeline to see if there are any differences in the color format interpretation or color conversion matrix.

Here’s a comparison of the VIC and VPI APIs with GStreamer:

API Color Format Interpretation Color Conversion Matrix
VIC Proprietary Proprietary
VPI Proprietary Proprietary
GStreamer Standard (YUV 4:2:0) Standard (BT.601 or BT.709)

In summary, the issue is likely due to the color format interpretation or the color conversion matrix in your VIC or VPI conversion code. Try adjusting the color conversion matrix or verifying the color format interpretation to resolve the issue.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

3

can u give me some sample or modify suggestion？

5

Hi,
Please refer to the sample:

/usr/src/jetson_multimedia_api/samples/09_argus_camera_jpeg

You can use VIC by calling createNvBuffer() and copyToNvBuffer(). VIC supports RGBA so you can change the format and get frame data in RGBA.

To get BGR, please refer to the patch:
Making sure you're not a bot!