we are converting NV12_709 images decoded by Gstreamer nvv4l2decoder to BGRA images using both VPI
NvBufSurfTransform with VIC backend. We noticed the converted RGB color is different from the result we got by using NPP
The test video we are using is just 10 secs of frames completely filled with RGB color (126,126,126). In the NV12_709 decoded buffer we can see the pixels have the expected YUV value of (124,128,128). After the conversion to BGRA we got RGB values of (138,138,138) using VPI/NvBufSurfTransform with VIC and RGB values of (125,125,125) using nppiNV12ToBGR_709CSC_8u_P2C3R.
Why this difference? It seems the NPP function is performing the conversion correctly while VPI/NvBuf is a little off.
We are missing something? Could you help us understand what’s wrong?
EDIT: attached test video
grey.zip (1.9 KB)
NPP uses CUDA to do the conversion.
Could you try to use the CUDA backend for VPI to see if the results become closer?
sadly the CUDA backend from VPI desent support NV12 Limited Range, only NV12 Extended Range (see documentation: https://docs.nvidia.com/vpi/group__VPI__ConvertImageFormat.html#gaae47f1e840253de76af91a4f6aea9b71) so we cannot perform the test you requested.
In the previous post i forgot to mention the source image format is NV12_709 Limited Range which seems only supported by VIC backend and NvBufSurfTransform.
Any ideas? Anyone else is experiencing this color conversion issue? Converting a limited range YUV value of (124,128,128) to RGB (138,138,138) seems wrong according to standard conversion formulas…
we further experimented with this issue. We created an artificial NV12_709 Limited Range image with pixel on the X axis with values ranging from [16,128,128] to [235,128,128]. The image in RGB is an horizontal grayscale gradient from 0 to 255.
We then converted this image to RGB with NPP and VPI/NvBufSurfTransform with VIC backend and printed and plotted the R value obtained (G and B values are the same since its a grayscale image). This is what we got:
npp_cuda.txt (6.5 KB)
vpi_vic.txt (7.9 KB)
NPP output a linear transformation from NV12 to RGB while VPI/NvBufSurfTransform VIC seems not linear. What kind of conversion maths is applied inside the VIC to perform this conversion? We’re not expert in this area but we want to have a clear understanding on what’s going on. NPP results seems consistent to the standard conversion formulas we found while researching this issue, while VIC not.
Thanks for sharing the details.
We need to discuss this with our internal team.
Could you share which VPI/JetPack version you use?
JetPack 5.1.2 and the VPI version bundled with it, should be VPI 2.3.
Thanks for the info.
While we are checking this with the internal team, would you mind sharing a reproducible source so we can test it in our environment?
yes, we can create a small reproducible example and share the code here. We’re pretty busy until friday (deadline looming), so it may take a while to produce the example. We hope to share the source by the end of the week.
Thank a lot!
Thanks, we will also update here if get further info about this issue.
here is the source code of a simple example:
color_conversion.zip (1.7 KB)
It reproduces the issue as described in the previous posts, it also prints to console the converted values and plot the two curves.
Please keep us updated, thanks!
Thanks for the source code.
Will let you know if we get any feedback from our internal team.
do you have any update from you internal team?
Sorry that our internal team is still checking this.
Will let you know once we get any feedback.
Sorry that our internal team needs some further checks about this issue.
So delete the comment and will update later after the checking is done.
thanks for the answer. We’ll check what you suggested and let you know.
Since in production code we’re using NvBufSurfTransform (on VIC backend) function to perform color conversions AND resize/crop in one step how we can apply your suggestion in this scenario? We couldn’t find anything similar to VPI color spec in the NvBufSurface object…there’s only the NvBufSurfaceColorFormat field and (for RGBA we can only use NVBUF_COLOR_FORMAT_BGRA).
Sorry that we have updated the comment above.
Please wait for our internal team for further info.