GRAY8 to NV12 conversation darkens preprocessed tensor

[Chereshnya]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson TX2
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only)
• TensorRT Version 7.1.3
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) question

I have 2 gstreamer pipelines.
First of them uses NV12 as format of input image for nvinfer:

multifilesrc location=/home/data/cher/14373/night_frames_png/frame_%05d.png caps=image/png !
pngdec ! video/x-raw,format=RGB !
videoconvert ! video/x-raw,format=GRAY8 !
nvvideoconvert ! video/x-raw(memory:NVMM),format=NV12,colorimetry=bt601 !
nvinfer name=nvinfer_cd ! ...

The other one uses RGBA:

multifilesrc location=/home/data/night_frames_png/frame_%05d.png caps=image/png!
pngdec ! video/x-raw,format=RGB !
videoconvert ! video/x-raw,format=RGBA !
nvvideoconvert ! video/x-raw(memory:NVMM),format=RGBA,colorimetry=bt709 !
nvinfer name=nvinfer_cd ! ...

Then, I dump preprocessed tensors from nvinfer using code from this topic from both pipelines.

If I compare them, I’ll see, that tensor from pipeline with NV12 is darker than another one from pipeline with RGBA.

|

Too dark frames cause worse car detection.

Is this a problem of NV12? Can I make preprocessed tensors in NV12 pipeline as light as in RGBA using available parameters of nvinfer or nvvideoconvert?

[yuweiw]

1. The colorimetry you used is different.
2. Why did you convert to GRAY8 first in the first pipeline? The losses of this conversion in the 1st pipeline(RGB->GRAY8->NV12 NVMM) is definitely higher than the 2nd(RGB->RGBA->RGBA NVMM).

[Chereshnya]

Ok, I’ve changed my RGBA pipeline to:

multifilesrc location=/home/data/cher/14373/night_frames_png/frame_%05d.png caps=image/png !
pngdec ! video/x-raw,format=RGB !
videoconvert ! video/x-raw,format=GRAY8 !
videoconvert ! video/x-raw,format=RGBA ! 
nvvideoconvert ! video/x-raw(memory:NVMM),format=RGBA,colorimetry=bt601 !
nvinfer name=nvinfer_cd ...

Now, it uses equal colorimetry and converts frame to GRAY8 first.
But results are the same. Using RGBA, I get light preprocessed tensors, while NV12 have dark tensors
Is this a problem of NV12 compression?

[yuweiw]

Yes. The NV12 is compressed relative to RGBA, so there may be losses of accuracy.

[Chereshnya]

Got it, thanks)