Argus captures encoded to JPEG output wrong black levels (16-235) vs. (0-255)

[dustinkerstein]

When capturing still images using Argus (nvarguscamerasrc, nvgstcapture-1.0, etc), the black levels are set incorrectly to video levels (16-235) rather than PC levels (0-255). I tested this with my Nano + IMX477 camera in a dark room with the lens cap on. The following commands all produce the same washed out / low contrast images.

nvgstcapture-1.0 --sensor-mode=0 --image-res=10 --aelock=1 --gainrange='1 1' --ispdigitalgainrange='1 1' --image-enc=1 --mode=1 --whitebalance=2 --exposuretimerange='34000 34000'

nvgstcapture-1.0 --sensor-mode=0 --image-res=10 --aelock=1 --gainrange='1 1' --ispdigitalgainrange='1 1' --image-enc=0 --mode=1 --whitebalance=2 --exposuretimerange='34000 34000'

gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor-mode=0 ispdigitalgainrange='1 1' num-buffers=1 aelock=true wbmode='incandescent' \
gainrange='1 1' exposuretimerange='34000 34000' ! 'video/x-raw(memory:NVMM),width=4032,height=3040,framerate=30/1' ! \
nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvjpegenc quality=80 ! filesink location=nvjpegenc_nvvidconv.jpg

gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor-mode=0 ispdigitalgainrange='1 1' num-buffers=1 aelock=true wbmode='incandescent' \
gainrange='1 1' exposuretimerange='34000 34000' ! 'video/x-raw(memory:NVMM),width=4032,height=3040,framerate=30/1' ! \
nvvidconv ! 'video/x-raw, format=(string)I420' ! jpegenc quality=80 ! filesink location=jpegenc_nvvidconv.jpg

Incorrect Black Levels:

True Black (for reference):

Is there any way to correct this in the Gstreamer pipeline? Let me know if I’m doing something incorrect in my pipeline setup. Regardless, nvgstcapture-1.0 will likely need a code fix. Thanks!

PS - The captured files can all be found here - https://www.dropbox.com/sh/4b0oz8tmu9vm53r/AAB4DRwC7m3n8QytYISCdfuTa?dl=0

[dustinkerstein]

Quick update - Converting to RGB first, and then to JPEG/PNG/TIFF results in correct levels.

gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor-mode=0 ispdigitalgainrange='1 1' num-buffers=1 aelock=true wbmode='incandescent' gainrange='1 1' exposuretimerange='34000 34000' ! 'video/x-raw(memory:NVMM),width=4032,height=3040,framerate=30/1' ! nvvidconv ! 'video/x-raw, format=BGRx' ! videoconvert ! 'video/x-raw, format=RGB' ! avenc_tiff ! filesink location=avenc_tiff.tif

gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor-mode=0 ispdigitalgainrange='1 1' num-buffers=1 aelock=true wbmode='incandescent' gainrange='1 1' exposuretimerange='34000 34000' ! 'video/x-raw(memory:NVMM),width=4032,height=3040,framerate=30/1' ! nvvidconv ! 'video/x-raw, format=BGRx' ! videoconvert ! 'video/x-raw, format=RGB' ! pngenc ! filesink location=pngenc.png

gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor-mode=0 ispdigitalgainrange='1 1' num-buffers=1 aelock=true wbmode='incandescent' gainrange='1 1' exposuretimerange='34000 34000' ! 'video/x-raw(memory:NVMM),width=4032,height=3040,framerate=30/1' ! nvvidconv ! 'video/x-raw, format=BGRx' ! videoconvert ! 'video/x-raw, format=RGB' ! nvjpegenc quality=80 ! filesink location=rgb_nvjpegenc.jpg

However, JPEG in YUV is entirely valid and should contain the necessary color space metadata so a viewer knows how to display it correctly. It seems that piece may be missing. And while the above does provide a workaround, nvgstcapture-1.0 is only able to output JPEG in YUV.

FYI, I’m running L4T 32.6.1

[ShaneCCC]

Looks like it’s know problem for the ISP output.

[dustinkerstein]

Ok. Do you have any more details? Is it documented somewhere publicly? Thanks.

[DaneLLL]

Hi,
In default release YUV420 limited range is supported. You would need to check the source code and do customization to support YUV420 extended range. Please download source code package and check nvarguscamerasrc and nvvidconv plugin. The buffers are all in
NvBufferColorFormat_YUV420 . For supporting YUV420 extended range, would need to modify the format to NvBufferColorFormat_YUV420_ER , and rebuild/replace the plugins.

[dustinkerstein]

Ok, thanks for the info. I’ll give that a try, but in my opinion, Nvidia should strongly consider providing a way to support both Studio Luminance Range and Extended Range (ie. Full Luminance Range) in nvarguscamerasrc and nvvidconv as a user configurable parameter. One could try to infer this setting by whether the user is capturing 1 frame (ie. still photo which definitely should be in Extended Range), or multiple frames (may want to be in Extended Range or Studio Luminance Range). But the choice should really be left to the user. Further, Extended Range should likely be the default as it offers statistically significant additional dynamic range. Studio Luminance Range has a long and somewhat strange history, but it is mainly only used / desired in professional video production. I imagine most people / companies using cameras on Jetson would prefer to to have the extra bits available for increased dynamic range, especially when working with 8bit formats.

[DaneLLL]

Hi,
We will check with teams and evaluate to support more use-cases in future releases. Thanks for the suggestion.