Nvcamerasrc colors difference between 3 cameras

enrique.ramirez · May 17, 2018, 4:49pm

Hi,

I am using nvcamerasrc to capture from 3 IMX274 cameras with the driver provided by leopard imaging (kernel 4.4), working on a TX2 using Jetpack 3.1.

There is an issue with the resultant images because they present different exposures. I tried to adjust the nvcamerasrc properties disabling white balance and auto-exposure, varying color-effect, contrast and saturation but there still variation between the images captured from the three cameras.

This is the pipeline used:

gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, framerate=(fraction)30/1' ! omxh264enc ! mpegtsmux ! filesink location=test_0_1080p.ts nvcamerasrc sensor-id=1 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, framerate=(fraction)30/1' ! omxh264enc ! mpegtsmux ! filesink location=test_1_1080p.ts nvcamerasrc sensor-id=2 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, framerate=(fraction)30/1' ! omxh264enc ! mpegtsmux ! filesink location=test_2_1080p.ts

It seems that there is a problem with the configuration file in the ISP that is not working in the same way for the three sensors. We also have noticed with other cameras that the position (bottom, center, top) on tegra-camera-platform affects the colors of the images. the center camera is the one exposing the difference.

Have you ever experienced something like this? Do you know if the ISP configuration depends on the position configured on tegra-camera-platform and how to avoid this problem?

Thanks in advance for your help

Regards,
-Enrique

SimonZhu · May 18, 2018, 5:05am

Hi EnriqueR,

Do you mean 3 cameras present different exposures even they point to the same scene?
Could you provide 3 images (with different exposures) to us?

BTW, we have a new driver for Jetpack 3.2 on Nvidia TX2 Developer kit. You can give it a try if needed.

enrique.ramirez · May 21, 2018, 2:55pm

Thanks, we are going to generate the samples for you to analyze what we are seeing. In the meantime, I have some questions about the ISP configuration:

How is it configured for each camera?

Does the configuration and ISP calibration depend on the position of the camera (bottom, front, center in the tegra-camera-platform node in the DTB)?

If that is the case, could you provide a configuration file with the same settings for all the cameras?

Thanks.

xingxing · May 22, 2018, 1:50am

if you just put one isp file named camera_overrides.isp in /var/nvidia/…/setting/ folder, all cameras from different positions will use this isp file. They should behaviour same if the lens and senor are same.

if you want sensors from different positions use different isp file. you will need several isp files in /var/nvidia/…/setting/ folder. and the name should be same as the position name.
for example:
ov9732_centerleft_P5V27C.isp ov9732_centerright_P5V27C.isp ov9732_topleft_P5V27C.isp

                 module1 {
                         badge = "ov9732_topleft_P5V27C";
                         position = "topleft";
                         orientation = "0";
                         status = "okay";
                         drivernode0 {
                                 /* Declare PCL support driver (classically known as guid)  */
                                 pcl_id = "v4l2_sensor";
                                 /* Driver's v4l2 device name */
                                 devname = "ov9732 2-0036";
                                 /* Declare the device-tree hierarchy to driver instance */
                                 proc-device-tree = "/proc/device-tree/i2c@7000c500/ov9732_f@36";
                                 status = "okay";
                         };
                 };

                 module2 {
                         badge = "ov9732_centerright_P5V27C";
                         position = "centerright";
                         orientation = "0";
                         status = "okay";
                         drivernode0 {
                                 /* Declare PCL support driver (classically known as guid)  */
                                 pcl_id = "v4l2_sensor";
                                 /* Driver's v4l2 device name */
                                 devname = "ov9732 6-0036";
                                 /* Declare the device-tree hierarchy to driver instance */
                                 proc-device-tree = "/proc/device-tree/host1x/i2c@546c0000/ov9732_e@36";
                                 status = "okay";
                         };
                 };
                 module3 {
                         badge = "ov9732_centerleft_P5V27C";
                         position = "centerleft";
                         orientation = "0";
                         status = "okay";
                         drivernode0 {
                                 /* Declare PCL support driver (classically known as guid)  */
                                 pcl_id = "v4l2_sensor";
                                 /* Driver's v4l2 device name */
                                 devname = "ov9732 6-0010";
                                 /* Declare the device-tree hierarchy to driver instance */
                                 proc-device-tree = "/proc/device-tree/host1x/i2c@546c0000/ov9732_c@10";
                                 status = "okay";
                         };

                 };

enrique.ramirez · May 23, 2018, 11:08pm

Thanks for your answers!

The problem we are facing, is that we want to capture a panoramic view using 3 cameras pointing in different directions.

We have been doing tests varying the nvcamerasrc properties in order to disable automatic features (auto-white-balance, auto-exposure) to find the same colors for the 3 images captured.

This is the reference pipeline to do the tests:

gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" intent=3 queue-size=30 scene-mode=12 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh264enc ! mpegtsmux ! filesink location=test_0.ts nvcamerasrc sensor-id=1 fpsRange="30 30" intent=3 queue-size=30 scene-mode=12 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh264enc ! mpegtsmux ! filesink location=test_1.ts nvcamerasrc sensor-id=2 fpsRange="30 30" intent=3 queue-size=30 scene-mode=12 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh264enc ! mpegtsmux ! filesink location=test_2.ts

These are the tests done and the reference for the attached result image:

Auto-White-Balance and Auto-Exposure enabled. (Default case) Reference: wb_auto_ae_auto.png
Auto-White-Balance enabled and Auto-Exposure disabled/locked. (auto-exposure=1 aeLock=true) Reference: wb_auto_ae_off_lock.png
Manual-White-Balance and Auto-Exposure enabled. (wbmode=9) Reference: wb_manual_ae_auto.png
Manual-White-Balance enabled and Auto-Exposure disabled/locked. (wbmode=9 auto-exposure=1 aeLock=true) Reference: wb_manual_ae_off_lock.png

We have seen that the when the auto-exposure is enabled, it affects the colors between the three images captured. However, when it is disabled and locked, the colors of the three images look the same.

In the tests done, the cameras are not pointing in the same direction, but they are capturing the same scene where the light exposure should be similar for the 3 cameras.
I wonder why the ISP is too sensible to light? Because in the resultant images we can observed that the colors of the center camera present very different light exposure when the auto-exposure is used.

Do you know if this auto-exposure could be controlled in order to have the same colors/light for the three images?