Issue with NVIDIA Jetson Orin Nano and IMX662 Camera in DOL-HDR Mode

[William_chen]

Hello everyone,

I’m currently facing an issue with the NVIDIA Jetson Orin Nano（ Jetson Linux version 35.3.1） platform while using the IMX662 camera driver. In the regular mode, the image display works perfectly fine.

However, when switching to the DOL-HDR mode, the entire image turns purple, as shown in the attached image.

Even when I set the gain to its lowest value, the image becomes darker, but the purple tint persists, especially in well-lit areas.

I have a couple of questions regarding this problem:

1. How can I resolve this issue and get the DOL-HDR mode to display the correct colors?
2. Is there a method to capture raw images of the 3 frames of HDR for further analysis?

I would greatly appreciate any insights or suggestions regarding this matter. Thank you in advance for your assistance!

[ShaneCCC]

Reference to below link to add those parameters to camera_overrides.isp file.
BTW for Orin should be wdr.DOL.v5.EnableDOL = TRUE;

IMX577 DOL HDR Device Tree

For WDR you still need create camera_overrides.isp file at /var/nvidia/nvcam/settings and change mode to 664 And add below to the overrides file. wdr.DOL.v4.EnableDOL = TRUE; ae.wdr.DreMin = FER; ae.wdr.DreMax = FER; Check the FER from below Also the exposure ratio needs to be fixed value, preferably a power of 2. This fixed exposure ratio needs to obeys the following constraints: a. Short exposure time lower limit * fixed exposure ratio >= Long exposure time lower limit. b. Short expos…

[ShaneCCC]

BTW, I would suggest using latest release due to some fix for DOL HDR fusion.

[William_chen]

Hi,ShaneCCC

I upgraded the version r35.4.1 & JetPack-5.1.2,The issue is resolved.

Later I try to modify the isp file method.

And what causes this problem in r35.3?

Thank you for your assistance!

[ShaneCCC]

Due to some fusion logic error cause the problem and fixed in r35.4.1 release for Orin/Xavier.

Thanks

[William_chen]

Hi,ShaneCCC

I wanted to share an update on my progress. After upgrading the SDK and making modifications to the ISP files, the image quality has improved significantly. However, there’s still an issue that remains: certain parts of the image, especially in high-brightness areas, exhibit a slight pinkish tint.

Specifically, I have a couple of questions:

1. DOL-HDR Raw Capture: Is there a way to capture raw images in DOL-HDR mode? I’m interested in analyzing the raw data to better understand and address the color discrepancy issue.
2. DOL3 Frame Mode: Our current mode is DOL3. Beyond that, do we need to make any additional changes to our isp files?

Thank you in advance for your expertise and guidance!

[ShaneCCC]

1. Try the nvargus_nvraw and v4l2-ctl command.
2. Current only support 2 frames fusion.

[ShaneCCC]

Try adjust below parameter for pinkish issue.
Here’s formula to calculate the saturationThreshold = 1 - (OB / 2^max CSI bits)

wdr.DOL.v5[0].saturationThreshold.long = 0.96;
wdr.DOL.v5[0].saturationThreshold.short = 0.999;

[William_chen]

Hi，ShaneCCC

That doesn’t seem to work
My change now is as follows:

#* add
wdr.DOL.v4.EnableDOL = TRUE;
ae.wdr.DreMin = 32;
ae.wdr.DreMax = 32;
wdr.DOL.v4[0].saturationThreshold.long = 0.55;
wdr.DOL.v4[0].saturationThreshold.short = 0.55;
#* end

Using .v5 throws an error，My board is Jetson Orin nano (R35.4.1).

Argus Version: 0.99.3.3 (multi-process)
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/Dispatcher.cpp, initialize:1025 No cameras available
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/Dispatcher.cpp, getInstance:1001 Initalization failed
(NvCameraUtils) Error InvalidState: Mutex already initialized (in Mutex.cpp, function initialize(), line 41)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function open(), line 54)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 262)
(Argus) Error InvalidState: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 106)
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/Dispatcher.cpp, initialize:1017 Failed to create CameraProvider
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/Dispatcher.cpp, getInstance:1001 Initalization failed

[William_chen]

Hi，ShaneCCC

Good news!These two parameters work

I was before I changed it:

#* add
wdr.DOL.v4.EnableDOL = TRUE;
ae.wdr.DreMin = 32;
ae.wdr.DreMax = 32;
#* end

• The effect is as follows:
  
  

After I changed it:

#* add
wdr.DOL.v4.EnableDOL = TRUE;
ae.wdr.DreMin = 32;
ae.wdr.DreMax = 32;
wdr.DOL.v5[0].saturationThreshold.long = 0.96;
wdr.DOL.v5[0].saturationThreshold.short = 0.99;
#* end

• The effect is as follows:
  
  

But,there are still problems with the fusion part.

[ShaneCCC]

Does the min_hdr_ratio/max_hdr_ratio in device tree the same as DreMin/DreMax?

[William_chen]

Yes, this part is the same

[ShaneCCC]

Those parameters are all you can adjust.

[William_chen]

Hi,ShaneCCC

Following your suggestions, the parameter wdr.DOL.v5[0].saturationThreshold.long has proven effective.

Through manual adjustments of both exposure and gain for the IMX662’s DOL long and short frames, I’ve achieved more realistic images.

However, the issue of pinkish highlights still persists. My assumption is that adjusting the parameter wdr.DOL.v5[0].saturationThreshold.short might be the key, but regardless of the values I set for wdr.DOL.v5[0].saturationThreshold.short, there seems to be no noticeable effect.

Given this situation, I’m curious if there are any other parameters I should consider tweaking. As a point of comparison, I’ve attached an image captured using my smartphone’s HDR mode of the same scene for reference.

I greatly appreciate your ongoing assistance and insights in resolving this color issue.

Thank you

[ShaneCCC]

Looks like the long exposure frame over exposure cause the pixel value ~= 255 cause the problem.

[William_chen]

Hi,ShaneCCC

I have one more question, does Jetson Orin Nano support ‘bayer_wdr_dol_rggb12’ format?
I’m applying to Sony for DOL 2 related configuration.
I added this format in */ kernel/nvidia/drivers/media/platform/prototypes/camera/sensor_common.c

[ShaneCCC]

Suppose yes.
Configure the csi_pixel_bit_depth to 12 and mode_type = “bayer_wdr_dol”

[ryan65]

Is there an imx662 driver available somewhere that is referenced by OP here?

[ShaneCCC]

Sorry to tell there’s no imx662 driver in default release.