IMX412 camera using the DOL-HDR mode on the Orin NX R36.4.4

[akvadak66]

Good day everyone.
We have an IMX412 camera that works.
This camera has a DOL-HDR mode that can be enabled by setting the corresponding bits in the register.
The nv_imx477 driver is suitable for the IMX412.
We modified the driver to enable DOL-HDR mode via ctrl.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)3840,height=(int)2160,framerate=30/1' ! nvvidconv flip-method=0 !  xvimagesink sync=false -e --verbose

When switching to DOL-HDR mode, the image is degraded.

Debug output

trace.txt (2.4 MB)

Can you tell me if there is a document that describes in detail the parameters of the MIPI output information?
How can I understand what is in the file above?

Do we need to modify our DTS somehow to enable DOL-HDR mode?

As far as we understand, DOL-HDR mode is an internal switching that occurs in the sensor.
However, the CSI stream parameters should not change.
We previously conducted similar experiments on a Jetson Nano for P32.7.2 and everything worked.

Could this be related to the embedded data that the sensor adds to the CSI stream?

embedded_metadata_height = “2”;

Is there a way to read this data from user space?

[ShaneCCC]

Have check below document for DOL HDR mode.

Sensor Software Driver Programming — NVIDIA Jetson Linux Developer Guide

[akvadak66]

Thanks for your reply.
My assumptions above were completely wrong.
I understood the interaction in the channel receiving the video stream from the camera.

I’ve started receiving images, but they’re heavily artifact.

It looks like the WDR modes are configured incorrectly.

mode0 { // IMX412_MODE_3840x2160 */								
mclk_khz = "24000";								
num_lanes = "4";								
tegra_sinterface = "serial_a";								
phy_mode = "DPHY";								
discontinuous_clk = "no";								
dpcm_enable = "false";								
cil_settletime = "0";								
lane_polarity = "6";

active_w = "3860"; // 4(LI) +8 + 3840 + 8 ;
active_h = "4592"; // (2160 + 64(OPB dum) +   16(OPB effec) + 32(OPB dumm) + 8 + 16(open dum)) * 2								

mode_type = "bayer";								
pixel_phase = "rggb";								
csi_pixel_bit_depth = "10";								
readout_orientation = "90";
line_length = "11200";								
inherent_gain = "1";								
mclk_multiplier = "80";//								
pix_clk_hz = "600000000";
pix_clk_hz = "300000000";								
gain_factor = "16";								
framerate_factor = "1000000";								
exposure_factor = "1000000";								
min_gain_val = "16"; // 1.00x			
max_gain_val = "356"; // 22x 
step_gain_val = "1";								
default_gain = "16"; // 1.00x 
min_hdr_ratio = "1";								
max_hdr_ratio = "1";								
min_framerate = "2000000"; // 2.0 fps
max_framerate = "30000000"; // 30.0 fps 
step_framerate = "1";								
default_framerate = "30000000"; // 30.0 fps 
min_exp_time = "13"; // us 								
max_exp_time = "683709"; // us 
step_exp_time = "1";								
default_exp_time = "2495"; // us
embedded_metadata_height = "2";	


num_of_exposure = "2";    
num_of_ignored_lines = "120";    
num_of_lines_offset_0 = "16";    
num_of_ignored_pixels = "8";    
num_of_left_margin_pixels = "8";    
num_of_right_margin_pixels = "8";							
};

Please check my DTS

I also added this to the /var/nvidia/nvcam/settings/camera_overrides.isp file.

wdr.DOL.v4.EnableDOL = TRUE;
ae.wdr.DreMin = 16;
ae.wdr.DreMax = 16;

[ShaneCCC]

The min_hdr_ratio/max_hdr_ratio need to be 16 like DreMin/DreMax.

[akvadak66]

Thanks for the reply.
We checked, but it didn’t help.

Unfortunately, there’s no information on the ISP.
Are these three lines enough to get the correct DOL-HDR mode?

[ShaneCCC]

Yes, that should be able to enable the DOL HDR mode.

@akvadak66

Are these three lines enough to get the correct DOL-HDR mode?

[akvadak66]

Thank you.

Most likely, the problem is with incorrectly configured WDR parameters.

num_of_ignored_lines = “120”;
num_of_lines_offset_0 = “16”;
num_of_ignored_pixels = “8”;
num_of_left_margin_pixels = “8”;
num_of_right_margin_pixels = “8”;

There’s a similar thread here, but it’s for the IMX585 sensor.

https://forums.developer.nvidia.com/t/how-to-calculate-device-tree-parameters-for-wdr-dol-2-frame-imx585/292363

Unfortunately, I don’t have the IMX412 DOL app documentation.

I would be very grateful if anyone has this document and wrote down the correct values.

I’m currently trying to get a result by iterating through the values ​.

[ShaneCCC]

You may consult with sensor vendor to get those information.

Thanks