IMX412 camera using the DOL-HDR mode on the Orin NX R36.5

Robotics & Edge Computing Jetson Systems Jetson Orin NX
1

Hello.

Continuing with the topic
https://forums.developer.nvidia.com/t/imx412-camera-using-the-dol-hdr-mode-on-the-orin-nx-r36-4-4/348609/8

We managed to get DOL-HDR mode working on the R36.4.4 camera, and the camera works well.
But the new R36.5 release has failed.

We’ve taken all the settings that work on the R36.4.4 and compiled them for the R36.5.
We can’t get image capture.
Image capture on the R36.5 works fine with a regular camera, too.
The only problem is with the DOL-HDR mode.

Here’s our DTS for sensor settings.

							mode0 { // IMX412_MODE_3840x2160 DOL-HDR*/
								mclk_khz = "24000";
								num_lanes = "4";
								tegra_sinterface = "serial_a";
								phy_mode = "DPHY";
								discontinuous_clk = "no";
								dpcm_enable = "false";
								cil_settletime = "0";
								lane_polarity = "6";
                                
                                active_w = "3864"; 
                                active_h = "4798"; 
                                mode_type = "bayer_wdr_dol";
								pixel_phase = "rggb";
								csi_pixel_bit_depth = "10";
								readout_orientation = "90";
                                line_length = "11200";

								inherent_gain = "1";
								mclk_multiplier = "80";
                                pix_clk_hz = "300000000";

								gain_factor = "16";
								framerate_factor = "1000000";
								exposure_factor = "1000000";
								min_gain_val = "16"; // 1.00x */
								max_gain_val = "356"; // 22x */
								step_gain_val = "1";
								default_gain = "16"; // 1.00x */
								min_hdr_ratio = "16";
								max_hdr_ratio = "16";
								min_framerate = "2000000"; // 2.0 fps */
								max_framerate = "30000000"; // 30.0 fps */
								step_framerate = "1";
								default_framerate = "30000000"; // 30.0 fps */
								min_exp_time = "13"; // us */
								max_exp_time = "683709"; // us */
								step_exp_time = "1";
								default_exp_time = "2495"; // us */
								embedded_metadata_height = "4";

	num_of_exposure = "2";
    num_of_ignored_lines = "200"; //"120";32
    num_of_lines_offset_0 = "39"; //"16";310
    num_of_ignored_pixels = "8"; 
    num_of_left_margin_pixels = "8";
    num_of_right_margin_pixels = "0";

};

Could you tell me what features of the R36.5 are preventing us from getting results?

4

Do you confirm by v4l2-ctl to capture the DOL HDR mode.

Thanks

5

When starting

v4l2-ctl -d /dev/video0 --set-fmt-video=width=3840,height=2160,pixelformat=RG10 --set-ctrl bypass_mode=0,sensor_mode=0 --stream-mmap --stream-count=100

Nothing happens

I’m attaching the debug trace output

trace.txt (85.2 MB)

6

It could be deskew calibration failed.

Adjust the pix_clk_hz/serdes_pix_clk_hz to verify it.

Skew calibration is required if sensor or deserializer is using DPHY, and the output data rate is > 1.5Gbps.
An initiation deskew signal should be sent by sensor or deserializer to perform the skew calibration. If the deskew signals is not sent, the receiver will stall, and the capture will time out.
You can calculate the output data rate with the following equation:

Output data rate = (sensor or deserializer pixel clock in hertz) * (bits per pixel) / (number of CSI lanes)
7

We recalculated and increased pix_clk_hz = “480000000”;
We also set serdes_pix_clk_hz = “1200000000”;
But this didn’t produce any results.
I have a question about the nv_imx 77 driver.
The driver completely removed the 4-line modes, leaving only the 2-line mode.
What’s the reason for this?

8

Do you confirm the output data rate less than 1.5G?

Due to Orin NX/Nano don’t have 4 lane HW design to remove the 4 lane sensor config.

Thanks

9

According to our calculations, the output data transfer rate is less than 1.5 Gbps per lane for my HDR mode (3080x2160@30fps, DOL-HDR, RAW10, 4 lanes on IMX412/IMX477).
Target pixel clock frequency: ~480 MHz
(from OP_PLL registers: PRE_DIV=2, MPY=384, SYS_DIV=1, PIX_DIV=10; INCK=24 MHz).
Data transfer rate per lane = 4.8 Gbps / 4 lanes = 1.2 Gbps/lane < 1.5 Gbps
This means that skew calibration is not required.

We need 4 lanes, so we use a custom-made board. The issue remains regarding the incompatibility between versions P36.4.4 and P36.5.
We wanted to test the new P36.5 release, but it only works with the same features that worked on P36.4.4.
This means that P36.5 adds checks for those features that weren’t present in P36.4.4.

10

When you set the serdes_pix_clk_hz to 1200000000 that tell the NVCSI/VI driver the output data rate is 1200000000*10/4 = 3.0G that trigger the deskew calibration.

You can remove it from the dts if your driver does’t need it.

Thanks

11

@ShaneCCC Our main concern is what happened in R36.5 to cause the issue while R35.4.4 works well? We would like to sync our system with the latest R36.5 since the release notes report the following fix done “On Jetson Orin, captured images using argus_camera with DOL HDR sensors may show a marginal increase in noise.“

12

Deskew calibration enable after R35.5 and JP6

13

@ShaneCCC I meant R36.4.4 vs R35.4.4
CORRECTED: Our main concern is what happened in R36.5 to cause the issue while R36.4.4 works well?

14

You can try the r36.4.4 RCE firmware on R36.5 to clarify.

Thanks