# How to calculate device-tree parameters for WDR DOL-2 frame [IMX585]

Here is the IMX585 image drawing for DOL 2-frame all-pixels Line Information (LI):

Here are the values from IMX585 DOL application note document:

HORIZONTAL DIRECTION
W1 = 8
W2 = 3840
W3 = 8
LI = 8 (see image drawing)

VERTICAL DIRECTION
H0* = (RHS1-2)/2 = (78-2)/2 = 38
H1 = 1
H2 = 1
H3 = 10
H4 = 10
H5 = 4
H6 = 8
H7 = 2160
H8 = 8
H9 = 1
H10*= (RHS1-2)/2 = (78-2)/2 = 38

*RHS1 register is set to 0x4E = 78.

Now, the question is how to properly set DOL-related device-tree properties:

active_w = W2 + LI + W1 + W3 = 3840 + 8 + 8 + 8 = 3864

active_h = (H7 + H4 + H5 + H6 + H8 + H9 + H0)*2 =
(2160 + 10 + 4 + 8 + 8 + 1 + 38) * 2 = 4458 ???

num_of_exposure = 2 (always 2 for DOL-2 frame)

num_of_ignored_lines = H3 + H4 + H5 + H6 =
10 + 10 + 4 + 8 = 32 ???

num_of_lines_offset_0 = H0 = 38

num_of_ignored_pixels = LI = 8

num_of_left_margin_pixels = W1 = 8

num_of_right_margin_pixels = W3 = 8

Uncertainties:

1. We are not sure for active_h if we shall include H3 (Ignored OB_1)?

Can you write down formula for active_h with the above information?

1. We are not sure for num_of_ignored_lines if we shall include H6 (Effective margin for color processing)?

Can you write down formula for num_of_ignored_lines with the above information?

1. Do you perhaps see any errors in calculation of other parameters? For now, we only think issues might be in the active_h and num_of_ignored_lines parameters.

NOTE: I have read this documentation many times for the IMX274 but cannot make sense how to calculate those values for IMX585.

Thanks,
Primoz

hello primoz.fiser,

butâ€¦ you may double check the active_h and image height, as we do not support odd resolution.
hereâ€™s an example,
`active_h = [OB (12) + height (1080) + VBP (5)+ effective (3) ]* 2 = 2200`
`height = (active_h (2200) /2) - num_of_ignored_lines (12) - num_of_lines_offset (5) = 1083`

however, such setting, `height = 1083` is not supported.
in this scenario, hereâ€™s a little trick to increase `num_of_ignored_lines` by 1-line to make it as even numbers.

FYI, `num_of_ignored_lines` is used to crop lines from top.
this applies to both short and long exposures frames, increasing `num_ignored_lines` will result in a crop of the 1st line of the actual image.

some Q&As as below.
>> (1) Ignored OB_1 this is usually not considered into active_h
>> (2) & (3) please refer to above example for `num_of_ignored_lines` settings.

Hi @JerryChang ,

I applied your suggestions and came up with the following values for device-tree:

``````active_w = "3864";
active_h = "4458";
num_of_exposure = "2";
num_of_ignored_lines = "33"; // NOTE: Add + 1 to have even number
num_of_lines_offset_0 = "38";
num_of_ignored_pixels = "8";
num_of_left_margin_pixels = "8";
num_of_right_margin_pixels = "8";
``````

NOTE: I added 1 to num_of_ignored_lines, so 32 + 1 = 33

However, nvargus_nvraw tool still doesnâ€™t report any modes:

``````nvidia@jetson:~\$ nvargus_nvraw --lps
nvargus_nvraw version 1.14.0
Number of supported sensor entries 0
Entry  Source Mode      Uniquename             Resolution   FR  BitDepth  Mode
Index  Index  Index                                             CSI Dyn   Type
``````

NOTE: If I remove DOL mode from device-tree, I can see other non-DOL modes just fine.

What am I doing wrong?

hello primoz.fiser,

you should also have below settings defined in ISP override file to enable WDR support.
for instance, /var/nvidia/nvcam/settings/camera_overrides.isp
`wdr.DOL.v4.EnableDOL = TRUE;`
and, please also adding below to overwrite DRE values for HDR ratio.
`ae.wdr.DreMin = 16;`
`ae.wdr.DreMax = 16;`

Hi @JerryChang ,

I have played with DTS settings and somehow got the DOL mode working.

My current settings for IMX585 DOL mode are:

``````                mode6 { // IMX585_MODE_3856x2180_DOL
mclk_khz = "37125";
num_lanes = "4";
tegra_sinterface = "serial_a";
phy_mode = "DPHY";
discontinuous_clk = "no";
dpcm_enable = "false";
cil_settletime = "0";
/* FIXME: DOL related settings */
active_w = "3864";
active_h = "4444";

dynamic_pixel_bit_depth = "12";
csi_pixel_bit_depth = "12";
mode_type = "bayer_wdr_dol";
pixel_phase = "rggb";

line_length = "6600";
inherent_gain = "1";
mclk_multiplier = "40.40";
pix_clk_hz = "1500000000";

gain_factor = "10";
min_gain_val = "0"; /* 0dB */
max_gain_val = "720"; /* 72dB */
step_gain_val = "3"; /* 0.3 */
default_gain = "0";
min_hdr_ratio = "1";
max_hdr_ratio = "64";
framerate_factor = "1000000";
min_framerate = "1500000"; /* 1.5 */
max_framerate = "30000000"; /* 30 */
step_framerate = "1";
default_framerate= "30000000";
exposure_factor = "1000000";
min_exp_time = "27";  /* us */
max_exp_time = "660000"; /* us */
step_exp_time = "1";
default_exp_time = "10000";/* us */

/* FIXME: DOL related settings */
num_of_exposure = "2";
num_of_ignored_lines = "24";
num_of_lines_offset_0 = "38";
num_of_ignored_pixels = "8";
num_of_left_margin_pixels = "8";
num_of_right_margin_pixels = "8";
};
``````

And in camera_override.isp I have:

``````# FIXME: DOL support
wdr.DOL.v4.EnableDOL = TRUE;
ae.wdr.DreMin = 16;
ae.wdr.DreMax = 16;
``````

However DOL frames look a bit distorted full of strange artifacts?

Non-DOL mode frame output:

DOL mode frame output:

Do you have any ideas?

Thanks,
BR,
Primoz

hello primoz.fiser,

itâ€™s glad to know that youâ€™ve DOL sensor mode enabled.
hereâ€™re several suggestions.
(1) since youâ€™re using 72dB gain, you may try lower the gain settings.
(2) you should have `min_hdr_ratio` = `max_hdr_ratio`, and also the same DRE values within ISP override file.
(3) indoor + outdoor scene might be too complex to identify the failure. please try change your target to LED lights to check long/short exposure frame fusion results.

BTW,
your following up issue looks like related to WDR image tuning.
unfortunately, we donâ€™t support tuning stuffs via forum discussion threads,
thanks

1 Like

Hi @JerryChang ,

thanks for you suggestions and answers.

I guess sensor tuning for DOL is indeed required.
LEF:

SEF:

As it seems, LEF is too exposed and SEF is too darkâ€¦

One more question regarding DOL and the expected 30 FPS.

We are only getting 15 FPS instead of 30 FPS?

I am using the following gst pipeline to check frame rate:

``````\$ gst-launch-1.0 nvarguscamerasrc sensor-mode=6 sensor-id=0 ! 'video/x-raw(memory:NVMM), width=3840, height=2160, format=NV12, framerate=30/1' ! queue ! fpsdisplaysink video-sink=fakesink -v
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:src: caps = video/x-raw(memory:NVMM, meta:GstVideoOverlayComposition), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM, meta:GstVideoOverlayComposition), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:video_sink: caps = video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3856 x 2180 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 27000, max 660000000;

GST_ARGUS: 3840 x 2160 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 27000, max 660000000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 27000, max 660000000;

GST_ARGUS: 3856 x 2180 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 15.848933; Exposure Range min 27000, max 660000000;

GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 15.848933; Exposure Range min 27000, max 660000000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 15.848933; Exposure Range min 27000, max 660000000;

GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 7.079458; Exposure Range min 864000, max 20480000;

GST_ARGUS: 3816 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 27000, max 660000000;

GST_ARGUS: 1924 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 27000, max 660000000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode  = 6
Output Stream W = 3840 H = 2160
seconds to Run    = 0
Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 9, dropped: 0, current: 16.98, average: 16.98
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 9, dropped: 0, current: 16.98, average: 16.98
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 17, dropped: 0, current: 14.99, average: 15.98
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 17, dropped: 0, current: 14.99, average: 15.98
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 25, dropped: 0, current: 15.01, average: 15.66
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 25, dropped: 0, current: 15.01, average: 15.66
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 33, dropped: 0, current: 15.00, average: 15.49
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 33, dropped: 0, current: 15.00, average: 15.49
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 41, dropped: 0, current: 14.99, average: 15.39
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 41, dropped: 0, current: 14.99, average: 15.39
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 49, dropped: 0, current: 15.00, average: 15.33
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 49, dropped: 0, current: 15.00, average: 15.33
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 57, dropped: 0, current: 15.01, average: 15.28
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 57, dropped: 0, current: 15.01, average: 15.28
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 65, dropped: 0, current: 15.00, average: 15.25
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 65, dropped: 0, current: 15.00, average: 15.25
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 73, dropped: 0, current: 15.00, average: 15.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 73, dropped: 0, current: 15.00, average: 15.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 81, dropped: 0, current: 15.01, average: 15.20
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 81, dropped: 0, current: 15.01, average: 15.20
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 89, dropped: 0, current: 14.99, average: 15.18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 89, dropped: 0, current: 14.99, average: 15.18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 97, dropped: 0, current: 15.00, average: 15.16
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 97, dropped: 0, current: 15.00, average: 15.16
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:06.965034937
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Freeing pipeline ...
``````

Can you suggest how to achieve 30 FPS?

BR,
Primoz

hello primoz.fiser,

usually, Iâ€™ll add VIC to convert the formats for checking.
for instance,
`\$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvvidconv ! fpsdisplaysink text-overlay=0 name=sink_0 video-sink=fakesink sync=0 -v`

Hi @JerryChang ,

even when using:
`gst-launch-1.0 nvarguscamerasrc sensor-mode=6 sensor-id=0 ! 'video/x-raw(memory:NVMM), width=3840, height=2160, format=NV12, framerate=30/1' ! nvvidconv ! fpsdisplaysink text-overlay=0 name=sink_0 video-sink=fakesink sync=0 -v`

we only get 15 FPS.

Do you have any reference information regarding DOL IMX274 Frame rate (FPS)?

@primoz.fiser
Please check by v4l2-ctl to confirm able get 30fps.

``````v4l2-ctl --stream-mmap -c bypass_mode=0
``````

Hi @JerryChang and @ShaneCCC ,

I did some more debugging and found out we had XVS sub-sampling enabled in the IMX585 driver.

After disabling XVS sub-sampling, the frame rate is now correct at 30 FPS but output frames are no longer in sync anymore!

Here is the imx585 driver change:

``````+++ b/kernel/nvidia/drivers/media/i2c/imx585_mode_tbls.h
@@ -943,7 +943,7 @@ static const imx585_reg mode_3856x2180_dol[] = {
{DATARATE_SEL,         0x02}, /* 1782 Mbps */
{WDMODE,               0x01}, /* DOL 2 frame mode */
{ADDMODE,              0x00}, /* all pixel */
-    {THIN_V_EN,            0x01}, /* enable (subsampling) */
+    {THIN_V_EN,            0x00}, /* disable (subsampling) */
{VCMODE,               0x00}, /* Enable Line information (LI) output */
{LANEMODE,             0x03}, /* 4 lane */
{SHR0_LOW,             0x06},
``````

Here is the resulting output fusioned frame:

Do you know if the reference IMX274 implementation had XVS sub-sampling enabled or not?

Or maybe why are we getting frame sync issues when disabling XVS subsamplingâ€¦

BR,
Primoz

hello primoz.fiser,

may I double confirm the Jetpack/L4T release version youâ€™re working with.
andâ€¦ whatâ€™s this XVS sub-sampling actually did. for example, is 3856x2180 the full resolution of IMX585?

@JerryChang

may I double confirm the Jetpack/L4T release version youâ€™re working with.

``````nvidia@jetson:~\$ cat /etc/nv_tegra_release
# R35 (release), REVISION: 4.1, GCID: 33958178, BOARD: t186ref, EABI: aarch64, DATE: Tue Aug  1 19:57:35 UTC 2023
nvidia@jetson:~\$
``````

andâ€¦ whatâ€™s this XVS sub-sampling actually did.

I disabled subsampling because VMAX register in our case is set to VMAX = 4500 for 30 FPS.

Here is snippet from IMX585 datasheet:

and short explanation:

for example, is 3856x2180 the full resolution of IMX585?

Total number of pixels 3856 x 2220

Number of effective pixels 3856 x 2180

Number of active pixels 3856 x 2176

Number of recommended recording pixels 3840 x 2160

hello primoz.fiser,

IMX274 doesnâ€™t had XVS sub-sampling enabled

such fusion shift issue has happened after you revise VMAX, right?
if yes, you may give it a try to configure VBP (Vertical Blank Period) to adjust long/short frames.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.