ISP Auto Exposure Issues

Almost have my MT9M021 driver ready to release, but i’m running into some difficulties with the ISP.

Background
Based on the dtsi and dmesg it looks like the ISP controls the following:

  1. Gain (min_gain_val, max_gain_val)
  2. HDR (min_hdr_ration, max_hdr_ration)
  3. Framerate (min_framerate, max_framerate)
  4. Exposure (min_exp_time, max_exp_time)

I also know that we are required to define at least these two Nvidia specific controls:

  • V4L2_CID_COARSE_TIME
  • V4L2_CID_FRAME_LENGTH

Assumptions:
I am assuming that V4L2_CID_COARSE_TIME is to set coarse integration time, and that V4L2_CID_FRAME_LENGTH is to set the number of pixel clock periods in one line (row) time. For the MT9M021, the former is specified in multiples of the latter. Is that the desired relationship?

The Issue
V4L2_COARSE_TIME is being set to 1 and is not being used by auto-exposure to adjust the image. Instead the system starts tweaking the gain, increasing it up to the maximum.

Here’s an example from my driver’s debug printing:

[  354.792097] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_FRAME_LENGTH
[  354.792122] daxc02 6-0010: mt9m021_write: 0x7fff to 0x300c
[  354.792470] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_COARSE_TIME
[  354.792491] daxc02 6-0010: mt9m021_write: 0x0014 to 0x3012
[  354.796590] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_FRAME_LENGTH
[  354.796613] daxc02 6-0010: mt9m021_write: 0x0672 to 0x300c
[  354.798327] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.798350] daxc02 6-0010: mt9m021_write: 0x0031 to 0x305e
[  354.798959] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_COARSE_TIME
[  354.798975] daxc02 6-0010: mt9m021_write: 0x0001 to 0x3012
[  354.832888] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.832904] daxc02 6-0010: mt9m021_write: 0x0038 to 0x305e
[  354.836406] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.836415] daxc02 6-0010: mt9m021_write: 0x0041 to 0x305e
[  354.840082] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.840092] daxc02 6-0010: mt9m021_write: 0x004b to 0x305e
[  354.843784] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.843796] daxc02 6-0010: mt9m021_write: 0x0056 to 0x305e
[  354.854696] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.854706] daxc02 6-0010: mt9m021_write: 0x0063 to 0x305e
[  354.858817] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.858826] daxc02 6-0010: mt9m021_write: 0x0071 to 0x305e
[  354.862810] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.862820] daxc02 6-0010: mt9m021_write: 0x0082 to 0x305e
[  354.866825] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.866837] daxc02 6-0010: mt9m021_write: 0x0096 to 0x305e
[  354.876496] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.876507] daxc02 6-0010: mt9m021_write: 0x00ac to 0x305e
[  354.880149] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.880159] daxc02 6-0010: mt9m021_write: 0x00c6 to 0x305e
[  354.884517] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.884531] daxc02 6-0010: mt9m021_write: 0x00e3 to 0x305e
[  354.889933] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  354.889946] daxc02 6-0010: mt9m021_write: 0x00ff to 0x305e

This shows that the first thing that happens is V4L2_CID_FRAME_LENGTH is set to the maximum value, followed by V4L2_CID_COARSE_TIME being set to 0x14. Next V4L2_CID_FRAME_LENGTH is set to its minimum value, Gain is set, and V4L2_CID_COARSE_TIME is set to 1. Then it starts increasing Gain until it reaches the maximum, and stops.

When Coarse Time (exposure) is set to 1 the image is almost non-existant, so increasing the Gain doesn’t really help. What I think should happen is the exposure would get set to a reasonable (not minimum) value, or at least it would try increasing it like it does gain.

Am I misunderstanding the definition of coarse time, or perhaps it is different for my sensor?

How are the min_exp_time and max_exp_time values in the dtsi utilized? I set the minimum to an exposure that is good for my sensor and I didn’t see that value pop up anywhere in my logs.

If I turn auto-exposure off for nvcamerasrc it allows me to use my image sensor’s own auto-exposure and I get a great image. However if I try to manually set Coarse Time and Gain through v4l2-ctl it immediately gets set back to 1 as soon as I set it to something reasonable. Like so:

[  513.086169] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_COARSE_TIME
[  513.086195] daxc02 6-0010: mt9m021_write: 0x01c2 to 0x3012
[  513.088981] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_COARSE_TIME
[  513.089002] daxc02 6-0010: mt9m021_write: 0x0001 to 0x3012
[  526.691114] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  526.691151] daxc02 6-0010: mt9m021_write: 0x0010 to 0x305e
[  526.707100] daxc02 6-0010: daxc02_s_ctrl: V4L2_CID_GAIN
[  526.707128] daxc02 6-0010: mt9m021_write: 0x0001 to 0x305e

Could someone verify my assumptions and tell me how exposure is supposed to be set?

Thanks!

Ok I think the issue was mainly that I shouldn’t have made my actual Coarse Time changeable. I went ahead and made Coarse Time the manufacturer recommended default, and made V4L2_CID_COARSE_TIME change the fine integration time, which is measure in pix clock cycles.

Now that there is an image the ISP seems to be doing what it is supposed to. Gain goes up and down until it settles on a value, rather than skyrocketing to the maximum.