Custom driver handler set_test_pattern() not called after power_off()

[alain.gautherot]

Hi,

I have an an IMX492 camera working on an Orin devkit with the latest Jetpack 5.1.2.
I need to control the test pattern for that sensor and have added a parameter “test_pattern” with the associated command TEGRA_CAMERA_CID_TEST_PATTERN to the kernel+driver.

I run “v4l2-ctl --set-ctrl test_pattern=11 -d 0” and can verify the sensor outputs the expected pattern.

The second time I run “v4l2-ctl --set-ctrl test_pattern=10 -d 0”, the test pattern doesn’t change and debug prints in the driver indicate that the handler “set_test_pattern” is not called the second time.

Looking at the output of dmesg, I indeed see only 1 call to set_test_pattern(), and a call to power_off() inbetween the 2 calls I made:

[  140.247870] imx492 30-001a: imx492_set_test_pattern: select test pattern 11   ## v4l2 command detected, handler called ==> OK
[  140.516140] imx492 30-001a: imx492_set_mode 880: HMAX=2193 => 1202
[  140.520336] imx492 30-001a: imx492_set_mode: mode 0, 8432x5648
[  140.520341] imx492 30-001a: imx492_set_mode: selecting test pattern 11
[  141.330743] imx492 30-001a: imx492_power_off: line 667  ## <== power_off causing issue?

## second call to v4l2-ctl, set_test_pattern() not called here
[  148.885530] imx492 30-001a: imx492_set_mode: mode 0, 8432x5648
[  148.885535] imx492 30-001a: imx492_set_mode: selecting test pattern 11  ## using stale value
[  149.779789] imx492 30-001a: imx492_power_off: line 667

So my question is: could it be that power_off() is causing the issue by preventing the call to set_test_pattern() in the driver?
If so, how can I make sure that set_test_pattern is called in all cases?

Thanks,
Alain

[JerryChang]

hello alain.gautherot,

did you implement this correctly? please check $ v4l2-ctl --all for all available camera controls.

[alain.gautherot]

@JerryChang

v4l2-ctl --all

I believe so:

$ v4l2-ctl --all
Driver Info:
	Driver name      : tegra-video
	Card type        : vi-output, imx492 30-001a
	Bus info         : platform:tegra-capture-vi:0
	Driver version   : 5.10.120
	Capabilities     : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : tegra-camrtc-ca
	Model            : NVIDIA Tegra Video Input Device
	Serial           : 
	Bus info         : 
	Media version    : 5.10.120
	Hardware revision: 0x00000003 (3)
	Driver version   : 5.10.120
Interface Info:
	ID               : 0x03000011
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000f (15)
	Name             : vi-output, imx492 30-001a
	Function         : V4L2 I/O
	Pad 0x01000010   : 0: Sink
	  Link 0x02000015: from remote pad 0x1000003 of entity '13e40000.host1x:nvcsi@15a00000-': Data, Enabled
Priority: 2
Video input : 0 (Camera 0: no power)
Format Video Capture:
	Width/Height      : 8432/5648
	Pixel Format      : 'RG12' (12-bit Bayer RGRG/GBGB)
	Field             : None
	Bytes per Line    : 16864
	Size Image        : 95247872
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             : 
Selection Video Capture: crop, Left 0, Top 0, Width 8432, Height 5648, Flags: 
Selection Video Output: crop, Left 0, Top 0, Width 8432, Height 5648, Flags: 

Camera Controls

                     group_hold 0x009a2003 (bool)   : default=0 value=0 flags=execute-on-write
                     hdr_enable 0x009a2004 (intmenu): min=0 max=1 default=0 value=0
				0: 0 (0x0)
				1: 1 (0x1)
                    sensor_mode 0x009a2008 (int64)  : min=0 max=12 step=1 default=0 value=0 flags=slider
                           gain 0x009a2009 (int64)  : min=1000000 max=16000000 step=1 default=1000000 value=1000000 flags=slider
                       exposure 0x009a200a (int64)  : min=21 max=500000 step=1 default=30000 value=21 flags=slider
                     frame_rate 0x009a200b (int64)  : min=2000000 max=10000000 step=1 default=10000000 value=2000000 flags=slider
                    black_level 0x009a2011 (int)    : min=0 max=2147483647 step=1 default=0 value=0
                   test_pattern 0x009a2012 (int)    : min=0 max=2147483647 step=1 default=0 value=0
           sensor_configuration 0x009a2032 (u32)    : min=0 max=4294967295 step=1 default=0 [22] flags=read-only, volatile, has-payload
         sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=4294967295 step=1 default=0 [1026] flags=read-only, volatile, has-payload
      sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=4294967295 step=1 default=0 [1026] flags=read-only, volatile, has-payload
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
				0: 0 (0x0)
				1: 1 (0x1)
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
				0: 0 (0x0)
				1: 1 (0x1)
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
				0: 1 (0x1)
				1: 65536 (0x10000)
				2: 131072 (0x20000)
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=4294967295 step=1 default=0 [30][18] flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=4294967295 step=1 default=0 [30][18] flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=4294967295 step=1 default=0 [30][36] flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=4294967295 step=1 default=0 [30][16] flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=12 flags=read-only

[JerryChang]

@alain.gautherot

added a parameter “test_pattern”

hello alain.gautherot,

please help me to understand this control property,
for example, is it something like a mode setting for writing different sensor mode registers? what’s the expected sensor behavior?

[alain.gautherot]

The idea is to configure the sensor to produce a known output (test pattern).
set_mode() still exists to select a resolution and bit depth, but there are a couple of extra registers to enable the test pattern generation and select which test pattern (black, white, grey, vertical stripes, horizontal stripes).

Setting the test pattern in realtime using v4l2-ctl and gstreamer works fine:

• with test pattern generation OFF:
  

  

• with test pattern set to10 (v4l2-ctl --set-ctrl test_pattern=10):
  

  

• with test pattern set to 11 (v4l2-ctl --set-ctrl test_pattern=11):
  

  

But if I rely only on v4l2-ctl to select the test pattern (off/vertical or horizontal stripes) and capture frames, it doesn’t work: I always get the test pattern that was first set.
Subsequent calls will not change the test pattern.

I think the call to power_off() that I see in “dmesg” output after the first call to “v4l2-ctl --set-ctrl test_pattern=XXX” is causing the problem.
Subsequent calls to “v4l2-ctl --set-ctrl test_pattern=XXX” doesn’t generate any log in dmesg, so driver is likely not called.
I’m wondering why that would be.

[JerryChang]

hello alain.gautherot,

please refer to below code flow, and it looks like it treat test_pattern=10 as init settings to configure the stream.
you may refer to the code snippet as following.
for example,
$public_sources/kernel_src/kernel/nvidia/drivers/media/platform/tegra/camera/tegracam_v4l2.c

static int v4l2sd_stream(struct v4l2_subdev *sd, int enable)
{
...
                err = sensor_ops->set_mode(tc_dev);

                /* update control ranges based on mode settings*/
                err = tegracam_init_ctrl_ranges_by_mode(
                        s_data->tegracam_ctrl_hdl, (u32) s_data->mode);

$public_sources/kernel_src/kernel/nvidia/drivers/media/platform/tegra/camera/tegracam_ctrls.c

int tegracam_init_ctrl_ranges(struct tegracam_ctrl_handler *handler)
{
...
        /* Use mode 0 control ranges as default */
        if (s_data->sensor_props.num_modes > 0) 
        {    
                err = tegracam_init_ctrl_ranges_by_mode(handler, 0);

hence,
please give it a try by adding your set_test_pattern CID controls into tegracam_init_ctrl_ranges_by_mode functions to update control properties.

[alain.gautherot]

@JerryChang

tegracam_init_ctrl_ranges_by_mode

thanks, I did that and ensured that values from the device tree are indeed applied.
Still, driver doesn’t seem to be called after the V4L framework calls power_off().

Note however that if I use my C++ wrapper around V4L2, the commands to select test_pattern are working as long as camera capture is started (sensor not turned off).

I guess it must be an issue specific to v4l2-ctl because controls can’t be set while streaming is enabled?

[JerryChang]

hello alain.gautherot,

that init ctrl only called once while sensor streaming started.
there’re some CID controls able to called while streaming, such as TEGRA_CAMERA_CID_GROUP_HOLD.