Is it possible to update the gain and exposure through v4l2 command after the camera stream is turned on?

Robotics & Edge Computing Jetson Systems Jetson AGX Xavier
1

Continuing the discussion from The v4l2 command to set exposure and gain cannot call the corresponding setting function of the driver:

The question we raised earlier is that we can set exposure and gain before the camera outputs an image, but we cannot set exposure and gain during the continuous imaging process in the application.

For example, I execute this command in a terminal:
“v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=50000 -d /dev/video0”

Execute in another window:
“v4l2-ctl -d /dev/video0 --set-ctrl exposure=22000,gain=248,bypass_mode=0,override_enable=1”

[ 1039.529260] ovx3c 10-0036: set max96716 init success ..., ret=0
[ 1039.757315] ovx3c 10-0036: set mode 0, ret=0
[ 1039.757328] ovx3c 10-0036: set gain 248 
[ 1039.757333] ovx3c 10-0036: [ovx3c]set gain 15 
[ 1039.757697] ovx3c_set_gain: val: 248 (/16) [times], gain: 15, again_shift:3, again_narrow_fixed=0x1f, 0x3509=0xf80
[ 1039.757709] ovx3c 10-0036: row_us:24
[ 1039.758070] ovx3c_set_exposure: val: 33000 [us], lines: 1367 ,hts:0x87f row_ms:24 [us], fact_times: 32808 [us]
[ 1039.758088] ovx3c 10-0036: set gain 110 
[ 1039.758091] ovx3c 10-0036: [ovx3c]set gain 6 
[ 1039.758456] ovx3c_set_gain: val: 110 (/16) [times], gain: 6, again_shift:2, again_narrow_fixed=0x1b, 0x3509=0x6c0
[ 1039.758467] ovx3c 10-0036: row_us:24
[ 1039.759202] ovx3c_set_exposure: val: 15000 [us], lines: 625 ,hts:0x87f row_ms:24 [us], fact_times: 15000 [us]
[ 1039.759215] start stream...
[ 1039.789785] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[ 1078.346741] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072

Exposure and gain are only set before the call to the start_streaming function in the driver.

Is there a way to adjust exposure and gain in user mode, because we need to implement a 3A algorithm in user mode?

2

Hello @DDEnginee

I see from your first discussion The v4l2 command to set exposure and gain cannot call the corresponding setting function of the driver - #6 by DDEnginee that executing the `v4l2-ctl -c override_enable=1’ command causes the controls to take effect.

Did you apply any changes to your driver so that the override_enable=1 control does not take effect anymore?

Furthermore, you can debug the tegracam_ctrls.c to check if the controls can be overridden or not.

Also, I see the following log messages:

Does the capture work properly with the default settings?

Regards!
Eduardo Salazar
Embedded SW Engineer at RidgeRun

Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com/
Website: www.ridgerun.com

3

The driver has not been modified. During the first discussion, it did not take effect. By setting v4l2-ctl -c override_enable=1, it can take effect before the driver calls start_streaming. As noted in my log, there are actions to set exposure and gain before start stream.... However, later there is a message tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072, which is due to some frame drops during the image capture test. My issue is that I cannot set exposure and gain during the image capture process.

4

I have no way to start this debug.

image

5

Hello @DDEnginee

Did you develop the driver or did a third party develop it? Do you have access to the source code driver? If so, could you please share the driver.c file?

Also, the tegracam_ctrls.c file is inside the NVIDIA kernel sources. Do you have access to them?

Regards.

6

What does this mean? I found the source code of tegracam_ctrls.c in the driver under the nvidia-oot directory.

nv_ovx3c.zip (4.7 KB)

7

Hello @DDEnginee

I meant that you should see the tegracam_ctrls.c at the following path:

./kernel/nvidia/drivers/media/platform/tegra/camera/tegracam_ctrls.c

I took a look at your driver.c file. In your ovx3c_set_exposure and ovx3c_set_gain, I only see a dev_info instance. Are the dev_info printed in dmesg when you execute the v4l2-ctl --set-ctrl commands?

Furthermore, could you successfully capture using the v4l2-ctl?

Regards!

8

When I execute this command: v4l2-ctl -d /dev/video0 --set-ctrl exposure=22000,gain=110,bypass_mode=0,override_enable=1, there is no output in dmesg.

I am using this path Linux_for_Tegra/source/nvidia-oot/drivers/media/platform/tegra/camera/tegracam_ctrls.c. Where is this ./kernel/nvidia/drivers/media/platform/tegra/camera/tegracam_ctrls.c located? I can’t find it in the directory where I am compiling the kernel.

9

Below is the running log

root@ubuntu:/home/byd/Desktop/driver# clear
root@ubuntu:/home/byd/Desktop/driver# dmesg -c 
root@ubuntu:/home/byd/Desktop/driver# sh install_driver.sh
root@ubuntu:/home/byd/Desktop/driver# v4l2-ctl -c override_enable=1
root@ubuntu:/home/byd/Desktop/driver# echo file tegracam_ctrls.c +p > /sys/kernel/debug/dynamic_debug/control
root@ubuntu:/home/byd/Desktop/driver# v4l2-ctl -c exposure=100
root@ubuntu:/home/byd/Desktop/driver# sudo dmesg | tail -100
[  239.599196] ovx3c 10-0036: load ko version [2.2.0]
[  239.599342] ovx3c 10-0036: tegracam sensor driver:ovx3c_v2.0.6
[  239.599403] tegra-camrtc-capture-vi tegra-capture-vi: subdev ovx3c 10-0036 bound
[  239.618372] isx031 9-001a: load ko version [2.1.0]
[  239.618493] isx031 9-001a: tegracam sensor driver:isx031_v2.0.6
[  239.618545] tegra-camrtc-capture-vi tegra-capture-vi: subdev isx031 9-001a bound
root@ubuntu:/home/byd/Desktop/driver# v4l2-ctl -l

Camera Controls

                     group_hold 0x009a2003 (bool)   : default=0 value=0 flags=execute-on-write
                    sensor_mode 0x009a2008 (int64)  : min=0 max=1 step=1 default=0 value=0 flags=slider
                           gain 0x009a2009 (int64)  : min=16 max=249 step=1 default=16 value=16 flags=slider
                       exposure 0x009a200a (int64)  : min=13 max=33001 step=1 default=11000 value=100 flags=slider
                     frame_rate 0x009a200b (int64)  : min=2000000 max=30000000 step=1 default=30000000 value=2000000 flags=slider
           sensor_configuration 0x009a2032 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[22] flags=read-only, volatile, has-payload
         sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[1026] flags=read-only, volatile, has-payload
      sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[1026] flags=read-only, volatile, has-payload
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=1 (1 0x1)
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0 (1 0x1)
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][18] flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][36] flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
    override_capture_timeout_ms 0x009a206f (int)    : min=-1 max=2147483647 step=1 default=2500 value=2500
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only
10 
root@ubuntu:/home/byd/Desktop/driver# sudo echo 7 >/sys/class/video4linux/video0/dev_debug
root@ubuntu:/home/byd/Desktop/driver# sudo dmesg | tail -100
root@ubuntu:/home/byd/Desktop/driver# v4l2-ctl -c exposure=500
root@ubuntu:/home/byd/Desktop/driver# sudo dmesg | tail -100
[  750.027005] videodev: v4l2_open: video0: open (0)
[  750.027030] video0: VIDIOC_QUERYCAP: driver=tegra-video, card=vi-output, ovx3c 10-0036, bus=platform:tegra-capture-vi:1, version=0x00050f94, capabilities=0x84200001, device_caps=0x04200001
[  750.027049] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a0001, type=6, name=Camera Controls, min/max=0/0, step=0, default=0, flags=0x00000044, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027060] video0: VIDIOC_TRY_EXT_CTRLS: which=0x0, count=0, error_idx=0, request_fd=0
[  750.027067] video0: VIDIOC_QUERYCTRL: id=0x9a0001, type=6, name=Camera Controls, min/max=0/0, step=0, default=0, flags=0x00000044
[  750.027079] video0: VIDIOC_G_SELECTION: error -25: type=vid-cap, target=0, flags=0x0, wxh=0x0, x,y=0,0
[  750.027088] video0: VIDIOC_QUERYCAP: driver=tegra-video, card=vi-output, ovx3c 10-0036, bus=platform:tegra-capture-vi:1, version=0x00050f94, capabilities=0x84200001, device_caps=0x04200001
[  750.027180] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a0001, type=6, name=Camera Controls, min/max=0/0, step=0, default=0, flags=0x00000044, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027190] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a0001, type=6, name=Camera Controls, min/max=0/0, step=0, default=0, flags=0x00000044, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027198] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2003, type=2, name=Group Hold, min/max=0/1, step=1, default=0, flags=0x00000200, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027214] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2008, type=5, name=Sensor Mode, min/max=0/1, step=1, default=0, flags=0x00000020, elem_size=8, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027222] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2009, type=5, name=Gain, min/max=16/249, step=1, default=16, flags=0x00000020, elem_size=8, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027229] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a200a, type=5, name=Exposure, min/max=13/33001, step=1, default=11000, flags=0x00000020, elem_size=8, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027238] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a200b, type=5, name=Frame Rate, min/max=2000000/30000000, step=1, default=30000000, flags=0x00000020, elem_size=8, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027247] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2032, type=258, name=Sensor configuration, min/max=0/4294967295, step=1, default=0, flags=0x00000184, elem_size=4, elems=22, nr_of_dims=1, dims=22,0,0,0
[  750.027256] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2033, type=258, name=Sensor mode I2C packet, min/max=0/4294967295, step=1, default=0, flags=0x00000184, elem_size=4, elems=1026, nr_of_dims=1, dims=1026,0,0,0
[  750.027265] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2034, type=258, name=Sensor control I2C packet, min/max=0/4294967295, step=1, default=0, flags=0x00000184, elem_size=4, elems=1026, nr_of_dims=1, dims=1026,0,0,0
[  750.027273] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2064, type=9, name=Bypass Mode, min/max=0/1, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027280] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2065, type=9, name=Override Enable, min/max=0/1, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027290] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2066, type=1, name=Height Align, min/max=1/16, step=1, default=1, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027298] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2067, type=9, name=Size Align, min/max=0/2, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027304] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2068, type=1, name=Write ISP format, min/max=1/1, step=1, default=1, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027311] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2069, type=258, name=Sensor Signal Properties, min/max=0/4294967295, step=1, default=0, flags=0x00000104, elem_size=4, elems=18, nr_of_dims=2, dims=30,18,0,0
[  750.027318] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a206a, type=258, name=Sensor Image Properties, min/max=0/4294967295, step=1, default=0, flags=0x00000104, elem_size=4, elems=16, nr_of_dims=2, dims=30,16,0,0
[  750.027326] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a206b, type=258, name=Sensor Control Properties, min/max=0/4294967295, step=1, default=0, flags=0x00000104, elem_size=4, elems=36, nr_of_dims=2, dims=30,36,0,0
[  750.027333] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a206c, type=258, name=Sensor DV Timings, min/max=0/4294967295, step=1, default=0, flags=0x00000104, elem_size=4, elems=16, nr_of_dims=2, dims=30,16,0,0
[  750.027340] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a206d, type=2, name=Low Latency Mode, min/max=0/1, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027347] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a206e, type=1, name=Preferred Stride, min/max=0/65535, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027354] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a206f, type=1, name=Override capture timeout ms, min/max=-1/2147483647, step=1, default=2500, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027362] video0: VIDIOC_QUERY_EXT_CTRL: id=0x9a2082, type=1, name=Sensor Modes, min/max=0/30, step=1, default=30, flags=0x00000004, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
[  750.027369] video0: VIDIOC_QUERY_EXT_CTRL: error -22: id=0xc09a2082, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000, elem_size=0, elems=0, nr_of_dims=0, dims=0,0,0,0
[  750.027397] video0: VIDIOC_S_EXT_CTRLS: which=0x9a0000, count=1, error_idx=0, request_fd=0, id/val=0x9a200a/0x1f4
[  750.027419] video0: VIDIOC_CROPCAP: error -25: type=vid-cap, bounds wxh=0x0, x,y=0,0, defrect wxh=0x0, x,y=0,0, pixelaspect 0/0
[  750.027429] video0: VIDIOC_CROPCAP: error -25: type=vid-out, bounds wxh=0x0, x,y=0,0, defrect wxh=0x0, x,y=0,0, pixelaspect 0/0
[  750.027473] videodev: v4l2_release: video0: release
11

Hello @DDEnginee

Which JetPack version are you using? Is your board devkit or a custom board?

Regards!

12

We are using JetPack 6.2 and the board devkit. Currently, We found the problem, it is related to the power supply settings.