ISP's autoexposure does not use the full exposure time range and instead increases gain

julian.morua · March 1, 2022, 6:11pm

Hello!

I have a sensor driver that implements the exposure and gain controls. I have validated the driver and its controls with laboratory equipment and the v4l2-utils. However, there is a strange behavior when I try to capture with GStreamer, specifically when using gst-launch-1.0 and nvarguscamerasrc.

The sensor has an exposure time range that goes up to 990 ms, and it has been configured accordingly in the device tree:

min_exp_time = "100";
max_exp_time = "990000";

The problem is that when I run a simple pipeline with autoexposure enabled, the ISP starts increasing the exposure time until it reaches 22 ms, after this it begins increasing the gain value. I do not know why it stops at 22 ms because the DT property max_exp_time is set to 990 ms. Interestingly, if I turn off the autoexposure (aelock=true), and I set the exposure time manually with v4l2-ctl, I can see that the sensor is configured correctly.

This makes me think that there is something else that is limiting the ISP’s autoexposure to 22 ms. I would expect the autoexposure algorithm to exhaust the sensor’s exposure time range before increasing gain, in order to avoid amplifying the noise. So, my question is:

What else could be limiting the ISP’s autoexposure?

Thanks for any help that you can provide!

kayccc · March 9, 2022, 6:25am

Sorry for the late response, I will inform our internal team to have suggestions. Thanks

ShaneCCC · March 9, 2022, 7:06am

Basically might be programming frameRate incorrectly as exposure Time is limited by
MaxExposureTime <= 1/FrameRate

julian.morua · March 9, 2022, 5:55pm

Hello ShaneCCC, thank you for your suggestion.

These are the device tree properties that are related to frame rate.

framerate_factor = "1000000";
step_framerate = "1000000";
min_framerate = "30000000";
max_framerate = "30000000";
default_framerate = "30000000";

Unfortunately, I think your suspicion is not the root cause of the problem. The frame rate device tree property is set to 30 fps, which means that 33.3 ms should be the exposure time upper bound. However, in my case, it is being capped at 22 ms, so something else must be limiting the exposure time.

Do you have any other ideas?

Thanks for your help!

ShaneCCC · March 10, 2022, 5:33am

Could you fixed the gain range to check the exposure?
You can use argus_camera to modify the min/max gain range to the same value to fix the gain.

julian.morua · March 11, 2022, 7:34pm

I tried two different methods for limiting the gain range:

With the device tree properties:

min_gain_val = "1";
max_gain_val = "1";
default_gain = "1";

With nvarsgucamerasrc property gainrange="1 1":

gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 aelock=false gainrange="1 1" ! nvvidconv ! xvimagesink

In both cases I see that the gain value stays constant at 1, but the exposure time is still being capped at 22 ms. I also tried with greater gain values, like `gainrange=“4 4”, but saw the same behavior.

ShaneCCC · March 12, 2022, 4:36am

Did you cover the lanes to check the AE?

julian.morua · March 14, 2022, 7:48pm

Yes, I covered the camera lenses and I also pointed the cameras towards a source of light.

When there is too much light I can see that the exposure values vary from 22 ms to 0.1 ms, but in normal/darker lighting conditions the exposure time does not go over 22 ms and instead the gain is adjusted. When gain range is limited to a single value, then no gain adjustments are performed but the ISP still will not request anything longer that 22 ms for the exposure time.

ShaneCCC · March 15, 2022, 1:58am

Are able set the exposure range >= 22 ms by argus_camera?

julian.morua · March 15, 2022, 7:39pm

I used this pipeline, hoping to set the exposure time specifically to 50 ms through the exposuretimerange="min max" property.

gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 exposuretimerange="50000000 50000000" ! nvvidconv ! xvimagesink

I also tried limiting the exposure time to a range instead of specific value, but always above 22 ms. For example, 40 ms to 50 ms with the following pipeline.

gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 exposuretimerange="40000000 50000000" ! nvvidconv ! xvimagesink

The results are interesting:

The exposure time does not stay fixed to 50 ms or within the 40-50 ms range. Instead, it varies depending on light conditions.
The gain value is incremented even though the exposure time has not reached its maximum value.
The exposure times can go above 50 ms when the room is dark enough, but not with unitary gain.
The behavior described above only happens when using the exposuretimerange property for different values; if the property is not used, then exposure times will not go above 22 ms.

See these logs that the driver’s controls print to the dmesg when limited to min = max = 50 ms. In the logs you can see that the exposure value requested by the ISP (value) is stuck at 33 ms, then gain is progressively increased to 8 and then the requested exposure times reach values around 70 ms.

[  469.227816] ar0820 42-0072: ar0820_set_gain: value 1 
[  469.231109] ar0820 42-0072: ar0820_set_exposure: value= 33330 us, pixclk= 156000000 Hz, ti= 5199480, llpck= 4440, n= 1, cit= 292 
[  469.231118] ar0820 42-0072: ar0820_start_streaming
[  470.022444] ar0820 42-0072: ar0820_set_gain: value 2 
[  470.073413] ar0820 42-0072: ar0820_set_exposure: value= 33329 us, pixclk= 156000000 Hz, ti= 5199324, llpck= 4440, n= 1, cit= 292 
[  470.172212] ar0820 42-0072: ar0820_set_gain: value 3 
[  470.222402] ar0820 42-0072: ar0820_set_gain: value 4 
[  470.225826] ar0820 42-0072: ar0820_set_exposure: value= 33330 us, pixclk= 156000000 Hz, ti= 5199480, llpck= 4440, n= 1, cit= 292 
[  470.272235] ar0820 42-0072: ar0820_set_gain: value 5 
[  470.322160] ar0820 42-0072: ar0820_set_gain: value 6 
[  470.374285] ar0820 42-0072: ar0820_set_exposure: value= 33329 us, pixclk= 156000000 Hz, ti= 5199324, llpck= 4440, n= 1, cit= 292 
[  470.422241] ar0820 42-0072: ar0820_set_gain: value 7 
[  470.425451] ar0820 42-0072: ar0820_set_exposure: value= 33330 us, pixclk= 156000000 Hz, ti= 5199480, llpck= 4440, n= 1, cit= 292 
[  470.472269] ar0820 42-0072: ar0820_set_gain: value 8 
[  470.475586] ar0820 42-0072: ar0820_set_exposure: value= 35273 us, pixclk= 156000000 Hz, ti= 5502588, llpck= 4440, n= 1, cit= 309 
[  470.523697] ar0820 42-0072: ar0820_set_exposure: value= 38114 us, pixclk= 156000000 Hz, ti= 5945784, llpck= 4440, n= 1, cit= 334 
[  470.573741] ar0820 42-0072: ar0820_set_exposure: value= 40683 us, pixclk= 156000000 Hz, ti= 6346548, llpck= 4440, n= 1, cit= 357 
[  470.623598] ar0820 42-0072: ar0820_set_exposure: value= 45037 us, pixclk= 156000000 Hz, ti= 7025772, llpck= 4440, n= 1, cit= 395 
[  470.673928] ar0820 42-0072: ar0820_set_exposure: value= 47337 us, pixclk= 156000000 Hz, ti= 7384572, llpck= 4440, n= 1, cit= 415 
[  470.723824] ar0820 42-0072: ar0820_set_exposure: value= 47487 us, pixclk= 156000000 Hz, ti= 7407972, llpck= 4440, n= 1, cit= 417 
[  470.773823] ar0820 42-0072: ar0820_set_exposure: value= 47640 us, pixclk= 156000000 Hz, ti= 7431840, llpck= 4440, n= 1, cit= 418 
[  470.823778] ar0820 42-0072: ar0820_set_exposure: value= 47765 us, pixclk= 156000000 Hz, ti= 7451340, llpck= 4440, n= 1, cit= 419 
[  470.873721] ar0820 42-0072: ar0820_set_exposure: value= 47842 us, pixclk= 156000000 Hz, ti= 7463352, llpck= 4440, n= 1, cit= 420 
[  470.926039] ar0820 42-0072: ar0820_set_exposure: value= 48055 us, pixclk= 156000000 Hz, ti= 7496580, llpck= 4440, n= 1, cit= 422 
[  474.573726] ar0820 42-0072: ar0820_set_exposure: value= 44223 us, pixclk= 156000000 Hz, ti= 6898788, llpck= 4440, n= 1, cit= 388 
[  474.623710] ar0820 42-0072: ar0820_set_exposure: value= 41679 us, pixclk= 156000000 Hz, ti= 6501924, llpck= 4440, n= 1, cit= 366 
[  474.673368] ar0820 42-0072: ar0820_set_exposure: value= 40290 us, pixclk= 156000000 Hz, ti= 6285240, llpck= 4440, n= 1, cit= 353 
[  474.727136] ar0820 42-0072: ar0820_set_exposure: value= 39496 us, pixclk= 156000000 Hz, ti= 6161376, llpck= 4440, n= 1, cit= 346 
[  474.773359] ar0820 42-0072: ar0820_set_exposure: value= 39019 us, pixclk= 156000000 Hz, ti= 6086964, llpck= 4440, n= 1, cit= 342 
[  474.823540] ar0820 42-0072: ar0820_set_exposure: value= 39016 us, pixclk= 156000000 Hz, ti= 6086496, llpck= 4440, n= 1, cit= 342 
[  474.873533] ar0820 42-0072: ar0820_set_exposure: value= 39858 us, pixclk= 156000000 Hz, ti= 6217848, llpck= 4440, n= 1, cit= 350 
[  476.623180] ar0820 42-0072: ar0820_set_exposure: value= 43710 us, pixclk= 156000000 Hz, ti= 6818760, llpck= 4440, n= 1, cit= 383 
[  476.673212] ar0820 42-0072: ar0820_set_exposure: value= 51754 us, pixclk= 156000000 Hz, ti= 8073624, llpck= 4440, n= 1, cit= 454 
[  476.723242] ar0820 42-0072: ar0820_set_exposure: value= 63025 us, pixclk= 156000000 Hz, ti= 9831900, llpck= 4440, n= 1, cit= 553 
[  476.773781] ar0820 42-0072: ar0820_set_gain: value 9 
[  476.777372] ar0820 42-0072: ar0820_set_exposure: value= 66660 us, pixclk= 156000000 Hz, ti= 10398960, llpck= 4440, n= 1, cit= 585 
[  476.885188] ar0820 42-0072: ar0820_set_gain: value 11 
[  476.988647] ar0820 42-0072: ar0820_set_gain: value 12 
[  477.090378] ar0820 42-0072: ar0820_set_exposure: value= 66659 us, pixclk= 156000000 Hz, ti= 10398804, llpck= 4440, n= 1, cit= 585 
[  477.188733] ar0820 42-0072: ar0820_set_gain: value 11 
[  477.292571] ar0820 42-0072: ar0820_set_exposure: value= 66660 us, pixclk= 156000000 Hz, ti= 10398960, llpck= 4440, n= 1, cit= 585 
[  477.789928] ar0820 42-0072: ar0820_set_exposure: value= 66659 us, pixclk= 156000000 Hz, ti= 10398804, llpck= 4440, n= 1, cit= 585 
[  477.890267] ar0820 42-0072: ar0820_set_exposure: value= 66660 us, pixclk= 156000000 Hz, ti= 10398960, llpck= 4440, n= 1, cit= 585 
[  482.888555] ar0820 42-0072: ar0820_set_gain: value 12 
[  482.988321] ar0820 42-0072: ar0820_set_gain: value 15 
[  483.092341] ar0820 42-0072: ar0820_set_gain: value 16 
[  483.096154] ar0820 42-0072: ar0820_set_exposure: value= 66976 us, pixclk= 156000000 Hz, ti= 10448256, llpck= 4440, n= 1, cit= 588 
[  483.190176] ar0820 42-0072: ar0820_set_exposure: value= 67217 us, pixclk= 156000000 Hz, ti= 10485852, llpck= 4440, n= 1, cit= 590 
[  483.290390] ar0820 42-0072: ar0820_set_exposure: value= 67428 us, pixclk= 156000000 Hz, ti= 10518768, llpck= 4440, n= 1, cit= 592 
[  483.390495] ar0820 42-0072: ar0820_set_exposure: value= 67751 us, pixclk= 156000000 Hz, ti= 10569156, llpck= 4440, n= 1, cit= 595 
[  483.490890] ar0820 42-0072: ar0820_set_exposure: value= 68118 us, pixclk= 156000000 Hz, ti= 10626408, llpck= 4440, n= 1, cit= 598 
[  483.590979] ar0820 42-0072: ar0820_set_exposure: value= 68474 us, pixclk= 156000000 Hz, ti= 10681944, llpck= 4440, n= 1, cit= 601 
[  483.691156] ar0820 42-0072: ar0820_set_exposure: value= 68851 us, pixclk= 156000000 Hz, ti= 10740756, llpck= 4440, n= 1, cit= 604 
[  483.791848] ar0820 42-0072: ar0820_set_exposure: value= 69300 us, pixclk= 156000000 Hz, ti= 10810800, llpck= 4440, n= 1, cit= 608 
[  483.891778] ar0820 42-0072: ar0820_set_exposure: value= 69831 us, pixclk= 156000000 Hz, ti= 10893636, llpck= 4440, n= 1, cit= 613 
[  483.992743] ar0820 42-0072: ar0820_set_exposure: value= 70406 us, pixclk= 156000000 Hz, ti= 10983336, llpck= 4440, n= 1, cit= 618 
[  484.092868] ar0820 42-0072: ar0820_set_exposure: value= 71014 us, pixclk= 156000000 Hz, ti= 11078184, llpck= 4440, n= 1, cit= 623 
[  484.193447] ar0820 42-0072: ar0820_set_exposure: value= 71780 us, pixclk= 156000000 Hz, ti= 11197680, llpck= 4440, n= 1, cit= 630 
[  484.294124] ar0820 42-0072: ar0820_set_exposure: value= 72554 us, pixclk= 156000000 Hz, ti= 1131842

So we seem to have taken a step forward, but the cause of the issue is still not clear. Evidently, the ISP’s autoexposure can request values greater than 22 ms, which we could not do before, but only when using the exposuretimerange property (and this range is not even respected).

Does this give you more clues towards the issue?

ShaneCCC · March 16, 2022, 4:08am

If the max frame rate is 30 then the max exposure should be 33ms,
Maybe have fpsdisplaysink like below to check the frame rate.

gst-launch-1.0 ...... ! nvvidconv ! fpsdisplaysink video-sink=xvimagesink sync=false

julian.morua · March 18, 2022, 5:15pm

Hello ShaneCCC,

Unfortunately our setup does not have an HDMI port; I have been connecting to the Jetson over SSH when using the xviamgesink element. The problem with this is that SSH introduces a lot of latency and this affects the measurements that fpsdisplaysinkcan get.

Instead of the element you proposed, I used RidgeRun’s open source gst-perf plugin for measuring the framerate. It is a filter element that prints performance metrics to the standard output. I ran several pipelines with it and I collected the following results.

Simple pipeline with no exposure or gain limit

gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 ! nvvidconv ! perf ! fakesink sync=false

Exposure times: under 22 ms.
Gain values: increased from 1 to 16 after exposure times reached 22 ms.
Frame rate: 20 fps.

Fixed unitary gain

gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 gainrange="1 1" ! nvvidconv ! perf ! fakesink sync=false

Exposure times: under 22 ms.
Gain values: fixed to 1.
Frame rate: 20 fps.

Fixed exposure time under 22 ms

gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 exposuretimerange="10000000 10000000" ! nvvidconv ! perf ! fakesink sync=false

Exposure times: fixed to 10 ms.
Gain values: immediately increased from 1 to 16.
Frame rate: 20 fps.

Fixed exposure time over 22 ms

gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 exposuretimerange="50000000 50000000" ! nvvidconv ! perf ! fakesink sync=false

Exposure times: started at 33 ms. Varied from 33 ms to 70 ms depending on light conditions.
Gain values: varied from 1 to 16 depending on light conditions.
Frame rate: started at 20 fps but progressively dropped as soon as the exposure time broke the 33 ms barrier. The frame rate reached 10 fps when the exposure time was about 66 ms.

Let me know if there is any other scenario that you would like me to test.

Thanks for your help!

ShaneCCC · March 21, 2022, 3:18am

Could you check if your system have /var/nvidia/nvcam/settings/camera_overrides.isp file
If yes remove it to try.

julian.morua · March 21, 2022, 4:37pm

Hello ShaneCCC,

Yes, I was using a camera_overrides.isp file to improve image quality. I removed it and I also removed the caches from the directory:

sudo mv /var/nvidia/nvcam/settings/camera_overrides.isp ~
sudo rm /var/nvidia/nvcam/settings/nvcam_cache_*
sudo rm /var/nvidia/nvcam/settings/serial_no_*

After removing the files I rebooted the Jetson and I can confirm that the change took effect on the system as the image quality decreased: without the ISP override file the image has a purple hue.

Unfortunately, removing the ISP override file does not solve the exposure time issues we’ve been discussing. I still see the exact same behavior that was described on a previous reply on this forum post

ShaneCCC · March 22, 2022, 8:13am

Below is my testing. It’s looks reasonable.
The exposure time increase from 70000 [us] to 99999 [us] (~= 1/10fps) when covering the lanes.
Does your sensor able output 30fps? why it always output 20fps when gst-launch-1.0 command default set framerate=30/1?

gst-launch-1.0 nvarguscamerasrc gainrange="1 1" ! 'video/x-raw(memory:NVMM), framerate=10/1,format=NV12' !  nvvidconv ! fpsdisplaysink video-sink=fakesink sync=false -v

[351104.310048] imx477 9-001a: imx477_set_frame_rate: Setting framerate control to: 10000000
[351104.310055] imx477 9-001a: imx477_set_frame_rate: val: 10000000e-6 [fps], frame_length: 12000 [lines]
[351104.310588] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0
[351104.924764] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 1
[351104.925021] imx477 9-001a: imx477_set_exposure: Setting exposure control to: 43983
[351104.925028] imx477 9-001a: imx477_set_exposure: val: 43983 [us], coarse_time: 5277 [lines]
[351104.926589] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0
[351105.024737] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 1
[351105.024971] imx477 9-001a: imx477_set_exposure: Setting exposure control to: 58036
[351105.025005] imx477 9-001a: imx477_set_exposure: val: 58036 [us], coarse_time: 6964 [lines]
[351105.025340] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0
[351105.124790] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 1
[351105.125112] imx477 9-001a: imx477_set_exposure: Setting exposure control to: 67018
[351105.125121] imx477 9-001a: imx477_set_exposure: val: 67018 [us], coarse_time: 8041 [lines]
[351105.125433] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0
[351105.224754] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 1
[351105.225042] imx477 9-001a: imx477_set_exposure: Setting exposure control to: 67741
[351105.225049] imx477 9-001a: imx477_set_exposure: val: 67741 [us], coarse_time: 8128 [lines]
[351105.225378] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0
[351105.324709] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 1
[351105.324967] imx477 9-001a: imx477_set_exposure: Setting exposure control to: 70000
[351105.324980] imx477 9-001a: imx477_set_exposure: val: 70000 [us], coarse_time: 8399 [lines]
[351105.325335] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0
[351120.924600] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 1
[351120.924860] imx477 9-001a: imx477_set_exposure: Setting exposure control to: 79999
[351120.924868] imx477 9-001a: imx477_set_exposure: val: 79999 [us], coarse_time: 9599 [lines]
[351120.928184] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0
[351121.924577] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 1
[351121.924803] imx477 9-001a: imx477_set_exposure: Setting exposure control to: 89999
[351121.924813] imx477 9-001a: imx477_set_exposure: val: 89999 [us], coarse_time: 10799 [lines]
[351121.925160] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0
[351122.024549] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 1
[351122.024772] imx477 9-001a: imx477_set_exposure: Setting exposure control to: 99999
[351122.024815] imx477 9-001a: imx477_set_exposure: exposure limited by frame_length: 11990 [lines]
[351122.024821] imx477 9-001a: imx477_set_exposure: val: 99999 [us], coarse_time: 11990 [lines]
[351122.025127] imx477 9-001a: imx477_set_group_hold: Setting group hold control to: 0

julian.morua · March 31, 2022, 3:17pm

Hello ShaneCCC,

You are right, there is a mismatch between the device tree fps and the real fps.

Actually, the sensor’s registers are configured to output 20 fps, this is working correctly and it is what I want. It is the device tree property that is wrong and should be set to 20 fps as well. However, I tried that and I get errors from GStreamer saying that a higher value than the one supported is being requested.

nvidia@localhost:~$ gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 ! nvvidconv ! perf ! fakesink sync=false
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3840 x 2160 FR = 20.000000 fps Duration = 50000000 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 100000, max 50000000;

GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 100000, max 22000000;

ARGUS_ERROR: Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute: 787 Frame Rate specified is greater than supported
GST_ARGUS: Running with following settings:
   Camera index = 2 
   Camera mode  = 0 
   Output Stream W = 3840 H = 2160 
   seconds to Run    = 0 
   Frame Rate = 20.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
ARGUS_ERROR: Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute: 1007 InvalidState.
GST_ARGUS: Cleaning up
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: CANCELLED
Additional debug info:
Argus Error Status
Execution ended after 0:00:01.063979700
Setting pipeline to PAUSED ...
Setting pipeline to READY ...

(gst-launch-1.0:7733): GStreamer-CRITICAL **: 15:15:02.027: gst_mini_object_set_qdata: assertion 'object != NULL' failed
Setting pipeline to NULL ...
Freeing pipeline ...

In particular, note this error message:

ARGUS_ERROR: Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute: 787 Frame Rate specified is greater than supported
GST_ARGUS

I will continue looking into this and try to get the device tree properties right to see if that helps the issue. Please let me know if you have any other thoughts on this.

Thank you!

ShaneCCC · April 1, 2022, 4:19am

Add “framerate=(fraction)20/1” to try

gst-launch-1.0 nvarguscamerasrc sensor-id=2 sensor-mode=0 !"video/x-raw(memory:NVMM), framerate=(fraction)20/1"! nvvidconv ! perf ! fakesink sync=false

ManuelLeiva · April 6, 2022, 8:35pm

Hello ShaneCCC,

I changed the frame rate value in the device tree, to set 20fps (to avoid the mismatch between the device tree fps and the real fps) and I have tried using “framerate=(fraction)20/1” and it fixes the problem, thank for your help.

Since now we can set a correct frame rate, we have changed the frame rate range in the device tree and now the ISP changes the exposure based on the limits indicated in the device tree. it is working as expected.

Thank you!

ManuelLeiva · April 7, 2022, 3:28pm

Hello,

Seems nvarguscamerasrc uses framerate=(fraction)30/1 by default, I was expecting when the driver reports a different framerate nvarguscamerasrc negotiate the new frame rate in the pipeline.

julian.morua · April 7, 2022, 4:00pm

Hello ShaneCCC,

I think we have finally found the root cause of the issue. To close this post, let me just summarize the solution:

Recently, it came to our attention that the nvarguscamerasrc element sets the framerate to 30 fps by default. This was exposed when we tried setting the max_framerate property to 20 fps in the device tree. In our previous testing no caps were provided for the GStreamer pipelines, so the framerate was set to 30 fps by default, thus throwing the error reported here.

As a workaround, explicitly setting the framerate with the GStreamer caps works sufficiently well:

 gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM), framerate=(fraction)20/1" ! ...

By doing this, the GStreamer pipeline is launched correctly with device tree properties that actually match the sensor’s behavior. With this, the ISP is using the full extent of the exposure time range defined in the device tree and the gain value is only increased after the exposure time range has been exhausted.

Thanks for all the help!