Hello,
I have been trying to use the Argus “setAeRegions” function under the IAutoControlSettings interface, but it has not changed the auto-exposure results. I have tried with both a repeat request, and single capture requests, as well as with two different cameras on a TX2 and Xavier. The exposure does not seem to be calculated using the region-of-interest. Since there are no samples using this function either, I was wondering if anyone can verify that this function works, or if there is a specific order that this must be called to enable the feature.
I am using a fisheye camera, so there is only a small region in the center that is actually exposed. Even when I set the region-of-interest to be a square at the center, the exposure quickly climbs such that the actual image is overexposed due to the black pixels outside of the visible region.
Thanks.
hello edexheim,
with below changes to modify interest region, I’m able to see different exposure values from preview frames.
could you please have a try with this modification,
thanks
diff --git a/public/samples/userAutoExposure/main.cpp b/public/samples/userAutoExposure/main.cpp
@@ -216,6 +216,15 @@ static bool execute(const UserAutoExposureSampleOptions& options)
+ std::vector<AcRegion> regions;
+ AcRegion testRegion(0, 0, 32, 32, 1.0);
+ regions.push_back(testRegion);
+
+ ac->setAeRegions(regions);
EXIT_IF_NOT_OK(iSession->repeat(request.get()), "Unable to submit repeat() request");
Hello JerryChang,
I have made the changes and am able to see different exposure values as well. However, it does not seem to change according to the specified testRegion. How did you determine that the auto-exposure is actually being calculated using only that region? From my tests, it seemed that changes in image intensity elsewhere in the image also affected the exposure time.
Thanks.
hello edexheim,
sorry for late reply,
could you please attach several capture results for reference.
you might also share the code snippet of what’s your interest region configuration.
thanks
I believe that I’m seeing this same issue on L4T 32.2 and Argus Version: 0.97.3 (multi-process). Our custom application argus code for auto exposure region of interest worked on previous releases.
The userAutoExposure example seems to be bad example for this, since it isn’t using the built-in Argus auto exposure.
I’ve replicated the issue in the eglImage Argus sample with this diff:
diff --git a/main.cpp b/main.cpp
--- a/main.cpp
+++ b/main.cpp
@@ -357,6 +357,16 @@ static bool execute(const EGLImageSampleOptions& options)
ORIGINATE_ERROR("Failed to get source settings request interface");
iSourceSettings->setSensorMode(sensorMode);
+ // Test Auto Exposure Region of Interest
+ IAutoControlSettings *iAutoControlSettings = interface_cast<IAutoControlSettings>(iRequest->getAutoControlSettings());
+ if (!iAutoControlSettings)
+ printf("Failed to get IAutoControlSettings interface\n");
+ std::vector<AcRegion> regions;
+ AcRegion testRegion(0, 0, 200, 200, 1.0);
+ regions.push_back(testRegion);
+ Argus::Status result = iAutoControlSettings->setAeRegions(regions);
+ printf("iAutoControlSettings->setAeRegions result = %d\n", result);
+
// Start the EGLImage rendering thread.
EGLImageRenderingThread renderingThread(outputStream.get(), options);
PROPAGATE_ERROR(renderingThread.initialize());
When the sensor is running with a well-lit scene, physically covering one corner with black (I tried all four) should ramp up the exposure a lot, resulting in blown out whites. This does not happen.
Can anyone else confirm this?
Edit: This thread is in AGX Xavier, but I’m seeing this on a TX2 4GB.
hello davidkh1m9,
could you also please share the capture results of region of interest AE not working well.
thanks
Hi JerryChang,
I’m attaching a video file here that has a single region of (0, 0, 400, 400, 1.0). I try all four corners to ensure that I’m not confused about the “top left” of the sensor, then I slide the card across the whole frame to show that auto exposure is calculating on the whole image. This is on an IMX377 sensor with the resolution set as 3840 x 2880:
static const Size2D<uint32_t> CAPTURE_SIZE(3840, 2880);
iOutputStreamSettings->setResolution(CAPTURE_SIZE);
My understanding is that the units of the region are the same as the resolution, so 400 should be roughly 10% of the width of the sensor.
nvargus-daemon doesn’t log anything when I call Argus::Status result = iAutoControlSettings->setAeRegions(regions); and the result of that call is 0.
In case it’s relevant, we have two IMX377 sensors and two RealSense USB depth sensors connected to the system. nvargus-daemon logs some BadParameter errors related to v4l2 devices, which seem similar to the thread with the custom libargus and libnvodm_imager_tx2. We’ve tried those libraries and they don’t change this auto exposure region of interest issue.
We also have another auto exposure issue that may or may not be related – we have a special low light mode that allows long exposures and higher gain, with the fps down to 10 or even slower. This worked great on previous JetPacks/libargus, but argus appears to be freezing (no frames received) on this version and logging this error very fast:
SCF: Error InvalidState: Session has suffered a critical failure (in src/api/Session.cpp, function capture(), line 667)
(Argus) Error InvalidState: (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
I’ve tried adjusting the limits of this mode and it seems that I need to lower them so much that the special mode is not much different than our normal mode.
argus-ae-roi.mov (26.1 MB)
hello davidkh1m9,
you should calling repeat() again to update the settings,
I’ve also adding debug prints to confirm that AeRegion has changed.
for example,
std::vector<AcRegion> regions;
AcRegion testRegion(0, 0, 192, 64, 1.0);
regions.push_back(testRegion);
ac->setAeRegions(regions);
iSession->repeat(request.get());
according to the below debug messages, the AeRegions has changes after sensor init.
Jerry populateToScfSettings numofReg= 1 AeRegions= (0, 0, 3840, 2160) weight= 1.000000 Line(100)
Jerry populateToScfSettings numofReg= 1 AeRegions= (0, 0, 3840, 2160) weight= 1.000000 Line(100)
Jerry populateToScfSettings numofReg= 1 AeRegions= (0, 0, 3840, 2160) weight= 1.000000 Line(100)
Jerry populateToScfSettings numofReg= 1 AeRegions= (0, 0, 192, 64) weight= 1.000000 Line(100)
Jerry populateToScfSettings numofReg= 1 AeRegions= (0, 0, 192, 64) weight= 1.000000 Line(100)
...
Hello JerryChang,
As mentioned previously in your first reply, I did call the repeat() request after setting the autoexposure regions. However, this does not change the behavior of the autoexposure algorithm similar to davidkh1m9’s post. Do you see a visible difference in the autoexposure after you make the change? I also printed out the new autoexposure regions, and can see the updated region.
Although the regions are updated, Argus does not seem to be using it, as the metadata from the bayerHistogram still covers the whole image. One would expect the histogram to have a much lower count after setting the region, but it has the same count as when the entire image is used.
Hi JerryChang and edexheim,
I can also confirm that we are calling repeat() after setAeRegions(). We actually do a stopRepeat() first, in case that matters.
Argus::ICaptureSession *iCaptureSession = Argus::interface_cast<Argus::ICaptureSession>(m_captureSession);
if (!iCaptureSession)
printf("Failed to get ICaptureSession interface\n");
iCaptureSession->stopRepeat();
if (iCaptureSession->repeat(m_request.get()) != Argus::STATUS_OK)
printf("Failed to submit repeating burst request\n");
Is there any example code that demonstrates the setAeRegions() feature? It’s also not implemented in nvarguscamerasrc, according to “gst-inspect-1.0 nvarguscamerasrc”.
This is still an open issue for us. What does Nvidia need from us to look into this further?
The region outside set using “setAeRegion” it gives more weight to the region selected not only have that region as metering.
@davidkh1m9
Please have a try on the next release should be the J4.4
Hi all,
Now that NVIDIA released the code for nvarguscamerasrc JP 4.4. I am implementing the aeregion property by myself. I have flashed JP 4.4 on Jetson TX2 EVM kit for this test. However I do not see any effect when changing the aeregion property. I have tried changing weight values and also regions.
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 1
Output Stream W = 2592 H = 1458
seconds to Run = 0
Frame Rate = 29.999999
Setting AeRegion thread 9:
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
GST_ARGUS: Setting AeRegion on the fly: 0 0 32 32 0.900000
GST_ARGUS: Setting AeRegion on the fly: 0 0 32 32 0.100000
Here is the code that I wrote:
std::vector < AcRegion > AeRegion;
AcRegion ae(left, top, right, bottom, weight);
AeRegion.push_back(ae);
GST_ARGUS_PRINT("Setting AeRegion on the fly: %d %d %d %d %f \n",AeRegion[0].left(), AeRegion[0].top(), AeRegion[0].right(), AeRegion[0].bottom(), AeRegion[0].weight());
l_iAutoControlSettings_ptr->setAeRegions(AeRegion);
l_iCaptureSession->repeat(l_captureRequest);
Should this feature be fixed on JP 4.4? am I missing something else on the code?
Thanks,
-Adrian
hello ACervantes,
it’s due to the fixes for setAeRegions did not catch JP 4.4 release date, hence you still observe the issue.
please expect next public release would include the fixes.
however,
could you please replace couple of pre-built library with attachment, Topic82744_May28.zip (3.0 MB) for verification.
you should also note that,
minimum supported AE Region size respected by user settings, while setting the region using below API,
thanks
virtual Size2D<uint32_t> getMinAeRegionSize() const = 0;
I just confirmed that this works on 4.4 DP and fixes the ROI bug!
Can we get a build that works on L4T 32.2 (JetPack 4.2.1) so we can deploy it on units in customer hands? I tried this build, but got the following error when running gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! nvoverlaysink:
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:851 Failed to start capture request
And the following new errors in the nvargus-daemon log:
nvargus-daemon[29518]: CAM: serial no file already exists, skips storing againSCF: Error BadParameter: Unable to create NvGlsiEglImage (in src/services/gl/EGLStreamProducer.cpp, function allocateAndRegisterBuffers(), line 260)
nvargus-daemon[29518]: SCF: Error BadParameter: (propagating from src/services/gl/EGLStreamProducer.cpp, function create(), line 123)
nvargus-daemon[29518]: SCF: Error BadParameter: (propagating from src/api/Session.cpp, function createEGLStreamProducer(), line 1335)
nvargus-daemon[29518]: (Argus) Error BadParameter: (propagating from src/api/EGLOutputStreamImpl.cpp, function connectProducer(), line 155)
nvargus-daemon[29518]: (Argus) Error BadParameter: (propagating from src/api/CaptureSessionImpl.cpp, function connectAllRequestStreams(), line 330)
nvargus-daemon[29518]: (Argus) Error InvalidState: Stream is not in CONNECTING state. (in src/api/EGLOutputStreamImpl.cpp, function connectProducer(), line 150)
nvargus-daemon[29518]: (Argus) Error InvalidState: (propagating from src/api/CaptureSessionImpl.cpp, function connectAllRequestStreams(), line 330)
hello davidkh1m9,
thanks for verification, and sorry, we don’t have plan to include setAeRegions fixes to previous release.
please based-on JetPack-4.4 DP and apply the libraries as temporary solution; or waiting for next JetPack formal release.
thanks
Hi JerryChang,
I implemented AeRegion based on the latest code for nvarguscamerasrc and JP 4.4. The feature is working very well, however we are seeing some issues when the ROI selected is small.
Questions:
- Do you know if there is a minimum ROI supported by SetAeRegion?
- if so, does it depend on the sensor resolution?
- Is there any other documentation for this:
virtual Size2D<uint32_t> getMinAeRegionSize() const = 0;
‘class Argus::ICameraProperties’ has no member named ‘getMinAeRegionSize’; did you mean ‘getMaxAeRegions’?
Thanks,
-Adrian
hello ACervantes,
since you’d verified AeRegion works on JP-4.4, I would suggest you initial another new discussion thread for better forum supports.
please open a new topic for your follow-up issues, you may also share your use-case and capture results for reference, thanks
hi all,
let’s follow-up the remaining AE region questions here: Minimum Region of Interest AeRegion,
thanks