Raspberry Pi HQ Camera in Jetson Nano

Hello,

We have switched to the RidgeRun drivers and still can’t change the exposure time. Any advice?

Thank you

@siderskiy

Exposure time GStreamer pipeline can be locked to the frame rate, or at least not greater than it (360 degree shutter).

If you think about it, it makes sense. In a video pipeline the shutter can’t be open for longer than a frame. You can try setting exposure after the pipeline starts on the Argus source but in my tests this breaks things.

For reference I’m working with the IMX 477 by Arducam and RidgeRun’s drivers using the patch method.

Thanks, I’ll give it a try.
I tried setting the frame rate very low and modulate the exposure, and still no luck. Autoexposure works, it somehow modulates the exposure time. Also changing the exposure time on the camera on the Pi works (but not using gstreamer).

Wondered if autoexposure somehow works with exposure time longer than the frame rate with some kind of sliding window buffering algorithm.

Hi All,

Thanks for the patience, the driver is already ported to JP 4.4.1 and tomorrow we will just do a review and then make it available.

@siderskiy, can you provide more details on how you are testing? We would like to give it a try, the driver has a exposure and the ISP makes use of that control.

-David

@siderskiy

IIRC, it does auto-gain by default unless you set both values of the ranges to the same, so there are at least these in play:

 $ gst-inspect-1.0 nvarguscamerasrc
...
  exposuretimerange   : Property to adjust exposure time range in nanoseconds                   Use string with values of Exposure Time Range (low, high)                        in that order, to set the property.                     eg: exposuretimerange="34000 358733000"
                        flags: readable, writable
                        String. Default: null
  gainrange           : Property to adjust gain range                   Use string with values of Gain Time Range (low, high)   in that order, to set the property.                      eg: gainrange="1 16"
                        flags: readable, writable
                        String. Default: null
  ispdigitalgainrange : Property to adjust digital gain range                   Use string with values of ISP Digital Gain Range (low, high)                     in that order, to set the property.                     eg: ispdigitalgainrange="1 8"
                        flags: readable, writable
                        String. Default: null
...
  exposurecompensation: property to adjust exposure compensation
                        flags: readable, writable
                        Float. Range:              -2 -               2 Default:               0 

For details of what exactly those do, you’ll have to dive into the source, which is in the public sources tarball, IIRC. To me at least, it’s not at all clear what these properties do.

Thank you @mdegans and @DavidSoto-RidgeRun,

TLDR; Real exposure time range for sensor mode 1: [34000,33333333]

With @aaronlozhkin:

We did an exhaustive test and here is the story.

  1. When using gst-launch-1.0 nvarguscamerasrc … it shows that both sensor modes have an exposure range of [13000,683709000]. However, both of those values output a warning ‘invalid time range’, and the camera defaults to some default exposure time.
    In:

gst-launch-1.0 nvarguscamerasrc exposuretimerange="13000 13000" wbmode=0 sensor_id=0 sensor-mode=1 ispdigitalgainrange="1 1" aelock=true gainrange="4 4" ! nvoverlaysink

Out:

GST_ARGUS: NvArgusCameraSrc: Setting Exposure Time Range : 13000 13000
GST_ARGUS: Invalid Exposure Time Range Input

  1. The values that don’t produce an error are [34000,358733008]. Suspiciously close to the gst-inspect-1.0 nvarguscamerasrc example.

  2. For sensor-mode=1: The exposure values from [34000,33333333] actually change the exposure time. However, the range [33333334,358733008] does not produce a warning message and defaults the exposure time to some arbitrary default value which seems to be relatively dim.

Here are some extra questions:
If sensor-mode=1 is 60 fps, how is it that we can set the exposure time to 33,333,333ns (i.e. 1/30s), instead of 16,666,666ns

How can I have an exposure time over 1/30s. My application is very low light.

Hello everybody,

we are using the RidgeRun drivers on Jetson Nano DK (JP4.4/L4T 32.4.3), but having a little issue (the video starts to blink yellow) when accidentally looking at the sun (which is very common in our use case).

This was recorded from our system using gstreamer:

We were able to reproduce it on lab tests using a flashlight:

I don’t know if it’s really a driver issue or just the expected behaviour when these conditions are met. Anyways, as we are not experts on the field of digital video, some advice would be nice.

As context, we are using nvarguscamerasrc with default parameters, maybe there’s some tuning we can try there?

Thank you very much in advance.

Hi everyone,

I’m happy to announce that the IMX477 driver porting for Jetpack 4.4.1 is ready for you all to give it a try. You can check it out in our Github repo:

https://github.com/RidgeRun/NVIDIA-Jetson-IMX477-RPIV3

The patches and the debian packages were updated. Don’t hesitate to let us know in case you find any issue.

Regards,
Carlos R

2 Likes

Hey @jbarria,

Could you please try disabling the Auto Exposure in nvarguscamerasrc? You can run the pipeline with:

nvarguscamerasrc aelock=true

I just want to see if the issue appears even if auto exposure is disabled.

You can also try with awblock=true and see how it behaves

@David:

Thanks for the new version, its working here on an full updated nano 4GB!
I marked the 2 files with apt-mark hold so that they are not update.
Just in time, we have an telco today with an potential customer - so wen can report that we can start with the demonstrator application programming.

BR
Erich

Thank you very much for your response @CarlosR92 . I just answered in the github issue to avoid spamming on this thread.

Regards.
Jorge

Hello again,

I want to start the jetson_io.py tool, but it is not starting correctly.
The window shortly is visible for a fraction of a second and then it dissapers.
No error message.

I tried the init.py hack and the dtb file is from your hack in the /boot/dtb directory.
Could there be a problem with your hack and the jetson_io tool?

BR
Erich

Hi @CarlosR92

Auto exposure still works even though I disabled auto exposure and set the exposure timerange.
When I see the laser beam, It flashes like @jbarria and stops.

My pipeline is shown below.

cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)4032, height=(int)3040,format=(string)NV12, framerate=(fraction)30/1, exposuretimerange = "34000 34000", aeLock=true, ispdigitalgainrange="1 1", gainrange="4 4" ! tee name=t t. ! queue ! nvv4l2h264enc bitrate=14000000 ! h264parse ! matroskamux ! filesink location=now.mkv t. ! queue ! nvvidconv ! video/x-raw, format=(string)I420, width=1000, height=700 ! appsink sync=0 ")

flashsquirrel
I think your camera configs are in the wrong location of the pipe, those commands are part of nvarguscamerasrc.
try:
nvarguscamerasrc exposuretimerange=“34000 34000”, aeLock=true, ispdigitalgainrange=“1 1”, gainrange=“4 4” | all the other stuff.

also if you try your pipe in the terminal does it work with nvoverlaysink?
Such as:
gst-launch-1.0 nvarguscamerasrc exposuretimerange="34000 34000 " wbmode=0 sensor_id=0 sensor-mode=1 ispdigitalgainrange=“1 1” aelock=true gainrange=“4 4” ! nvoverlaysink

1 Like

@CarlosR92
@DavidSoto-RidgeRun,
@juan.cruz
@JerryChang

Hello, I hope the above is clear, and if you have questions feel free to ask.
I’m currently married to the Raspberry Pi HQ Camera for my project and hope I can get long exposure time with the Jetson Nano (because I’m using two cameras). But if it’s not possible at the current stage of the driver, I’ll switch back to using a two Raspberry Pi 4.

I hope I’m not being rude for pinging all 4 of you, but I have reviewed your posts and it looks like you can provide advice on the future state of this project and capabilities.

Hi @siderskiy,

Sorry for the slow response. We have been pretty busy and we had the task of being sure that the sensor works correctly in JP 4.5, we hope to release that in a week or two. Moreover, one of our engineers will start testing the exposure control as you described here in order to see if we can replicate your issue. I think we can have news in one week due to the work of active customers.

-David

1 Like

Thanks. Just saw the post that you updated the Wiki on how to install the driver.
I was using the Wiki instructions to install back then so maybe I missed the installation of the ISP calibration file? Could that have caused the exposure time issue?

I have a question about the Arducam driver. Do you need to use Arducam’s IMX477 camera, or the stock RaspberryPi HD camera work too?

I was under the suspicion that their PCB layout is different and pairs with their driver.

or are you using the one camera (the RaspberryPi HD camera ) and swapping between the drivers (and both work). If so, how do you swap the drivers?

@siderskiy

I have Arducam’s IMX477, but currently I build patch the kernel and build from source since other changes need to be made as well for our use case. As far as I know, there are no differences Arducam has made. I use the RidgeRun patches directly.

The stock camera should work as well if modified, but I’m hesitant to do it since they would no longer work with my Pi boards. Besides, the arducam ones are the same price or cheaper and just as good. They even have a version that can toggle the ir filter on and off.

If you’re US based, just search arducam imx477 on amazon and you’ll get lots of results.

1 Like

Hi @siderskiy

Regarding the minimum exposure of 34000 (34us), this value is hardcoded in nvarguscamerasrc. I patched the element to allow a smaller value in case you want (patch attached). With that modification it will allow you to set an exposure of 10000 (10us).

Regarding the maximum exposure values, as you said, for 60fps the maximum exposure is 16.6ms. If you try to set something beyond that, the exposure will be configured to 8.33 ms.

I updated the way the driver is configuring the sensor for a desired exposure and will be uploading the update in the next couple of days.

An example pipeline that sets the exposure to 10ms is the following:

gst-launch-1.0 nvarguscamerasrc exposuretimerange="10000000 10000000" wbmode=0 sensor_id=0 ispdigitalgainrange="1 1" aelock=true gainrange="4 4" ! 'video/x-raw(memory:NVMM),width=1920,height=1080,format=NV12,framerate=60/1' ! queue ! nvoverlaysink sync=false

nvarguscamerasrc.patch (2.4 KB)

Hope this helps.

2 Likes