I am looking to use v4l2 for long exposure captures (which will require further driver / DTB tweaks). However, it looks like I am seeing a few issues which prevent me from further testing raw bayer long exposures. Note that I’m exclusively testing on a Nano 2GB Dev Kit + Arducam IMX477 camera module (both regular + mini) - across both Nvidia stock and Arducam drivers.
$ cat /etc/nv_tegra_release # R32 (release), REVISION: 6.1, GCID: 27863751, BOARD: t210ref, EABI: aarch64, DATE: Mon Jul 26 19:20:30 UTC 2021 $ uname -a Linux developer-desktop 4.9.253-tegra #1 SMP PREEMPT Sat Oct 9 07:43:58 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
1). Regardless of using the Nvidia / Arducam driver, v4l2-ctl does not seem to respect set frame_rate control. On the stock Nvidia drivers, the commands below result in 60fps and 30fps output, respectfully. One interesting behavior is that does seem to respect frame_rate control after the stream has started if you run the command from another shell:
v4l2-ctl -c frame_rate=10000000
v4l2-ctl -d /dev/video0 --verbose --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0,sensor_mode=1,frame_rate=10 --stream-mmap --stream-count=100 v4l2-ctl -d /dev/video0 --verbose --set-fmt-video=width=3840,height=2160,pixelformat=RG10 --set-ctrl bypass_mode=0,sensor_mode=0,frame_rate=10 --stream-mmap --stream-count=100
2). Interestingly, Gstreamer does respect the user-defined frame rate running the following command (and this applies across all sensor modes, from their device tree specified min and max framerates, and even works at 4032x3040 from 2fps → 30fps when running the Arducam driver):
gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor-mode=0 ispdigitalgainrange='1 1' num-buffers=300 wbmode=3 aelock=true gainrange='1 1' exposuretimerange='300000000 300000000' ! 'video/x-raw(memory:NVMM),width=3840,height=2160,framerate=10/1' ! nvvidconv ! nvv4l2h265enc bitrate=100000000 ! h265parse ! mp4mux ! filesink location=out.mp4
3). v4l2 doesn’t seem to be able to lower the framerate to the specified minimum in the driver / DTB of 2fps. It appears 5fps is the minimum across all modes (including the additional 4032x3040 mode with the Arducam driver). Gstreamer, however, is able to use frame rates down to the specified min.
v4l2-ctl -d /dev/video0 --verbose --set-fmt-video=width=3840,height=2160,pixelformat=RG10 --set-ctrl bypass_mode=0,sensor_mode=0,frame_rate=2 --stream-mmap --stream-count=100 gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor-mode=0 ispdigitalgainrange='1 1' num-buffers=30 wbmode=3 aelock=true gainrange='1 1' exposuretimerange='300000000 300000000' ! 'video/x-raw(memory:NVMM),width=3040,height=2160,framerate=2/1' ! nvvidconv ! nvv4l2h265enc bitrate=100000000 ! h265parse ! mp4mux ! filesink location=out.mp4
4). v4l2 commands sometimes break Gstreamer until the next reboot, and vice versa. Though this doesn’t always happen… Gstreamer seems to break v4l2 more than the other way around. I haven’t isolated whether this behavior is driver dependent. When v4l2 is broken, it will throw this error when running the commands from above and throw this error:
video4linux video0: frame start syncpt timeout!0 Note there is a potentially related issue where v4l2 doesn’t ever work, even on the stock Nvidia drivers. It seems to vary from board to board (or camera to camera) and has yet to be isolated. See here for more details.
5). There’s also the curiosity of how Gstreamer is able to handle 4032x3040 30fps whereas v4l2 is not (when running the Arducam drivers). Based on the Nano’s MIPI CSI specs, 4032x3040 at 30fps seems to be pushing too much bandwidth, but somehow Gstreamer is able to handle it with no issues. The fact that Gstreamer can handle 4032x3040 at 30fps (and v4l2 can’t) doesn’t seem to make much sense to me as the MIPI CSI transfer occurs before the ISP processing. Any info on why this is happening would be appreciated.
Let me know if there’s any further info/debug that would be helpful. Thanks!