V4l2 modes not picked up if same resolution

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

I have a camera where I need to have different modes based on frame rate with the same resolution. The reason is I need to adjust the i2c settings based on frame rate.

Unfortunately tegra seems to ignore the frame rate and just pick modes based on resolution. What is the best way to overcome this?

I’d like the second entry to reflect what is is the mode tables of 60 fps, not 30 fps as the first

root@jetson-xavier-nx-devkit:~# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'Y16 ' (16-bit Greyscale)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.011s (90.000 fps)
3

Do you mean you can have driver report two sensor mode with same resolution but different frame rate?
If yes that could be your driver or dts have problem.

4

@ShaneCCC as an example of the problem I took the imx219 driver and cut it down to just two modes. The issue is somehow if the resolutions are the same the framerate is ignored. The second framerate should be 120 fps, but as you see it just uses the fps from the first mode

root@jetson-xavier-nx-devkit:~# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'RG10' (10-bit Bayer RGRG/GBGB)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)
5

I don’t have try this kind of case before. Usually we don’t report the duplicate resolution with different frame rate. We report can control by the frame rate control ID for the frame rate change.

6

That’s the problem. These below select the same mode 4, rather than mode 3 and mode 4 in the device tree. So, if in the device tree for different frame rates you have anything different you’re ignoring the device tree and just picking the last in the list. My issue is I have different line_length and so the correct mode needs to be selected based on frame rate.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, format=NV12, framerate=60/1' ! \
nvoverlaysink

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, format=NV12, framerate=120/1' ! \
nvoverlaysink
7

Are you able to check if argus_camera can select the mode by GUI interface?

8

i’m in yocto so I don’t have access to GUI.

FWIW, if I set use_sensor_mode_id = "false";

then it always selects mode 3, regardless of the frame rate so that’s weird.

9

Have a try using sensor-mode=3 in the pipeline to try.

10

Have reference to below patch to add duplicate size but different frame rate.
The result should like,
nvidia@nvidia-desktop:~$ ./v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘BG10’
Name : 10-bit Bayer BGBG/GRGR
Size: Discrete 2592x1944
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.008s (120.000 fps)

diff --git a/drivers/media/i2c/ov5693_mode_tbls.h b/drivers/media/i2c/ov5693_mode_tbls.h
index ee68db2..00d26c5 100644
--- a/drivers/media/i2c/ov5693_mode_tbls.h
+++ b/drivers/media/i2c/ov5693_mode_tbls.h
@@ -2380,14 +2380,19 @@ static const int ov5693_30fps[] = {
        30,
 };

+static const int ov5693_30_120fps[] = {
+        30,
+       120,
+};
+
 static const int ov5693_120fps[] = {
        120,
 };

 static const struct camera_common_frmfmt ov5693_frmfmt[] = {
        {{2592, 1944},  ov5693_30fps,   1, 0,   OV5693_MODE_2592X1944},
-       {{2592, 1458},  ov5693_30fps,   1, 0,   OV5693_MODE_2592X1458},
-       {{1280, 720},   ov5693_120fps,  1, 0,   OV5693_MODE_1280X720_120FPS},
+       {{1280, 720},   ov5693_30_120fps,       2, 0,   OV5693_MODE_2592X1458},
+//     {{1280, 720},   ov5693_120fps,  2, 0,   OV5693_MODE_1280X720_120FPS},
 #if ENABLE_EXTRA_MODES
        {{640, 480},    ov5693_30fps,   1, 0,   OV5693_MODE_640X480},
        {{1920, 1080},  ov5693_30fps,   1, 0,   OV5693_MODE_1920X1080},
11

@ShaneCCC you’ll note from the OP that my goal is to have the frame rate select the mode. I have seen your suggestion in the v4l2 documentation that NVIDIA has (here Welcome — Jetson Linux<br/>Developer Guide 34.1 documentation in case anyone else runs across this post ).

I’ve been through the code the last few days and it seems it should be in camera_common_try_fmt but I can’t figure it out.

12

@ShaneCCC to me it looks like what I want to do isn’t possible.

I can’t seem to get the v4l2src plugin to read the framerate from the caps. Basically, I never see my set_frame_rate function being called. Do you know where I could look to see the caps consumed that should then call the set_frame_rate function?

13

@ShaneCCC I should note that nvarguscamerasrc does call set_frame_rate and I’m basing off the imx219.c driver. But, as I’m using v4l2src is there a way to get set_frame_rate to be called?

14

You can set the frame by argus API setFrameDurationRange(), have a confirm with the argus_camera.

  --framerate=RATE  set the sensor frame rate to RATE. If RATE is 0 then VFR
                    (variable frame rate) is enabled. Valid values need to
                    be in the range [1.5, 30]. Default is '30'.

https://docs.nvidia.com/jetson/l4t-multimedia/classArgus_1_1ISourceSettings.html#a46f05e2816befd695fd3ae68d856cf96