V4l2-ctl frame_rate setting not taking effect if set before streaming

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
,
1

I’m using JetPack 5.0 with Xavier NX. I was able to bring up all my cameras with your help and they all shows max (default) fps when I send v4l2-ctl commands (v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RG10 --stream-mmap --stream-count=360 -d /dev/video0).

However it ignores all my --set-ctrl flag in the same message, for example: v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RG10 --set-ctrl frame_rate=60000000 --stream-mmap --stream-count=3600 -d /dev/video0 will still yield 100 fps.

v4l2-ctl --set-ctrl frame_rate=60000000 changes the frame_setting but no effect on the stream fps. There were also no camera power on/off info printed out in dmesg during this process. It was not this case in JP4.4. Is this expected behavior in JP5?

nvidia@tegra-ubuntu:~$ v4l2-ctl --set-ctrl frame_rate=60000000
nvidia@tegra-ubuntu:~$ v4l2-ctl -d /dev/video0 -l | grep frame_rate
                     frame_rate 0x009a200b (int64)  : min=10000000 max=100000000 step=1 default=100000000 value=60000000 flags=slider

Another test I did is requesting frames from one terminal and change the fps from another terminal and that does work.

nvidia@tegra-ubuntu:~$ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RG10 --set-ctrl frame_rate=100000000 --stream-mmap --stream-count=3600 -d /dev/video0
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 64.42 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 70.75 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 78.45 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 79.97 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 85.25 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 93.58 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 100.04 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 100.04 fps

The sudo media-ctl -p -d /dev/media0 looks ok, but it seems missing some numbers after 15a00000- in the line of - entity 1: 13e10000.host1x:nvcsi@15a00000- (2 pads, 2 links) . Maybe this is not related but thought might be worth mentioning. Thanks!

nvidia@tegra-ubuntu:~$ sudo media-ctl -p -d /dev/media0
[sudo] password for nvidia: 
Media controller API version 5.10.65

Media device information
------------------------
driver          tegra-camrtc-ca
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  5.10.65

Device topology
- entity 1: 13e10000.host1x:nvcsi@15a00000- (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "ov7251 30-0060":0 [ENABLED]
	pad1: Source
		-> "vi-output, ov7251 30-0060":0 [ENABLED]

- entity 4: 13e10000.host1x:nvcsi@15a00000- (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
	pad0: Sink
		<- "imx385 31-001a":0 [ENABLED]
	pad1: Source
		-> "vi-output, imx385 31-001a":0 [ENABLED]

- entity 7: ov7251 30-0060 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev2
	pad0: Source
		[fmt:SRGGB10_1X10/640x480 field:none colorspace:srgb]
		-> "13e10000.host1x:nvcsi@15a00000-":0 [ENABLED]

- entity 9: vi-output, ov7251 30-0060 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "13e10000.host1x:nvcsi@15a00000-":1 [ENABLED]

- entity 23: imx385 31-001a (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
	pad0: Source
		[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb]
		-> "13e10000.host1x:nvcsi@15a00000-":0 [ENABLED]

- entity 25: vi-output, imx385 31-001a (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Sink
		<- "13e10000.host1x:nvcsi@15a00000-":1 [ENABLED]
3

Suppose it could be the v4l2-ctl set configure sequence problem.
Could you print the frame rate set from the sensor driver to check the behavior.

Thanks

4

I tried to do that and added print line to the set_frame_rate function:

static int ov7251_set_frame_rate(struct tegracam_device *tc_dev, s64 val)
{
	struct camera_common_data *s_data = tc_dev->s_data;
	struct ov7251 *priv = (struct ov7251 *)tc_dev->priv;
	struct device *dev = tc_dev->dev;
	ov7251_reg reg_list[2];
	int err;
	u32 frame_length;
	const struct sensor_mode_properties *mode =
		&s_data->sensor_props.sensor_modes[s_data->mode_prop_idx];
	int i = 0;
	printk(">>>>> ov7251_set_frame_rate,val = %lld\n",val);
...
}

This function is linked to .set_frame_rate of tegracam_ctrl_ops:

static struct tegracam_ctrl_ops ov7251_ctrl_ops = {
...
	.set_frame_rate = ov7251_set_frame_rate,
...
};

This function seems never get called.

nvidia@tegra-ubuntu:~$ v4l2-ctl --set-ctrl frame_rate=100000000
nvidia@tegra-ubuntu:~$ v4l2-ctl -d /dev/video0 -l | grep frame_rate
                     frame_rate 0x009a200b (int64)  : min=10000000 max=100000000 step=1 default=100000000 value=100000000 flags=slider
nvidia@tegra-ubuntu:~$ v4l2-ctl --set-ctrl frame_rate=60000000
nvidia@tegra-ubuntu:~$ v4l2-ctl -d /dev/video0 -l | grep frame_rate
                     frame_rate 0x009a200b (int64)  : min=10000000 max=100000000 step=1 default=100000000 value=60000000 flags=slider

Dmesg didn’t print out anything during these two operations.

This is same for gain and exposure. Thanks!

5

Suppose the v4l2 framework will store the configure and set it after device open and stream on.
Please continuous investigate to figure it out.

Thanks

6

Does your driver report a bypass_mode control ?

v4l2-ctl -d0 --all

If yes, it may use this bypass to argus with ISP processing with various auto exposure/gain/wb tuning. Try adding:

--set-ctrl bypass_mode=0

to get rid of this.

7

Hi Honey_Patouceul, thanks for the suggestion. bypass_mode default value is 0 for my camera and I also tried passing bypass_mode=0 and there is no effect. Thanks!

8

Hi Shane,

I will debug more and keep you posted.
Do you know why there is no -1 or -2 after nvcsi@15a00000- in the media-ctl -p results?

9

Hi Shane,

I just checked on production Xavier NX and a Xavier NX dev kit carrier board with IMX219 using the stock driver/dtb.

I observed exactly same behavior:

I think this is a bug in JetPack 5.0 DP.

10

Doesn’t this problem show on J4.x?

11

I’m using a TX2 in JP4.4 as a reference for JP5 bringing up and this doesn’t happen in JP4.X on TX2.

I think I never tested frame_rate with v4l2 on Xavier NX in JP4.X, but I can test it and let you know.

12

Hi Shane,

I just flashed the same hardware (production Xavier NX and Xavier NX dev kit carrier board, IMX219) with JetPack4.6.1. And:

  • v4l2-ctl --set-ctrl not taking effect if set before or with the streaming cmd (same as JetPack 5.0).
  • media-ctl -p shows numbers at the end of the entity name.

I guess maybe the media-ctl -p info is a small bug, while the v4l2-ctl issue has been like that since Xavier NX? Thanks!

Screen Shot 2022-05-10 at 2.17.44 PM
Screen Shot 2022-05-10 at 2.17.44 PM1065×1047 209 KB

13

Does the same sensor on TX2 in J4.4 from your previous comment?

14

Yes. I was using the same sensor with TX2 and JP44 in my previous comment.

nvidia@tegra-ubuntu:~$ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RG10 --set-ctrl frame_rate=100000000 --stream-mmap --stream-count=360 -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 101.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 100.50 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 100.33 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
nvidia@tegra-ubuntu:~$ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RG10 --set-ctrl frame_rate=10000000 --stream-mmap --stream-count=360 -d /dev/video1
<<<<<<<<<<<< 10.09 fps
<<<<<<<<<< 10.04 fps
<<<<<<<<<< 10.03 fps
<<<<<<<<<< 10.02 fps
<<<<<<<<<< 10.01 fps
<<<<<<<<<< 10.01 fps
<<<<<<<<<< 10.01 fps
<<<<<<<<<< 10.01 fps
<<<<<<<<<< 10.01 fps
<<<<<<<<<< 10.00 fps
<<<<<<<<<< 10.00 fps
<<<<<<<<<< 10.00 fps
15

So it’s the same sensor for J4.4 and J4.6.1? And found J4.6.1 have the same problem?

Thanks

16

Sorry for the confusion. Seems like an Xavier NX issue. Maybe another VI5 issue?

  • JP4.4 + TX2 + any sensor, working good.

  • JP4.6.1 + Xavier NX + any sensor (tested IMX219 with stock driver/dtb, dev kit carrier board), got problem

  • JP5.0 + Xavier NX + any sensor (tested IMX219 with stock driver/dtb, dev kit carrier board), got problem

17

Yes, looks like the problem only for the Xavier base platform.
Looks like the CID was set before the stream_on() for Xavier base platform to cause the problem.

18

Looks like the problem cause by the override_enable parameter.
Have below command to set override_enable=1 for Xavier base platform to fix the problem.

v4l2-ctl -c override_enable=1
3 Likes
19

Hi Shane,

This works and it sets the capture parameters correctly when the override_enable=1 was passed together with the streaming request. Thank you so much!

21

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.