Got different fps when using V4l2 and Gst-launch nvarguscamera

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
,
1

Hi!

I have a official orin nano 8g devkit which can drive imx577, it’s flashed with jetpack 5.1.1:

image
image691×1195 50.3 KB

The result of v4l2-ctl --list-formats-ext -d /dev/video1 here:

ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'RG10' (10-bit Bayer RGRG/GBGB)
                Size: Discrete 4056x3040
                        Interval: Discrete 0.025s (40.000 fps)
                Size: Discrete 4056x2288
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 2024x1144
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 2028x1520
                        Interval: Discrete 0.008s (130.000 fps)
                Size: Discrete 2028x1112
                        Interval: Discrete 0.005s (240.000 fps)
                Size: Discrete 4056x3040
                        Interval: Discrete 0.025s (40.000 fps)
                Size: Discrete 4056x2288
                        Interval: Discrete 0.017s (60.000 fps)

I’m trying to test whether the sensor mode 4 can reach 240 fps, when I try with

v4l2-ctl  -d  /dev/video1  --set-ctrl sensor_mode=4,bypass_mode=0,frame_rate=240000000 --stream-mmap

1af807eada0f1d39940da0c63be3b92
1af807eada0f1d39940da0c63be3b922843×129 6.36 KB

I can got the result of 238.

But when I use gst-launch with nvarguscamera

gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM), width=(int)2028, height=(int)1112, format=(string)NV12, framerate=(fraction)240/1" ! nvvidconv ! fpsdisplaysink text-overlay=false video-sink=fakesink --verbose

can only get a framerate below 110:

11fdca0401e56de7d13ffedaee5240f
11fdca0401e56de7d13ffedaee5240f2595×729 52.5 KB

Looks like there are kinds of limitations when using the nvarguscamera plugin, but no idea on what’s happening. Beg for your help,

3

hello EmondCai,

it looks nvarguscamerasrc already select sensor-mode=4 for streaming.
could you please try below commands to boost all the VI/CSI/ISP clocks.
for instance,

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

BTW,
is it possible for moving to the latest release, Jetpack-5.1.2 for quick confirmation?

4

still the same result even I boost the clock.

I tryied the gst command with nvarguscamerasec, the framerate is about 100 to 115. And can’t get any result with the v4l2 command on 5.1.2 so we choose 5.1.1

5

hello EmondCai,

please also give it a try with 2028x1520@130-fps sensor mode,
let’s check it could reach 130-fps via nvarguscamerasrc plugin?

6

2028_1520_130fps
2028_1520_130fps1874×2017 789 KB

The 2028x1520@130-fps also can’t get the highest frame via nvarguscamerasrc .

Both can got the highest-frame with v4l2

v4l2
v4l21888×308 31.9 KB

7

hello EmondCai,

may I have your helps, as I don’t have such high frame-rate sensor to test this locally.
let’s try convert the format as I420, and adding queue elements to the gst pipeline.
for instance,
$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM), width=(int)2028, height=(int)1112, format=(string)NV12, framerate=(fraction)240/1" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! queue ! fpsdisplaysink text-overlay=false video-sink=fakesink --verbose

8

Hey Jerry,

Appreciate for your help, the pipeline you provide can reach a framerate over170+(no matter have queue or not in the pipeline), through still can’t get the highest.

image
image1991×1157 320 KB

Can you explain to me why convert the format as I420can get higher framerate?

1 Like
9

For mode 2028x1520@130fps, the pipeline can reach the highest framerate:

image
image1897×1059 225 KB

11

Hi,
Along with enabling ISP and CSI at maximum clocks, please enable VIC at maximum clock by executing the script:
VPI - Vision Programming Interface: Performance Benchmark

And run the command with fakesink sync=0:

$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM), width=(int)2028, height=(int)1520, format=(string)NV12, framerate=(fraction)130/1" ! nvvidconv ! fpsdisplaysink text-overlay=false video-sink=fakesink sync=0 --verbose
12

Hi,
The framerate I can get goes up to over 206 after I use the execute the clock.sh():

6edb1a864e5d4c50c24b006756af730
6edb1a864e5d4c50c24b006756af730977×81 6.82 KB

e038422d002460461f6cd193af3daf8
e038422d002460461f6cd193af3daf82069×1121 230 KB

13

hello EmondCai,

thanks for testing, it might be performance bottleneck as we’ve never test with such high frame-rate sensors.
let me check this with dev team internally.

15

Okay, thank you so much with your support

16

hello EmondCai,

BTW, may I have more details of your 206-fps results.
besides boosting ISP, CSI, and VIC clock to maximum. do you still doing the trick with converting the format as I420 ?

17

The result of pipline with I420 here:

image
image1371×1215 78.3 KB

18

In summary, before executing the clock.sh:

110fps

gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM), width=(int)2028, height=(int)1112, format=(string)NV12, framerate=(fraction)240/1" ! nvvidconv ! fpsdisplaysink text-overlay=false video-sink=fakesink --verbose

174fps

 gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM), width=(int)2028, height=(int)1112, format=(string)NV12, framerate=(fraction)240/1" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! queue ! fpsdisplaysink text-overlay=false video-sink=fakesink --verbose

After executing the clock.sh:
206fps:

$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM), width=(int)2028, height=(int)1112, format=(string)NV12, framerate=(fraction)240/1" ! nvvidconv ! fpsdisplaysink text-overlay=false video-sink=fakesink sync=0 --verbose

182fps:

 gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM), width=(int)2028, height=(int)1112, format=(string)NV12, framerate=(fraction)240/1" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! queue ! fpsdisplaysink text-overlay=false video-sink=fakesink --verbose
19

May I also ask about the difference between these commands and the clock.sh your provide? Both of them are useing for boosting the VI/CSI/ISP clocks?

20

as you can see, those were command to boost VI/ISP/NVCSI/EMC clocks.
the script file, clocks.sh has further to increase VIC clock.

21

Hi JerryChang

For these two ways to boost clock, only executing the clock.sh can make a difference on the highest framerate. Could you please explain why boost these clocks can make the sensor got higher framerate with gstreamer + nvarguscamerasrc and Why the v4l2 command doesn’t require the boost.

Many thanks

22

hello EmondCai,

FYI,
v4l and nvarguscamerasrc they’re using different pipelines, as nvarguscamerasrc had more loadings at buffer transmits, for example, EGL is used by ARGUS APIs for buffer/stream management.
in addition, it might be performance bottleneck of video converter as well, (as you executed clocks.sh to to increase VIC clock shows better fps results)

hence,
please take your 206-fps evaluation results in comment #18 for current nvarguscamerasrc maximum frame-rate.

let me emphasize this again, we’ve never test such high frame-rate sensors.
please contact with sensor vendor for further supports. or, you may based-on 60-fps, or 120-fps sensor modes for your camera solution development.

24

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.