Why is the frame rate of v4l2 60fps, but gstreamer shows 41fps?

[wxsrite]

Hi all：
Why is the frame rate of v4l2 60fps, but gstreamer shows 41fps?

Device：
Orin NX

Soft Version：
jp5.1.2

Command:

gst-launch-1.0 -ev v4l2src device=/dev/video3 ! timeoverlay ! clockoverlay halignment=right valignment=top time-format=“%Y/%m/%d %H:%M:%S” ! video/x-raw, width=1920, height=1536 ! videoconvert ! fpsdisplaysink sync=false 2>&1

root@tegra:/home/robot# more /etc/nv_tegra_release
# R35 (release), REVISION: 4.1, GCID: 33958178, BOARD: t186ref, EABI: aarch64, DATE: Tue Aug  1 19:57:35 UTC 2023
root@tegra:/home/robot# v4l2-ctl --list-formats-ext --device /dev/video3
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1536
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1920x1280
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1380
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1440
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1600x1300
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1536
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1920x1280
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1380
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1440
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1600x1300
                        Interval: Discrete 0.033s (30.000 fps)

root@tegra:/home/robot# v4l2-ctl -d /dev/video3 --get-fmt-video
Format Video Capture:
        Width/Height      : 1920/1536
        Pixel Format      : 'YUYV' (YUYV 4:2:2)
        Field             : None
        Bytes per Line    : 3840
        Size Image        : 5898240
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             :
root@tegra:/home/robot# v4l2-ctl -d /dev/video3 --set-ctrl sensor_mode=1 --set-fmt-video=width=1920,height=1536 --stream-mmap
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 59.86 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<^C
root@tegra:/home/robot# gst-launch-1.0 -ev v4l2src device=/dev/video3 ! timeoverlay ! clockoverlay halignment=right valignment=top time-format="%Y/%m/%d %H:%M:%S" ! video/x-raw, width=1920, height=1536 ! videoconvert ! fpsdisplaysink sync=false  2>&1

DTS:
dts.txt (447.2 KB)

The CPU usage is not high either, so why do the displayed frame rates differ when the commands are different?
60fps:

gst-launch-1.0 -ev v4l2src device=/dev/video3 ! timeoverlay ! rawvideoparse format=4 width=1920 height=1536 ! videoconvert ! fpsdisplaysink video-sink=xvimagesink sync=false

52fps

gst-launch-1.0 -ev v4l2src device=/dev/video3 ! timeoverlay ! video/x-raw, width=1920, height=1536, framerate=60/1 ! videoconvert ! fpsdisplaysink sync=false 2>&1

41fps

gst-launch-1.0 -ev v4l2src device=/dev/video3 ! timeoverlay ! clockoverlay halignment=right valignment=top time-format=“%Y/%m/%d %H:%M:%S” ! video/x-raw, width=1920, height=1536, framerate=60/1 ! videoconvert ! fpsdisplaysink sync=false 2>&1

[proventusnova]

Hello @wxsrite,

It might be that the extra processing you are adding to the pipeline is taking a toll on your frame rate.

There are some steps you could take to debug the problem:

1. Add 1 change at the time in order to narrow down the element that is adding the overhead.
2. Check the output caps of the elements you are adding on top of your 60fps pipeline, this is because your videoconvert could be doing more work on one pipeline than in the other.
3. We would suggest using a different videosink or at least defining the videosink that the fpsdisplaysink is using so that you could have a bit more control of that.
4. We would suggest changing the pipeline to avoid using the videoconvert, that element tends to be very expensive.

Now, may we ask a couple of questions:

1. Whats your intended use case?
2. What do you want to achieve?
3. What are your goal metrics?

With this in mind, we could try to help you write a pipeline that fits your requirements.

regards,
Andrew

Home - Proventus Nova