Camera's frame rate unstable

Hi

We are porting a camera driver for 4 lane 60 frame camera.
And the frame rate we got most of time is around 30 fps, rarely around 59 fps.

The command we used is
gst-launch-1.0 -vvv -e v4l2src device="/dev/video0" ! “video/x-raw, width=1920,height=1080, framerate=60/1, format=(string)UYVY” ! nvvidconv ! “video/x-raw(memory:NVMM), format=(string)I420, framerate=60/1” ! nvvidconv ! fpsdisplaysink sync=false

The command log is as attached.
And there is no error message from driver.

Is there any we missed?

Thank you for any advice,msg_1disp.txt (14.0 KB)

hello HuiW,

please exclude the last video converter plugin to check the sensor frame-rate.
for example,
$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

you may also have v4l2 standard control to check frame-rate,
for example,
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY--set-ctrl bypass_mode=0 --stream-mmap --stream-count=1000

Hi JerryChang,

Thank you for your prompt support.
Here are the results I got.

  1. gst-launch-1.0 fpsdisplay still got 30 fps. ( see below)
  2. v4l2-ctl got 60fps.

nvidia@nvidia-desktop:~$ gst-launch-1.0 v4l2src device="/dev/video0" ! “video/x-raw, width=1920,height=1080, framerate=60/1, format=(string)UYVY” ! nvvidconv ! “video/x-raw(memory:NVMM), width=640, height=320, format=(string)I420” ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)320, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420, pixel-aspect-ratio=(fraction)8/9
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)320, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420, pixel-aspect-ratio=(fraction)8/9
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)320, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420, pixel-aspect-ratio=(fraction)8/9
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)320, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420, pixel-aspect-ratio=(fraction)8/9
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)320, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420, pixel-aspect-ratio=(fraction)8/9
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)320, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420, pixel-aspect-ratio=(fraction)8/9
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 17, dropped: 0, current: 32.57, average: 32.57
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 33, dropped: 0, current: 31.53, average: 32.06
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 49, dropped: 0, current: 31.45, average: 31.86
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 65, dropped: 0, current: 31.71, average: 31.82
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 81, dropped: 0, current: 31.73, average: 31.80
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 97, dropped: 0, current: 31.72, average: 31.79
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 113, dropped: 0, current: 31.25, average: 31.71
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 129, dropped: 0, current: 31.34, average: 31.66
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 145, dropped: 0, current: 31.63, average: 31.66
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 161, dropped: 0, current: 31.39, average: 31.63
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 177, dropped: 0, current: 31.41, average: 31.61
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 193, dropped: 0, current: 31.43, average: 31.60
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 209, dropped: 0, current: 31.27, average: 31.57
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 225, dropped: 0, current: 31.32, average: 31.55
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 241, dropped: 0, current: 31.20, average: 31.53
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 257, dropped: 0, current: 31.59, average: 31.53
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 273, dropped: 0, current: 31.42, average: 31.53
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 289, dropped: 0, current: 31.42, average: 31.52
^Z
[1]+ Stopped gst-launch-1.0 v4l2src device="/dev/video0" ! “video/x-raw, width=1920,height=1080, framerate=60/1, format=(string)UYVY” ! nvvidconv ! “video/x-raw(memory:NVMM), width=640, height=320, format=(string)I420” ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

streamaccount=600
廿廿年九月十七日 (週四) 十七時三分四秒
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
廿廿年九月十七日 (週四) 十七時三分十四秒

Thank you for any advice,

hello HuiW,

may I know what’s the available sensor modes of your camera sensor had?
also, could you please check $ cat /etc/nv_tegra_release, to share the JetPack release version you’re working with.
thanks

Hi JerryChang,

Thank you for your prompt support.

Here are the information .

  1. one 1080p 60f only.

video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)60/1;

  1. nvidia@nvidia-desktop:~$ cat /etc/nv_tegra_release

R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020

nvidia@nvidia-desktop:~$

3.Jetpack4.4 DP

Thank you for any advice,

Hi,
Please share the information of executing v4l2-ctl -d /dev/video0 --list-formats-ext

And you may try

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1,format=UYVY' ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink sync=0 -v
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=59/1,format=UYVY' ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink sync=0 -v

Hi DaneLLL,

Thank you your prompt support.

Here is the information of v4l2-ctl .
nvidia@nvidia-desktop:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'NV16'
Name        : Y/CbCr 4:2:2
	Size: Discrete 1920x1080
		Interval: Discrete 0.017s (60.000 fps)

Index       : 2
Type        : Video Capture
Pixel Format: 'UYVY'
Name        : UYVY 4:2:2
	Size: Discrete 1920x1080
		Interval: Discrete 0.017s (60.000 fps)

Sorry. The command you provided did not work as I tested.

nvidia@nvidia-desktop:~$ gst-launch-1.0 nvv4l2camerasrc device="/dev/video0" ! “video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1,format=(string)UYVY” ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink sync=0 -v
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: sync = false
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
ERROR: from element /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.402842856
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

nvidia@nvidia-desktop:~$ gst-launch-1.0 nvv4l2camerasrc device="/dev/video0" ! “video/x-raw(memory:NVMM),width=1920,height=1080,framerate=59/1,format=(string)UYVY” ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink sync=0 -v
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: sync = false
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
ERROR: from element /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.401310191
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Thank you for any advice,

hello HuiW,

you don’t need to enable video converter for frame-rate prints,
please have another try to check your fps as following,
for example,

$ gst-launch-1.0 v4l2src device=/dev/video1 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080" ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

Hi Jerry Chang

Thank you for your prompt support.
Yes. The command you provided works.
Running the command gets 60 frame rate.

Would you suggest command for multi channels preview?
The multi channel command we used does not work now. (1st post)
To display multi channels preview, we need to enable video converter.

Another problem is we still got frame rate unstable on deepstream.
Most of time, launching with 30 fps and got 60 fps for a long run.
Should I create another topic or any suggestions?

Thank you,

hello HuiW,

please initial another new discussion thread since you’re able to reach 60-fps after exclude video converter.
thanks