Camera's frame rate unstable

hello HuiW,

may I know what’s the available sensor modes of your camera sensor had?
also, could you please check $ cat /etc/nv_tegra_release, to share the JetPack release version you’re working with.
thanks

Hi JerryChang,

Thank you for your prompt support.

Here are the information .

  1. one 1080p 60f only.

video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)60/1;

  1. nvidia@nvidia-desktop:~$ cat /etc/nv_tegra_release

R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020

nvidia@nvidia-desktop:~$

3.Jetpack4.4 DP

Thank you for any advice,

Hi,
Please share the information of executing v4l2-ctl -d /dev/video0 --list-formats-ext

And you may try

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1,format=UYVY' ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink sync=0 -v
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=59/1,format=UYVY' ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink sync=0 -v

Hi DaneLLL,

Thank you your prompt support.

Here is the information of v4l2-ctl .
nvidia@nvidia-desktop:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'NV16'
Name        : Y/CbCr 4:2:2
	Size: Discrete 1920x1080
		Interval: Discrete 0.017s (60.000 fps)

Index       : 2
Type        : Video Capture
Pixel Format: 'UYVY'
Name        : UYVY 4:2:2
	Size: Discrete 1920x1080
		Interval: Discrete 0.017s (60.000 fps)

Sorry. The command you provided did not work as I tested.

nvidia@nvidia-desktop:~$ gst-launch-1.0 nvv4l2camerasrc device=“/dev/video0” ! “video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1,format=(string)UYVY” ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink sync=0 -v
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: sync = false
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)60/1
ERROR: from element /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.402842856
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

nvidia@nvidia-desktop:~$ gst-launch-1.0 nvv4l2camerasrc device=“/dev/video0” ! “video/x-raw(memory:NVMM),width=1920,height=1080,framerate=59/1,format=(string)UYVY” ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink sync=0 -v
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: sync = false
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)59/1
ERROR: from element /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.401310191
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Thank you for any advice,

hello HuiW,

you don’t need to enable video converter for frame-rate prints,
please have another try to check your fps as following,
for example,

$ gst-launch-1.0 v4l2src device=/dev/video1 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080" ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

Hi Jerry Chang

Thank you for your prompt support.
Yes. The command you provided works.
Running the command gets 60 frame rate.

Would you suggest command for multi channels preview?
The multi channel command we used does not work now. (1st post)
To display multi channels preview, we need to enable video converter.

Another problem is we still got frame rate unstable on deepstream.
Most of time, launching with 30 fps and got 60 fps for a long run.
Should I create another topic or any suggestions?

Thank you,

hello HuiW,

please initial another new discussion thread since you’re able to reach 60-fps after exclude video converter.
thanks

Hi Jerry,

We try nvvideoconvert and scaling to 640 x 360. The frame rate will drop to ~30 fps. Is it normal?
$ gst-launch-1.0 v4l2src device=/dev/video0 ! “video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080” ! nvvideoconvert ! “video/x-raw(memory:NVMM), width=640, height=360, format=(string)I420” ! nvvideoconvert ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)360, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)360, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)360, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)360, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)360, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)360, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)360, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)360, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 17, dropped: 0, current: 33.20, average: 33.20
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 33, dropped: 0, current: 31.21, average: 32.21
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 49, dropped: 0, current: 31.30, average: 31.90
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 65, dropped: 0, current: 31.26, average: 31.74
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 81, dropped: 0, current: 31.20, average: 31.63

David

Hi, We fixed the issue by correct pipeline.

David

$gst-launch-1.0 v4l2src -v device=/dev/video0 ! video/x-raw ! nvvidconv ! “video/x-raw(memory:NVMM), format=I420, width=640, height=360” ! fpsdisplaysink text-overlay=0 video-sink=nv3dsink


/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 31, dropped: 0, current: 60.51, average: 60.51
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 61, dropped: 0, current: 59.93, average: 60.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 91, dropped: 0, current: 59.96, average: 60.14
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 121, dropped: 0, current: 59.95, average: 60.09
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 151, dropped: 0, current: 59.96, average: 60.06

Hi Jerry,

It’s weird if we reboot system, all status will be back to original. The frame rate will be dropped down to 38 ~40 fps when using nvvidconv or nvvideoconvert for scaling down. It’s normal without scaling down and can reach 60 fps. We try to use Deepstream 5.0 with primary-gie disable and just for preview. The result is the same. (~38fps).

We will create another topic for this issue.

David

hello davidcoo882,

am just wondering had you done any system configurations for performance changes?
since it may back to default states while you reboot the system.

please also share the topic-id here,
let’s tracking your follow-up questions in the new discussion thread.
thanks

Hi Jerry,

We already created another topic before. Please link here.

David

Hi Jerry,

We can solve low frame rate issue in deepstream with DaneLLL’s patch. How to implement in gstreamer with correct plugin?

David

Hi Jerry, do you have idea how to solve this issue in gstreamer?

David

Hi,
For multiple cameras capturing through v4l2, we would suggest use nvv4l2camerasrc and set VIC to max clock. Please check this sample pipeline:

And how to set VIC at max clock:

Hi DaneLLL,

We ran VIC in max clock at 1036800000. We try to use nvcompositor to combine all sources into a video wall but fail.

gst-launch-1.0 -v nvcompositor name=comp
sink_0::xpos=0 sink_0::ypos=0
sink_1::xpos=480 sink_1::ypos=0
sink_2::xpos=960 sink_2::ypos=0
sink_3::xpos=1440 sink_3::ypos=0
sink_4::xpos=0 sink_4::ypos=270
sink_5::xpos=480 sink_5::ypos=270
sink_6::xpos=960 sink_6::ypos=270
sink_7::xpos=1440 sink_7::ypos=270
! nvvideoconvert ! nvegltransform ! fpsdisplaysink text-overlay=0 video-sink=nveglglessink sync=0 -v
nvv4l2camerasrc device=/dev/video0 bufapi-version=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),width=480, height=270, format=NV12’ ! comp.sink_0
nvv4l2camerasrc device=/dev/video1 bufapi-version=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),width=480, height=270, format=NV12’ ! comp.sink_1
nvv4l2camerasrc device=/dev/video2 bufapi-version=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),width=480, height=270, format=NV12’ ! comp.sink_2
nvv4l2camerasrc device=/dev/video3 bufapi-version=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),width=480, height=270, format=NV12’ ! comp.sink_3
nvv4l2camerasrc device=/dev/video4 bufapi-version=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),width=480, height=270, format=NV12’ ! comp.sink_4
nvv4l2camerasrc device=/dev/video5 bufapi-version=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),width=480, height=270, format=NV12’ ! comp.sink_5
nvv4l2camerasrc device=/dev/video6 bufapi-version=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),width=480, height=270, format=NV12’ ! comp.sink_6
nvv4l2camerasrc device=/dev/video7 bufapi-version=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=60/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),width=480, height=270, format=NV12’ ! comp.sink_7


/GstPipeline:pipeline0/GstNvCompositor:comp.GstAggregatorPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)540, framerate=(fraction)60/1, format=(string)RGBA, chroma-site=(string)NULL, colorimetry=(string)sRGB
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)540, framerate=(fraction)60/1, format=(string)RGBA, chroma-site=(string)NULL, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstEglGlesSink:eglglessink0: sync = false
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.116894351
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

If we use nvstreammux and nvmultistreamtiler, it could work. May I know where we need to modify if we would like to use nvcompositor?

Thanks,
David

Hi,

The nvcompositor plugin is not compatible with DeepStream SDK. Please refer to

Hi DaneLLL,

May I know how to set VIC clock to max in dts?

David

Hi,
The VIC governor is configured by our boot init script “/etc/systemd/nv.sh”. You can modify the nv.sh script to set static max value for VIC clock.

Snippet of VIC configuration in /etc/systemd/nv.sh

case "${SOCFAMILY}" in
        tegra186 | tegra194)
                VIC_GOV_PARAM_PATH="/sys/devices/13e10000.host1x/15340000.vic"
                VIC_DEV_PATH="${VIC_GOV_PARAM_PATH}/devfreq/15340000.vic"

                if [ -e "${VIC_DEV_PATH}/available_governors" ]; then
                        read governors < "${VIC_DEV_PATH}/available_governors"

                        case "${governors}" in
                                *wmark_active*)
                                        DEVFREQ_WMARK_ACTIVE_GOV=1
                                ;;
                        esac
                fi

                if [ "${DEVFREQ_WMARK_ACTIVE_GOV}" -eq 1 ]; then
                        if [ -e "${VIC_DEV_PATH}/governor" ]; then
                                echo wmark_active > "${VIC_DEV_PATH}/governor"
                        fi
                        if [ -e "${VIC_GOV_PARAM_PATH}/load_target" ]; then
                                echo 700 > "${VIC_GOV_PARAM_PATH}/load_target"
                        fi
                        if [ -e "${VIC_GOV_PARAM_PATH}/load_max" ]; then
                               echo 900 > "${VIC_GOV_PARAM_PATH}/load_max"
                        fi
                        if [ -e "${VIC_GOV_PARAM_PATH}/block_window" ]; then
                                echo 0 > "${VIC_GOV_PARAM_PATH}/block_window"
                        fi
                        if [ -e "${VIC_GOV_PARAM_PATH}/smooth" ]; then
                                echo 0 > "${VIC_GOV_PARAM_PATH}/smooth"
                        fi
                        if [ -e "${VIC_GOV_PARAM_PATH}/freq_boost_en" ]; then
                                echo 0 > "${VIC_GOV_PARAM_PATH}/freq_boost_en"
                        fi
                fi
                ;;
        *)