Frame drop in gstreamer while using two raw dump with v4l2src

Robotics & Edge Computing Jetson Systems Jetson Xavier NX
1

Hi all,

we are using xavier nx with gstreamer 1.20.7, while playing two pipeline with BGRx dump + GRAY8 dump we are observing frame drops.

Also,

nvpmodel -m 0
sudo jetson_clocks

already enabled.

Our observations here are:

  1. 20 fps while recording (BGRx) alone.
  2. 20 fps while recording (GRAY8) alone.
  3. 20 fps while recording (BGRx) & 17 to 18 fps approx. while recording (GRAY8) parallely.

what could be the reason for this frame drop?

Pipeline for BGRx:
gst-launch-1.0 -e v4l2src device=/dev/video4 ! ‘video/x-raw, format=YUY2, width=1920, height=1082, framerate=20/1’ ! queue ! perf name=bef_vc ! nvvidconv ! perf name=aft_vc ! ‘video/x-raw, format=BGRx’ ! filesink location=raw.bgrx

Pipeline for GRAY8:
gst-launch-1.0 v4l2src device=/dev/video0 ! ‘video/x-raw, format=(string)GRAY8, width=(int)1600, height=(int)1300’ ! perf name=v4l2 ! queue ! filesink location=gray8_dump.raw

Any inputs would be of great help.

Thanks,
Jai

3

Hi,
Running the two pipelines takes CPU resources and CPU capability dominates performance. You may run $ sudo tegrastats to check whether CPU cores are at 100% loading.

One thing you may try is to use nvv4l2camerasrc to capture YUY2 data. By default it supports UYVY and you can apply this patch to enable YUY2:
Macrosilicon USB - #5 by DaneLLL

So that certain loading is shifted to hardware converter. Should have some improvement.

Source code of nvv4l2camerasrc is in

Jetson Linux | NVIDIA Developer
Driver Package (BSP) Sources

4

Hi DaneLLL,

Thank you for the reply.

I have applied the patch and able to use nvv4l2camerasrc for YUY2, but however i am still receiving 17 fps in GRAY8 when both pipelines are played concurrently.

Attaching pipeline and logs for reference.

GRAY8:
gst-launch-1.0 v4l2src device=/dev/video0 ! “video/x-raw, format=(string)GRAY8, width=(int)1600, height=(int)1300” ! perf name=v4l2 ! queue ! filesink location=gray8_dump.raw
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
INFO:
perf: v4l2; timestamp: 0:09:44.485029161; bps: 0.000; mean_bps: 0.000; fps: 0.000; mean_fps: 0.000
Redistribute latency…
INFO:02.1 / 99:99:99.
perf: v4l2; timestamp: 0:09:45.496862447; bps: 249600000.000; mean_bps: 249600000.000; fps: 19.766; mean_fps: 19.766
INFO:03.1 / 99:99:99.
perf: v4l2; timestamp: 0:09:46.549741915; bps: 316160000.000; mean_bps: 282880000.000; fps: 17.096; mean_fps: 18.431
INFO:04.1 / 99:99:99.
perf: v4l2; timestamp: 0:09:47.559269088; bps: 266240000.000; mean_bps: 277333333.333; fps: 16.840; mean_fps: 17.901
INFO:05.2 / 99:99:99.
perf: v4l2; timestamp: 0:09:48.592874959; bps: 299520000.000; mean_bps: 282880000.000; fps: 19.350; mean_fps: 18.263
INFO:06.1 / 99:99:99.
perf: v4l2; timestamp: 0:09:49.643123323; bps: 299520000.000; mean_bps: 286208000.000; fps: 15.234; mean_fps: 17.657
INFO:07.3 / 99:99:99.
perf: v4l2; timestamp: 0:09:50.736435366; bps: 266240000.000; mean_bps: 282880000.000; fps: 16.464; mean_fps: 17.458
INFO:08.3 / 99:99:99.
perf: v4l2; timestamp: 0:09:51.769187433; bps: 282880000.000; mean_bps: 282880000.000; fps: 19.366; mean_fps: 17.731
^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:09.431836286
Setting pipeline to NULL …
Freeing pipeline …

BGRx:
gst-launch-1.0 -e nvv4l2camerasrc device=/dev/video4 ! “video/x-raw(memory:NVMM), format=YUY2, width=1920, height=1082, framerate=20/1” ! queue ! perf name=bef_vc ! nvvidconv ! perf name=aft_vc ! “video/x-raw, format=BGRx” ! filesink location=raw.bgrx
Setting pipeline to PAUSED …

Pipeline is live and does not need PREROLL …
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
INFO:
perf: bef_vc; timestamp: 0:09:37.885248861; bps: 0.000; mean_bps: 0.000; fps: 0.000; mean_fps: 0.000
INFO:
perf: aft_vc; timestamp: 0:09:37.902441900; bps: 0.000; mean_bps: 0.000; fps: 0.000; mean_fps: 0.000
Redistribute latency…
INFO:01.2 / 99:99:99.
perf: bef_vc; timestamp: 0:09:38.885824530; bps: 9216.000; mean_bps: 0.000; fps: 24.986; mean_fps: 24.986
INFO:01.3 / 99:99:99.
perf: aft_vc; timestamp: 0:09:38.946579979; bps: 1196605440.000; mean_bps: 0.000; fps: 24.901; mean_fps: 24.901
INFO:02.3 / 99:99:99.
perf: bef_vc; timestamp: 0:09:39.935583676; bps: 10240.000; mean_bps: 10240.000; fps: 20.005; mean_fps: 22.495
INFO:
perf: aft_vc; timestamp: 0:09:39.995559095; bps: 1329561600.000; mean_bps: 1329561600.000; fps: 20.019; mean_fps: 22.460
INFO:03.3 / 99:99:99.
perf: bef_vc; timestamp: 0:09:40.935676094; bps: 10240.000; mean_bps: 10240.000; fps: 19.998; mean_fps: 21.663
INFO:03.4 / 99:99:99.
perf: aft_vc; timestamp: 0:09:41.045545963; bps: 1329561600.000; mean_bps: 1329561600.000; fps: 20.000; mean_fps: 21.640
INFO:04.3 / 99:99:99.
perf: bef_vc; timestamp: 0:09:41.935783772; bps: 10240.000; mean_bps: 10240.000; fps: 19.998; mean_fps: 21.247
INFO:04.4 / 99:99:99.
perf: aft_vc; timestamp: 0:09:42.045873822; bps: 1329561600.000; mean_bps: 1329561600.000; fps: 19.993; mean_fps: 21.229
INFO:05.3 / 99:99:99.
perf: bef_vc; timestamp: 0:09:42.993726160; bps: 10240.000; mean_bps: 10240.000; fps: 19.850; mean_fps: 20.967
INFO:05.4 / 99:99:99.

Please let me know if there is anyother thing we can do for acheiving 20 fps in concurrent streams.

Thanks,
Jai

5

Hi,
Probably the bootleneck is file IO speed. Please try the command and see if it is at 20fps:

$ gst-launch-1.0 -v v4l2src device=/dev/video0 ! “video/x-raw, format=(string)GRAY8, width=(int)1600, height=(int)1300” ! perf name=v4l2 ! queue ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0
6

Hi DaneLLL,

Thanks for the pipeline. Using the same i am receiving only 17 fps approximately. attaching last few logs.

perf: v4l2; timestamp: 0:06:57.791345643; bps: 282880000.000; mean_bps: 305760000.000; fps: 16.826; mean_fps: 18.376
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 160, dropped: 0, current: 21.34, average: 18.68
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 170, dropped: 0, current: 14.53, average: 18.37
INFO:10.5 / 99:99:99.
perf: v4l2; timestamp: 0:06:58.795568350; bps: 299520000.000; mean_bps: 305066666.667; fps: 19.916; mean_fps: 18.547
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 182, dropped: 0, current: 23.45, average: 18.63
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 193, dropped: 0, current: 20.25, average: 18.72
INFO:11.6 / 99:99:99.
perf: v4l2; timestamp: 0:06:59.846253404; bps: 282880000.000; mean_bps: 302848000.000; fps: 19.987; mean_fps: 18.691
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 203, dropped: 0, current: 13.23, average: 18.34
INFO:12.5 / 99:99:99.
perf: v4l2; timestamp: 0:07:00.891258980; bps: 332800000.000; mean_bps: 305570909.091; fps: 13.397; mean_fps: 18.210
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 213, dropped: 0, current: 19.26, average: 18.39
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 221, dropped: 0, current: 15.67, average: 18.27
INFO:13.5 / 99:99:99.
perf: v4l2; timestamp: 0:07:01.891959544; bps: 249600000.000; mean_bps: 300906666.667; fps: 16.988; mean_fps: 18.108
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 230, dropped: 0, current: 17.90, average: 18.26
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 241, dropped: 0, current: 20.14, average: 18.33
INFO:14.5 / 99:99:99.
perf: v4l2; timestamp: 0:07:02.910839660; bps: 282880000.000; mean_bps: 299520000.000; fps: 20.611; mean_fps: 18.300
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 251, dropped: 0, current: 14.08, average: 18.12
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 259, dropped: 0, current: 13.77, average: 17.94
INFO:15.6 / 99:99:99.
perf: v4l2; timestamp: 0:07:03.987616480; bps: 266240000.000; mean_bps: 297142857.143; fps: 13.002; mean_fps: 17.922
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 270, dropped: 0, current: 17.54, average: 17.93
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 281, dropped: 0, current: 21.69, average: 18.05
INFO:16.7 / 99:99:99.
perf: v4l2; timestamp: 0:07:05.046934153; bps: 266240000.000; mean_bps: 295082666.667; fps: 17.936; mean_fps: 17.923
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 289, dropped: 0, current: 13.36, average: 17.87
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 298, dropped: 0, current: 16.68, average: 17.84
INFO:17.8 / 99:99:99.
perf: v4l2; timestamp: 0:07:06.061578123; bps: 282880000.000; mean_bps: 294320000.000; fps: 16.755; mean_fps: 17.850
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 308, dropped: 0, current: 19.51, average: 17.89
^Chandling interrupt.

Is there anything wrong? or anyother potential issues that could cause this fps drop?

Thanks,
Jai

7

Hi,

Looks like the camera source cannot reach 20fps. Which interface is used? CSI, USB or PCIe?

8

Hi DaneLLL,

The camera is using CSI interface.
Also, when plyed induvidually it is able to reach 20fps without any issues.

gst-launch-1.0 v4l2src device=/dev/video0 ! “video/x-raw, format=(string)GRAY8, width=(int)1600, height=(int)1300” ! perf name=v4l2 ! queue ! filesink location=gray8_dump.raw
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
INFO:
perf: v4l2; timestamp: 0:01:59.601331658; bps: 0.000; mean_bps: 0.000; fps: 0.000; mean_fps: 0.000
Redistribute latency…
INFO:01.6 / 99:99:99.
perf: v4l2; timestamp: 0:02:00.635240737; bps: 149760000.000; mean_bps: 0.000; fps: 23.213; mean_fps: 23.213
INFO:02.6 / 99:99:99.
perf: v4l2; timestamp: 0:02:01.679699002; bps: 332800000.000; mean_bps: 332800000.000; fps: 20.106; mean_fps: 21.659
INFO:03.7 / 99:99:99.
perf: v4l2; timestamp: 0:02:02.723706752; bps: 332800000.000; mean_bps: 332800000.000; fps: 20.115; mean_fps: 21.145
INFO:04.7 / 99:99:99.
perf: v4l2; timestamp: 0:02:03.767892677; bps: 349440000.000; mean_bps: 338346666.667; fps: 20.111; mean_fps: 20.886
INFO:05.8 / 99:99:99.
perf: v4l2; timestamp: 0:02:04.812068630; bps: 332800000.000; mean_bps: 336960000.000; fps: 20.112; mean_fps: 20.731
INFO:06.8 / 99:99:99.
perf: v4l2; timestamp: 0:02:05.856060561; bps: 332800000.000; mean_bps: 336128000.000; fps: 20.115; mean_fps: 20.629
INFO:07.8 / 99:99:99.
perf: v4l2; timestamp: 0:02:06.900264811; bps: 332800000.000; mean_bps: 335177142.857; fps: 20.111; mean_fps: 20.555
INFO:08.9 / 99:99:99.
perf: v4l2; timestamp: 0:02:07.944352065; bps: 332800000.000; mean_bps: 334880000.000; fps: 20.113; mean_fps: 20.500
^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:09.400895554
Setting pipeline to NULL …
Freeing pipeline …

Thanks,
Jai

9

Hi,
Please boost the clock and check if it helps:

Jetson/l4t/Camera BringUp - eLinux.org

If the issue persists, would suggest inspect whether power supply to the two cameras are sufficient. It seems like power may be insufficient for launching two cameras simultaneously.

10

Hi DaneLLL,

we have tried boosting the clocks however we are still receiving 17 fps.

we have observed that GR3D frequency is 0% in tegrastats. while we are using NVMM memory it should utilize the GR3D right?

08-07-2025 06:07:12 RAM 1062/15524MB (lfb 37x4MB) CPU [0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729] EMC_FREQ 0%@2133 GR3D_FREQ 0%@[305,0] VIC_FREQ 115 APE 174 CV0@51.187C CPU@53.937C SOC2@50.812C SOC0@52.343C CV1@50.906C GPU@51.281C tj@55.093C SOC1@55.093C CV2@51.093C
08-07-2025 06:07:13 RAM 1062/15524MB (lfb 37x4MB) CPU [0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729] EMC_FREQ 0%@2133 GR3D_FREQ 0%@[305,0] VIC_FREQ 115 APE 174 CV0@51.312C CPU@53.718C SOC2@50.781C SOC0@52.187C CV1@51.125C GPU@51.093C tj@55C SOC1@55C CV2@51C
08-07-2025 06:07:14 RAM 1062/15524MB (lfb 37x4MB) CPU [0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729] EMC_FREQ 0%@2133 GR3D_FREQ 0%@[305,0] VIC_FREQ 115 APE 174 CV0@51.218C CPU@53.593C SOC2@50.656C SOC0@52.125C CV1@50.937C GPU@51.062C tj@55.25C SOC1@55.25C CV2@50.937C
08-07-2025 06:07:15 RAM 1062/15524MB (lfb 37x4MB) CPU [0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729] EMC_FREQ 0%@2133 GR3D_FREQ 0%@[305,0] VIC_FREQ 115 APE 174 CV0@51.125C CPU@53.468C SOC2@50.812C SOC0@52.343C CV1@50.875C GPU@50.937C tj@55.031C SOC1@55.125C CV2@51.125C
08-07-2025 06:07:16 RAM 1062/15524MB (lfb 37x4MB) CPU [0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729,0%@729] EMC_FREQ 0%@2133 GR3D_FREQ 0%@[305,0] VIC_FREQ 115 APE 174 CV0@51.062C CPU@53.875C SOC2@50.687C SOC0@52.218C CV1@50.75C GPU@51.093C tj@55.031C SOC1@55.031C CV2@51.062C

Are we missing something here?

Thanks,
Jai.

11

Hi,
NVMM buffer and GR3D loading are independent. If GPU is not in-use, the loading is 0%.

CPU, EMC_FREQ, GR3D_FREQ are all 0% in your tegrastats. This looks incorrect. Please run with sudo like $ sudo tegrstats

And also suggest check from hardware perspective. Somehow the two camera sources are interfering each other while they are running simultaneously.

12

Hi DaneLLL,

we are using all commands with root user, for some reasons we have removed sudo.

we are checking with the HW team for the interference.

Is there anyother approaches to give a try for this issue?

Thanks,
Jai

13

Hi,
You may run both pipelines with fakesink to fully eliminate filesink, and see how it goes.