# Severe frame dropping with multiple cameras on Xavier/GStreamer

**URL:** <https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772>\
**Category:** Jetson AGX Xavier\
**Created:** [September 16, 2019, 11:59am UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772 "2019-09-16T11:59:31Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![adev99](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@adev99](https://forums.developer.nvidia.com/u/adev99)\
**Post date:** [September 16, 2019, 11:59am UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/1 "2019-09-16T11:59:31Z")

</div>

Hi there,

I am running a 5-sensor (12MP/30fps) setup on Xavier w/ latest L4T R32.2.1 (problem also observed on R32.1). The image sensor driver is set up specifically for 5 sensors, and each sensor has been individually verified to provide a solid 30fps at 12MP. However, when attempting multiple simultaneous camera streams, I am seeing some strange bandwidth limitation with nvarguscamerasrc in GStreamer, even with the simplest of pipelines, as shown here:

gst-launch-1.0   
nvarguscamerasrc sensor-id=0 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! fakesink   
nvarguscamerasrc sensor-id=1 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! fakesink   
nvarguscamerasrc sensor-id=2 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! fakesink   
nvarguscamerasrc sensor-id=3 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! fakesink   
nvarguscamerasrc sensor-id=4 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! fakesink

This pipeline is not a problem on the TX2 with R28/nvcamerasrc using the same image sensors, and a very similar driver; all 5 sensors run steady at 30 fps, simultaneously. But on Xavier/R32/nvarguscamerasrc, I am getting a lot of frame drops, with each camera seeing unstable frame rates of 15-20 fps (as observed from gst-shark tracer data).

If I reduce the number of cameras, or reduce nvarguscamerasrc’s output resolution, the problem can be reduced or eliminated, indicating there is some unusual bandwidth limitation that should not be present. I should note that the driver only has one mode (4056x3040x30fps), so the fact that I can get a solid 30fps on all 5 cameras, when specifying a lower resolution in the caps (meaning the ISP has to do the scaling), indicates that the driver is successfully supplying 4056x3040x30fps to the ISP for all 5 sensors, simultaneously.

Hope you have some insight into this problem. All 8 of Xavier’s CPUs are running, mode is MAXN, and jetson\_clocks.sh has been run/stored to maximize all CPU/GPU clock frequencies.

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [September 17, 2019, 2:35am UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/2 "2019-09-17T02:35:36Z")

</div>

Hi,  
Please enable the property in nvarguscamerasrc:

```auto
maxperf : set or unset the max performace
                    flags: readable, writable
                    Boolean. Default: false

```

---

<div class="post-metadata">

**Author:** ![adev99](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@adev99](https://forums.developer.nvidia.com/u/adev99)\
**Post date:** [September 17, 2019, 7:47pm UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/3 "2019-09-17T19:47:01Z")

</div>

Hi DaneLLL,

Thanks for the reply, that did indeed allow all sensors to run at 30 fps with the example pipeline. However, if I add in some simple NV12 → I420 colour conversion to each camera output using nvvidconv, I again see frame drops, with performance that does not match what we’ve measured on nvcamerasrc/TX2/R28 (can run all 5 cameras at 30fps with colour conversion). Is there a way to optimize nvvidconv as well?

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [September 18, 2019, 4:51am UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/4 "2019-09-18T04:51:14Z")

</div>

Hi,  
The hardware engine(VIC) of nvvidconv should always run at max clocks. Looks not expected if you observe worse performance on Xavier. We need to check if we can reproduce the issue. We don’t have cameras with 4056x3040, but will try with what we have.

For more information, you also launch all cameras in one process on R28? Please share the command you run on R28.

---

<div class="post-metadata">

**Author:** ![adev99](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@adev99](https://forums.developer.nvidia.com/u/adev99)\
**Post date:** [September 24, 2019, 1:05am UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/5 "2019-09-24T01:05:03Z")

</div>

Hi Dane,

Here is the pipeline I run on TX2/R28:

gst-launch-1.0   
nvcamerasrc sensor-id=0 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink   
nvcamerasrc sensor-id=1 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink   
nvcamerasrc sensor-id=2 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink   
nvcamerasrc sensor-id=3 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink   
nvcamerasrc sensor-id=4 wbmode=3 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink

Each camera runs at 30fps no problem (maybe a dropped frame here or there).

Here is the equivalent pipeline I run on Xavier/R32:

gst-launch-1.0   
nvarguscamerasrc sensor-id=0 wbmode=3 maxperf=true num-buffers=300 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink   
nvarguscamerasrc sensor-id=1 wbmode=3 maxperf=true num-buffers=300 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink   
nvarguscamerasrc sensor-id=2 wbmode=3 maxperf=true num-buffers=300 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink   
nvarguscamerasrc sensor-id=3 wbmode=3 maxperf=true num-buffers=300 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink   
nvarguscamerasrc sensor-id=4 wbmode=3 maxperf=true num-buffers=300 ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=NV12, framerate=(fraction)30/1’ ! queue max-size-buffers=4 leaky=2 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)4056, height=(int)3040, format=I420, framerate=(fraction)30/1’ ! fakesink

Each camera has erratic frame rates that average ~21fps.

Thank you for your assistance.

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [September 24, 2019, 6:46am UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/6 "2019-09-24T06:46:33Z")

</div>

Hi,  
For more information, please replace ‘fakesink’ with ‘fpsdisplaysink video-sink=fakesink text-overlay=false’ and run with ‘-v’ to print out the frame rate information.

We don’t observe the issue but the board we have is six ov5693 running in 2592x1944. It still has a gap in required bandwidth comparing to yours.

---

<div class="post-metadata">

**Author:** ![adev99](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@adev99](https://forums.developer.nvidia.com/u/adev99)\
**Post date:** [September 24, 2019, 7:43pm UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/7 "2019-09-24T19:43:58Z")

</div>

Hi Dane,

I tried as you suggested, and the results matched my previous results using gst-shark (erratic frame rates going between 17 and 25 fps, averaging ~21fps.

Yes, unfortunately six ov5693’s running is only half the bandwidth we are using (30MP per set of frames, vs 60MP per set), which I believe is the reason you are not seeing the issue.

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [September 25, 2019, 1:53am UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/8 "2019-09-25T01:53:21Z")

</div>

Hi,  
For confirmation, you have flashed jetson-xavier-maxn.conf?

---

<div class="post-metadata">

**Author:** ![JerryChang](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/jerrychang/32/14043_2.png) [@JerryChang](https://forums.developer.nvidia.com/u/JerryChang)\
**Post date:** [September 25, 2019, 5:22am UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/9 "2019-09-25T05:22:59Z")

</div>

hello adev99,

could you please narrow down the issue,

1. let’s using v4l2 standard controls to exclude other component to check the CSI capability first.  
for example,

```auto
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=4056,height=3040,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10000
$ v4l2-ctl -d /dev/video1 --set-fmt-video=width=4056,height=3040,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10000
$ v4l2-ctl -d /dev/video2 --set-fmt-video=width=4056,height=3040,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10000
$ v4l2-ctl -d /dev/video3 --set-fmt-video=width=4056,height=3040,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10000
$ v4l2-ctl -d /dev/video4 --set-fmt-video=width=4056,height=3040,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10000

```

1. please reduce the number of cameras in the same pipeline, for example, please check you still see frame drops by launching only dual camera.

2. you might also try to exclude video converter, please remove nvvidconv component and use fakesink to analysis the frame-rate.

---

<div class="post-metadata">

**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\
**Post date:** [October 18, 2021, 6:23pm UTC](https://forums.developer.nvidia.com/t/severe-frame-dropping-with-multiple-cameras-on-xavier-gstreamer/81772/10 "2021-10-18T18:23:54Z")

</div>