# Performance drop when using multiple sources
**URL:** https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041
**Category:** DeepStream SDK
**Created:** [April 9, 2024, 8:43pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041 "2024-04-09T20:43:11Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 9, 2024, 8:43pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/1 "2024-04-09T20:43:11Z")
Please provide complete information as applicable to your setup.
**• Hardware Platform (Jetson / GPU)** Jetson Xavier AGX
**• DeepStream Version** 6.3.0
**• JetPack Version (valid for Jetson only)** 5.1
**• TensorRT Version** 8.5.2.2
I know AGX Xavier supports 52x 1080p30 (H.265), and you can even use `deepstream-test3-app` with several sources that you hardly see FPS being dropped. But in my script there is some kind of bottleneck, or even a synchronous behavior, that is reducing the FPS, almost 20% every source is added.
My source is an RTSP stream 1920x1080@25fps. Using 1 uridecodebin as a source bin, I get 24.2fps. With 2 source bins I get 18.2fps, with 3 source bins I get13.4fpf. And this decreases on and on as I increase the number of source bins. And it shouldn’t because AGX Xavier is capable to keep 24-25fps for many inputs like that. So I am missing something.
My pipeline goes like this:
```auto
uridecodebin [1..N] -> nvstreammux -> nvinfer -> nvtracker -> nvtee -> nvstreamdemux -> nvvideoconv[1..N] -> nvosd[1..N]
```
I read some posts on queue and how it can provide asynchronous behavior. So I added a queue between every element, just like this:
```auto
uridecodebin [1..N] -> nvstreammux -> queue1 -> nvinfer -> queue2 -> nvtracker -> queue3 -> nvtee -> queue4 -> nvstreamdemux -> queue_demux_conv[1..N]-> nvvideoconv[1..N] -> queue_conv_osd[1..N] -> nvosd[1..N]
```
If queues were an issue, there must be something more, because it made no difference to the decaying performance. Thus, I undid the inclusion of queues and I am back to the original pipeline. Does any one can give me any tip about what’s wrong? This is my script
[flavio\_forum.py.txt](https://forums.developer.nvidia.com/uploads/short-url/50Cz2ty2SWav704VFRJ4KEqNxv6.txt) (22.5 KB)
.
Thanks in advance.
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 10, 2024, 8:12am UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/3 "2024-04-10T08:12:37Z")
why do you comment out “pgie.set\_property(“batch-size”, number\_sources)”? if using multiple sources, the engine batch-size should be updated accordingly.
please refer to [deepstream\_test\_3.py](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/master/apps/deepstream-test3/deepstream_test_3.py). which has a similar media pipeline.
please refer to this [topic](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_troubleshooting.html#performance) for performance improvement.
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 10, 2024, 12:23pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/4 "2024-04-10T12:23:40Z")
batch-size=1 because “Backend has maxBatchSize 1 whereas 3 has been requested”. I am using Yolov5. I downloaded the yolov5s.pt and converted it to .onnx, and this onnx is the one being loaded.
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 11, 2024, 5:44am UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/5 "2024-04-11T05:44:13Z")
if commenting out “pgie.set\_property(“batch-size”, number\_sources)”, nvinfer will use fixed batch-size1 engine. it is not reasonable. if using multiple sources, nvinfer need to use a batch-size \>1 engine for a higher performance.
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 11, 2024, 1:59pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/6 "2024-04-11T13:59:11Z")
This is clear to me, but my point is the error (“Backend has maxBatchSize 1 whereas 3 has been requested”). The exporting from .pt to .onnx was done with --dymanics, which mean that the model should support a dynamic size of batches. So, when I use batch-size=3, for instance, the pipeline gives me such error, even though the model should support it.
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 11, 2024, 2:44pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/7 "2024-04-11T14:44:14Z")
could you share a whole log and nvinfer configuration file?
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 12, 2024, 6:06pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/8 "2024-04-12T18:06:28Z")
I managed to export .onnx using dynamic axes and batch-size in config can be set to any number \>=1, no errors anymore. Now, I am back to the original problem that is the performance drop. Using 3 RTSP stream 1920x1080@25fps, pgie and streammux with batch-size=3, I still get 16fps for each stream. I should be 24-25fps, and the value goes down whenever I increase the number of sources.
This is the config
[config\_infer\_primary\_yoloV5.txt](https://forums.developer.nvidia.com/uploads/short-url/t4zMkPAvd1DUlhKY8if5r4nJt3a.txt) (984 Bytes)
And this is the log
[run.log.tar.gz](https://forums.developer.nvidia.com/uploads/short-url/qmoXCoTaQJeVhBzwwzcyvvT5mFu.gz) (45.7 MB)
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 14, 2024, 12:45pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/9 "2024-04-14T12:45:19Z")
1. please refer to the code of deepstream-test3. please make sure the batch-size of nvstreammux and nvinfner is the same with the number\_sources.
2. if want high fps, you can use fakesink.
3. noticing using deeptream-test3, you can get high fps. can you modify deepstream-test3 step by step to customize? for example, test “nvinfer + fakesink” first, then test “nvinfer + nvtracker+ fakesink”, then test the application with other elements.
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 15, 2024, 2:11pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/10 "2024-04-15T14:11:21Z")
1. batch-size for nvstreammux and nvinfer are the same, set to 3, and I am using 3 RTSP stream sources.
2. The performance logger doesn’t show any difference between fakesink and nv3dsink, and this seems to me that the bottleneck is not the sync (by now).
3. Now, I created a new pipeline with the same elements from deepstream-test3 with and additional tracker element, and linked them exactly the same. However, I still get no more than 13.4fps/each for 3 sources 1920x1080@25fps. I have removed the osd buffer probe, just in case of any interference. This is the script now
[flavionew.py.txt](https://forums.developer.nvidia.com/uploads/short-url/cc8gAORFquSOKpTKrhkjoxr2sGr.txt) (22.0 KB)
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 16, 2024, 3:17pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/11 "2024-04-16T15:17:08Z")
there is too many custom codes in flavionew.py.txt. it is too hard to directly find the root cause for low fps issue. here is “nvinfer + fakesink” piepline base on deepstream\_test3. [deepstream\_test\_3.py](https://forums.developer.nvidia.com/uploads/short-url/BwI7LBf02FPVpcz4AwDzUVZTup.py) (16.7 KB), if it can run with a high fps, then you can continue to add other elements step by step.
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 19, 2024, 1:31pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/12 "2024-04-19T13:31:49Z")
I managed use `deepstream_test_3.py` as you advised, and finally got the same results from test 3 script. Both `flavionew.py` and `deepstream_test_3.py`. For both script I change the prediction model by changing the config.txt file. This are the benchmarks I get:
Using Resnet10 and config file [deepstream\_app\_config.txt](https://forums.developer.nvidia.com/uploads/short-url/gBT8fJHvJmXxFfXCptfy54fKaXn.txt) (1.1 KB):
`deepstream_test_3.py` \*\*PERF: {‘stream0’: 24.96, ‘stream1’: 24.96, ‘stream2’: 24.96}
`flavionew.py` \*\*PERF: {‘stream0’: 24.97, ‘stream1’: 24.97, ‘stream2’: 24.97}
Using Yolov5s and config file [config\_infer\_primary\_yoloV5.txt](https://forums.developer.nvidia.com/uploads/short-url/1GQz5pRhY4kwfJFoN26KCKv3E8u.txt) (1.1 KB):
`deepstream_test_3.py` \*\*PERF: {‘stream0’: 14.85, ‘stream1’: 14.85, ‘stream2’: 14.85}
`flavionew.py` \*\*PERF: {‘stream0’: 15.99, ‘stream1’: 15.99, ‘stream2’: 15.99}
I know Yolov5 is much more complex and has much more layers than Resnet10, and thus the former is slower than the latter. However, as far as I know, my hardware can easily support 24.xxxfps for Yolo ([https://github.com/NVIDIA-AI-IOT/jetson\_benchmarks#for-jetson-xavier-nx](https://github.com/NVIDIA-AI-IOT/jetson_benchmarks#for-jetson-xavier-nx)). So, I am back to the original question of this post to find out a solution for this performance drop. Where can I find Nvidia approach for that?
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 22, 2024, 2:43pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/13 "2024-04-22T14:43:10Z")
About yolov5s test, why do you need to set network-mode=0, fp32 accuracy has worse performance than fp16 or int8. please refer to this [configuration](https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps/blob/master/configs/nvinfer/yolov5_tao/pgie_yolov5_config.yml).
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 22, 2024, 8:51pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/14 "2024-04-22T20:51:43Z")
In fact, fp16 will increase frame rate, according to my experience, about 1.9x to 2.2x times. And int8 also increases it, but it will be necessary a calibration table, which I don’t have.
I use fps32 for benchmark purpose, since its a baseline for comparing models performance in many hardware, so I can’t change network-mode. Otherwise, I would be comparing different things.
What I see in jtop, is the model running in GPU with 68% of usage, thus with an opportunity to fill the processing pipeline with more instructions. I guess there is something that still need to done to improve the fps.
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 23, 2024, 6:02am UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/15 "2024-04-23T06:02:27Z")
> [@flavio.mello](#):
>
> n fact, fp16 will increase frame rate, according to my experience, about 1.9x to 2.2x times. And int8 also increases it
noticing you are using rtsp source with fps25. the max fps of pipeline should be close to 25. please test fp16 accuracy and please refer to this [link](https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/docs/INT8Calibration.md) for how to create the int8 calibration file.
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 23, 2024, 2:36pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/16 "2024-04-23T14:36:38Z")
I am sure fp16 will improve performance, but I can’t change network-mode to 2. This solution of changing it to fp16, ou even int8, just cloaks the problem, the GPU must perform at fp32.
Note that GPU is being used at 68%, so there is plenty idle processing that needs to be used. How can we identify the pipeline agressor?
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 24, 2024, 9:13am UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/17 "2024-04-24T09:13:08Z")
please refer to this [topic](https://forums.developer.nvidia.com/t/increase-the-fps/286062/23). if testing two streams with fp32 accuracy, please share the log of “trtexec --loadEngine=saved.engine --fp16”.
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 24, 2024, 3:14pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/18 "2024-04-24T15:14:32Z")
So, this is the log for `ubuntu@ubuntu:~/EdgeServer$ /usr/src/tensorrt/bin/trtexec --loadEngine=/home/ubuntu/EdgeServer/model_b4_gpu0_fp32.engine --fp16`:
[log.txt](https://forums.developer.nvidia.com/uploads/short-url/g9WefL66bQD3eQNt9x3zYA0THZv.txt) (9.0 KB)
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 25, 2024, 6:25am UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/19 "2024-04-25T06:25:23Z")
thanks for the sharing! from the log, the theoretical max fps of inference is 54, if using two streams, the theoretical max fps of each stream should be 25. you can use “src-\> pgie → fakesink” to verify. I have provide the code on Apr 16.
---
### Author:  [@flavio.mello](https://forums.developer.nvidia.com/u/flavio.mello)
#### Post date: [April 25, 2024, 3:23pm UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/20 "2024-04-25T15:23:53Z")
So, running the pipeline with src-\>streammux-\>queue-\>pgie-\>fakesink (`python3 deepstream_test_3.py --silent --no-display -i rtsp://admin:hbyt12345@10.21.45.19 rtsp://admin:hbyt12345@10.21.45.19 rtsp://admin:hbyt12345@10.21.45.19 rtsp://admin:hbyt12345@10.21.45.19`)
I get:
\*\*\*\*PERF: {‘stream0’: 15.19, ‘stream1’: 15.19, ‘stream2’: 15.19}
\*\*\*\*PERF: {‘stream0’: 15.6, ‘stream1’: 15.6, ‘stream2’: 15.6}
\*\*\*\*PERF: {‘stream0’: 15.6, ‘stream1’: 15.6, ‘stream2’: 15.6}
\*\*\*\*PERF: {‘stream0’: 15.39, ‘stream1’: 15.39, ‘stream2’: 15.39}
\*\*\*\*PERF: {‘stream0’: 15.59, ‘stream1’: 15.59, ‘stream2’: 15.59}
\*\*\*\*PERF: {‘stream0’: 15.59, ‘stream1’: 15.59, ‘stream2’: 15.59}
\*\*\*\*PERF: {‘stream0’: 15.39, ‘stream1’: 15.39, ‘stream2’: 15.39}
\*\*\*\*PERF: {‘stream0’: 15.6, ‘stream1’: 15.6, ‘stream2’: 15.6}
\*\*\*\*PERF: {‘stream0’: 15.58, ‘stream1’: 15.58, ‘stream2’: 15.58}
\*\*\*\*PERF: {‘stream0’: 15.4, ‘stream1’: 15.4, ‘stream2’: 15.4}
\*\*\*\*PERF: {‘stream0’: 15.58, ‘stream1’: 15.58, ‘stream2’: 15.58}
---
### Author:  [@fanzh](https://forums.developer.nvidia.com/u/fanzh)
#### Post date: [April 26, 2024, 7:46am UTC](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041/21 "2024-04-26T07:46:41Z")
noticing you are testing with four streams, from the logs, the total fps should be about 15x4=60(each stream fps x stream number). it is close to the theoretical max fps 54.
[Next page](https://forums.developer.nvidia.com/t/performance-drop-when-using-multiple-sources/289041.md?page=2)