Deepstream deployment of yolov3 with low FPS

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

My system configuration:ubuntu 18.04 + cuda10 + deepstream_sdk_4.0 + RTX2080Ti

I am using objectDetector_Yolo demo to deployment yolov3 model(80 categories were detected).After configuration according to redamen.Use "deepstream-app -c deepstream_app_config_yoloV3.txt " to run it.
But this results in very low FPS.

It’s results:

**PERF: 16.12 (16.12) 14.90 (14.90) 14.95 (14.95) 14.76 (14.76) 14.71 (14.71) 14.77 (14.77) 14.85 (14.85)
**PERF: 15.07 (15.57) 15.07 (14.99) 15.06 (15.01) 15.08 (14.94) 15.06 (14.90) 15.08 (14.94) 15.07 (14.97)
**PERF: 15.00 (15.37) 15.00 (14.99) 15.00 (15.01) 15.00 (14.96) 15.00 (14.93) 15.00 (14.96) 15.00 (14.98)
**PERF: 15.11 (15.30) 15.08 (15.02) 15.11 (15.03) 15.08 (14.99) 15.11 (14.98) 15.10 (15.00) 15.08 (15.01)
**PERF: 15.07 (15.26) 15.10 (15.03) 15.08 (15.04) 15.10 (15.01) 15.07 (15.00) 15.09 (15.01) 15.10 (15.03)
**PERF: 14.87 (15.19) 14.84 (15.00) 14.87 (15.01) 14.85 (14.99) 14.85 (14.97) 14.85 (14.99) 14.84 (14.99)

You can see from this that the FPS for each source is only 15.

Here is the configuration file i’am used(deepstream_app_config_yoloV3.txt and config_infer_primary_yoloV3.txt)：

deepstream_app_config_yoloV3.txt:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=10
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=2
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=rtsp://camera_url/h264/ch1/main/av_stream
num-sources=1
num-extra-surfaces=10
gpu-id=0
#drop-frame-interval=5

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=rtsp://camera_url/h264/ch1/main/av_stream
num-sources=6
num-extra-surfaces=10
gpu-id=0
#drop-frame-interval=5

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
qos=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=1
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=1
#batch-size=7
batch-size=2
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000

Set muxer output width and height

width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]
enable=1
gpu-id=0
#model-engine-file=model_b1_int8.engine
labelfile-path=labels.txt
#batch-size=7
batch-size=2
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
#interval=2
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3.txt

[tests]
#file-loop=0
file-loop=1

config_infer_primary_yoloV3.txt:

[property]
gpu-id=0
net-scale-factor=1
#0=RGB, 1=BGR
model-color-format=0
custom-network-config=yolov3.cfg
model-file=yolov3.weights
model-engine-file=model_b1_int8.engine
labelfile-path=labels.txt
int8-calib-file=yolov3-calibration.table.trt5.1

0=FP32, 1=INT8, 2=FP16 mode

network-mode=1
num-detected-classes=80
gie-unique-id=1
is-classifier=0
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV3
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so

I also referred to the Troubleshooting documentation, but it was hardly helpful.

Can you give me some useful advice?Thank you very much.

2

When I run it, the GPU usage information is:

optical@optical-X10DRi:~$ nvidia-smi -i 0 -q -d UTILIZATION

==============NVSMI LOG==============

Timestamp : Wed Mar 4 12:35:09 2020
Driver Version : 440.59
CUDA Version : 10.2

Attached GPUs : 1
GPU 00000000:81:00.0
Utilization
Gpu : 74 %
Memory : 35 %
Encoder : 0 %
Decoder : 2 %
GPU Utilization Samples
Duration : 98.32 sec
Number of Samples : 99
Max : 79 %
Min : 0 %
Avg : 25 %
Memory Utilization Samples
Duration : 98.32 sec
Number of Samples : 99
Max : 35 %
Min : 0 %
Avg : 6 %
ENC Utilization Samples
Duration : 98.32 sec
Number of Samples : 99
Max : 0 %
Min : 0 %
Avg : 0 %
DEC Utilization Samples
Duration : 98.32 sec
Number of Samples : 99
Max : 0 %
Min : 0 %
Avg : 0 %

3

My camera input is 25fps, 1080p in real time

4

You can set streammux batch size = 7

7 = soure0 1 stream + source1 6 streams

5

Thanks,and after i change streammux batch size =7 ,primary gie batch size = 7.Then run it,here is the output fps:

** INFO: <bus_callback:175>: Pipeline running

**PERF: 14.12 (14.12) 14.15 (14.15) 14.15 (14.15) 14.15 (14.15) 14.15 (14.15) 14.15 (14.15) 14.02 (14.02)
**PERF: 15.09 (14.63) 15.09 (14.64) 15.09 (14.64) 15.09 (14.64) 15.09 (14.64) 15.09 (14.64) 15.09 (14.58)
**PERF: 14.93 (14.73) 14.93 (14.74) 14.93 (14.74) 14.93 (14.74) 14.93 (14.74) 14.93 (14.74) 14.93 (14.70)
**PERF: 14.94 (14.78) 14.94 (14.79) 14.94 (14.79) 14.94 (14.79) 14.94 (14.79) 14.94 (14.79) 14.94 (14.76)
**PERF: 15.01 (14.83) 15.01 (14.84) 15.01 (14.84) 15.01 (14.84) 15.01 (14.84) 15.01 (14.84) 15.01 (14.81)
**PERF: 15.03 (14.86) 15.03 (14.87) 15.03 (14.87) 15.03 (14.87) 15.03 (14.87) 15.03 (14.87) 15.03 (14.85)
**PERF: 15.06 (14.89) 15.06 (14.90) 15.06 (14.90) 15.06 (14.90) 15.06 (14.90) 15.06 (14.90) 15.06 (14.88)
**PERF: 15.28 (14.94) 15.28 (14.95) 15.28 (14.95) 15.28 (14.95) 15.28 (14.95) 15.28 (14.95) 15.28 (14.93)
**PERF: 15.25 (14.98) 15.25 (14.98) 15.25 (14.98) 15.25 (14.98) 15.25 (14.98) 15.25 (14.98) 15.25 (14.96)
**PERF: 15.27 (15.01) 15.27 (15.01) 15.27 (15.01) 15.27 (15.01) 15.27 (15.01) 15.27 (15.01) 15.27 (15.00)

From the result ,The FPS didn’t get higher.

Here is the full project output:

optical@optical-X10DRi:~/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yoloptical@optical-X10DRi:~/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo$ deepstream-app -c deepstream_app_config_yoloV3.txt 
** WARN: <create_pipeline:930>: Num of Tiles less than number of sources, readjusting to 4 rows, 2 columns
Creating LL OSD context new
0:00:01.963940697 17974 0x55c5ff8418d0 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:00:03.286669271 17974 0x55c5ff8418d0 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:00:03.289021956 17974 0x55c5ff8418d0 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:checkEngineParams(): Requested Max Batch Size is less than engine batch size
0:00:03.293926080 17974 0x55c5ff8418d0 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 62001757
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-leaky     3 x 608 x 608      32 x 608 x 608    992   
(2)   conv-bn-leaky    32 x 608 x 608      64 x 304 x 304    19680 
(3)   conv-bn-leaky    64 x 304 x 304      32 x 304 x 304    21856 
(4)   conv-bn-leaky    32 x 304 x 304      64 x 304 x 304    40544 
(5)   skip             64 x 304 x 304      64 x 304 x 304        - 
(6)   conv-bn-leaky    64 x 304 x 304     128 x 152 x 152    114784
(7)   conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    123232
(8)   conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    197472
(9)   skip            128 x 152 x 152     128 x 152 x 152        - 
(10)  conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    205920
(11)  conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    280160
(12)  skip            128 x 152 x 152     128 x 152 x 152        - 
(13)  conv-bn-leaky   128 x 152 x 152     256 x  76 x  76    576096
(14)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    609376
(15)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    905312
(16)  skip            256 x  76 x  76     256 x  76 x  76        - 
(17)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    938592
(18)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1234528
(19)  skip            256 x  76 x  76     256 x  76 x  76        - 
(20)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1267808
(21)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1563744
(22)  skip            256 x  76 x  76     256 x  76 x  76        - 
(23)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1597024
(24)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1892960
(25)  skip            256 x  76 x  76     256 x  76 x  76        - 
(26)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1926240
(27)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2222176
(28)  skip            256 x  76 x  76     256 x  76 x  76        - 
(29)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2255456
(30)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2551392
(31)  skip            256 x  76 x  76     256 x  76 x  76        - 
(32)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2584672
(33)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2880608
(34)  skip            256 x  76 x  76     256 x  76 x  76        - 
(35)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2913888
(36)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    3209824
(37)  skip            256 x  76 x  76     256 x  76 x  76        - 
(38)  conv-bn-leaky   256 x  76 x  76     512 x  38 x  38    4391520
(39)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    4523616
(40)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    5705312
(41)  skip            512 x  38 x  38     512 x  38 x  38        - 
(42)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    5837408
(43)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    7019104
(44)  skip            512 x  38 x  38     512 x  38 x  38        - 
(45)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    7151200
(46)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    8332896
(47)  skip            512 x  38 x  38     512 x  38 x  38        - 
(48)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    8464992
(49)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    9646688
(50)  skip            512 x  38 x  38     512 x  38 x  38        - 
(51)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    9778784
(52)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    10960480
(53)  skip            512 x  38 x  38     512 x  38 x  38        - 
(54)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    11092576
(55)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    12274272
(56)  skip            512 x  38 x  38     512 x  38 x  38        - 
(57)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    12406368
(58)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    13588064
(59)  skip            512 x  38 x  38     512 x  38 x  38        - 
(60)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    13720160
(61)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    14901856
(62)  skip            512 x  38 x  38     512 x  38 x  38        - 
(63)  conv-bn-leaky   512 x  38 x  38    1024 x  19 x  19    19624544
(64)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    20150880
(65)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    24873568
(66)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(67)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    25399904
(68)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    30122592
(69)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(70)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    30648928
(71)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    35371616
(72)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(73)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    35897952
(74)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    40620640
(75)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(76)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    41146976
(77)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    45869664
(78)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    46396000
(79)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    51118688
(80)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    51645024
(81)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    56367712
(82)  conv-linear    1024 x  19 x  19     255 x  19 x  19    56629087
(83)  yolo            255 x  19 x  19     255 x  19 x  19    56629087
(84)  route                  -            512 x  19 x  19    56629087
(85)  conv-bn-leaky   512 x  19 x  19     256 x  19 x  19    56761183
(86)  upsample        256 x  19 x  19     256 x  38 x  38        - 
(87)  route                  -            768 x  38 x  38    56761183
(88)  conv-bn-leaky   768 x  38 x  38     256 x  38 x  38    56958815
(89)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    58140511
(90)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    58272607
(91)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    59454303
(92)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    59586399
(93)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    60768095
(94)  conv-linear     512 x  38 x  38     255 x  38 x  38    60898910
(95)  yolo            255 x  38 x  38     255 x  38 x  38    60898910
(96)  route                  -            256 x  38 x  38    60898910
(97)  conv-bn-leaky   256 x  38 x  38     128 x  38 x  38    60932190
(98)  upsample        128 x  38 x  38     128 x  76 x  76        - 
(99)  route                  -            384 x  76 x  76    60932190
(100) conv-bn-leaky   384 x  76 x  76     128 x  76 x  76    60981854
(101) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61277790
(102) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61311070
(103) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61607006
(104) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61640286
(105) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61936222
(106) conv-linear     256 x  76 x  76     255 x  76 x  76    62001757
(107) yolo            255 x  76 x  76     255 x  76 x  76    62001757
Output blob names :
yolo_83
yolo_95
yolo_107
Total number of layers: 257
Total number of layers on DLA: 0
Building the TensorRT Engine...
0:00:06.622258349 17974 0x55c5ff8418d0 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:01:01.201233855 17974 0x55c5ff8418d0 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
Building complete!
0:01:02.394238529 17974 0x55c5ff8418d0 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /home/optical/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo/model_b7_int8.engine
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:01:02.434166539 17974 0x55c5ff8418d0 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:01:02.437768607 17974 0x55c5ff8418d0 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF: FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	FPS 3 (Avg)	FPS 4 (Avg)	FPS 5 (Avg)	FPS 6 (Avg)	
**PERF: 0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

Creating LL OSD context new
** INFO: <bus_callback:175>: Pipeline running

**PERF: 14.12 (14.12)	14.15 (14.15)	14.15 (14.15)	14.15 (14.15)	14.15 (14.15)	14.15 (14.15)	14.02 (14.02)	
**PERF: 15.09 (14.63)	15.09 (14.64)	15.09 (14.64)	15.09 (14.64)	15.09 (14.64)	15.09 (14.64)	15.09 (14.58)	
**PERF: 14.93 (14.73)	14.93 (14.74)	14.93 (14.74)	14.93 (14.74)	14.93 (14.74)	14.93 (14.74)	14.93 (14.70)	
**PERF: 14.94 (14.78)	14.94 (14.79)	14.94 (14.79)	14.94 (14.79)	14.94 (14.79)	14.94 (14.79)	14.94 (14.76)	
**PERF: 15.01 (14.83)	15.01 (14.84)	15.01 (14.84)	15.01 (14.84)	15.01 (14.84)	15.01 (14.84)	15.01 (14.81)	
**PERF: 15.03 (14.86)	15.03 (14.87)	15.03 (14.87)	15.03 (14.87)	15.03 (14.87)	15.03 (14.87)	15.03 (14.85)	
**PERF: 15.06 (14.89)	15.06 (14.90)	15.06 (14.90)	15.06 (14.90)	15.06 (14.90)	15.06 (14.90)	15.06 (14.88)	
**PERF: 15.28 (14.94)	15.28 (14.95)	15.28 (14.95)	15.28 (14.95)	15.28 (14.95)	15.28 (14.95)	15.28 (14.93)	
**PERF: 15.25 (14.98)	15.25 (14.98)	15.25 (14.98)	15.25 (14.98)	15.25 (14.98)	15.25 (14.98)	15.25 (14.96)	
**PERF: 15.27 (15.01)	15.27 (15.01)	15.27 (15.01)	15.27 (15.01)	15.27 (15.01)	15.27 (15.01)	15.27 (15.00)	
q
Quitting

Any other Suggestions? Thank you!

6

Can you show this here:

$ nvidia-smi dmon

7

And there was another problem in my project,when i change [source 0]num-sources=1 and [source 1] num-sources=8,this project can’t run ,The output window flashes away,and then get error:

optical@optical-X10DRi:~/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo$ deepstream-app -c deepstream_app_config_yoloV3.txt 
** WARN: <create_pipeline:930>: Num of Tiles less than number of sources, readjusting to 5 rows, 2 columns
Creating LL OSD context new
0:00:01.986597833 18359 0x564d3cfd9860 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:00:03.329597964 18359 0x564d3cfd9860 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:00:03.332073830 18359 0x564d3cfd9860 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:checkEngineParams(): Requested Max Batch Size is less than engine batch size
0:00:03.336992778 18359 0x564d3cfd9860 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 62001757
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-leaky     3 x 608 x 608      32 x 608 x 608    992   
(2)   conv-bn-leaky    32 x 608 x 608      64 x 304 x 304    19680 
(3)   conv-bn-leaky    64 x 304 x 304      32 x 304 x 304    21856 
(4)   conv-bn-leaky    32 x 304 x 304      64 x 304 x 304    40544 
(5)   skip             64 x 304 x 304      64 x 304 x 304        - 
(6)   conv-bn-leaky    64 x 304 x 304     128 x 152 x 152    114784
(7)   conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    123232
(8)   conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    197472
(9)   skip            128 x 152 x 152     128 x 152 x 152        - 
(10)  conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    205920
(11)  conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    280160
(12)  skip            128 x 152 x 152     128 x 152 x 152        - 
(13)  conv-bn-leaky   128 x 152 x 152     256 x  76 x  76    576096
(14)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    609376
(15)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    905312
(16)  skip            256 x  76 x  76     256 x  76 x  76        - 
(17)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    938592
(18)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1234528
(19)  skip            256 x  76 x  76     256 x  76 x  76        - 
(20)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1267808
(21)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1563744
(22)  skip            256 x  76 x  76     256 x  76 x  76        - 
(23)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1597024
(24)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1892960
(25)  skip            256 x  76 x  76     256 x  76 x  76        - 
(26)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1926240
(27)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2222176
(28)  skip            256 x  76 x  76     256 x  76 x  76        - 
(29)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2255456
(30)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2551392
(31)  skip            256 x  76 x  76     256 x  76 x  76        - 
(32)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2584672
(33)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2880608
(34)  skip            256 x  76 x  76     256 x  76 x  76        - 
(35)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2913888
(36)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    3209824
(37)  skip            256 x  76 x  76     256 x  76 x  76        - 
(38)  conv-bn-leaky   256 x  76 x  76     512 x  38 x  38    4391520
(39)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    4523616
(40)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    5705312
(41)  skip            512 x  38 x  38     512 x  38 x  38        - 
(42)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    5837408
(43)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    7019104
(44)  skip            512 x  38 x  38     512 x  38 x  38        - 
(45)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    7151200
(46)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    8332896
(47)  skip            512 x  38 x  38     512 x  38 x  38        - 
(48)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    8464992
(49)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    9646688
(50)  skip            512 x  38 x  38     512 x  38 x  38        - 
(51)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    9778784
(52)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    10960480
(53)  skip            512 x  38 x  38     512 x  38 x  38        - 
(54)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    11092576
(55)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    12274272
(56)  skip            512 x  38 x  38     512 x  38 x  38        - 
(57)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    12406368
(58)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    13588064
(59)  skip            512 x  38 x  38     512 x  38 x  38        - 
(60)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    13720160
(61)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    14901856
(62)  skip            512 x  38 x  38     512 x  38 x  38        - 
(63)  conv-bn-leaky   512 x  38 x  38    1024 x  19 x  19    19624544
(64)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    20150880
(65)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    24873568
(66)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(67)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    25399904
(68)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    30122592
(69)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(70)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    30648928
(71)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    35371616
(72)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(73)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    35897952
(74)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    40620640
(75)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(76)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    41146976
(77)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    45869664
(78)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    46396000
(79)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    51118688
(80)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    51645024
(81)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    56367712
(82)  conv-linear    1024 x  19 x  19     255 x  19 x  19    56629087
(83)  yolo            255 x  19 x  19     255 x  19 x  19    56629087
(84)  route                  -            512 x  19 x  19    56629087
(85)  conv-bn-leaky   512 x  19 x  19     256 x  19 x  19    56761183
(86)  upsample        256 x  19 x  19     256 x  38 x  38        - 
(87)  route                  -            768 x  38 x  38    56761183
(88)  conv-bn-leaky   768 x  38 x  38     256 x  38 x  38    56958815
(89)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    58140511
(90)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    58272607
(91)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    59454303
(92)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    59586399
(93)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    60768095
(94)  conv-linear     512 x  38 x  38     255 x  38 x  38    60898910
(95)  yolo            255 x  38 x  38     255 x  38 x  38    60898910
(96)  route                  -            256 x  38 x  38    60898910
(97)  conv-bn-leaky   256 x  38 x  38     128 x  38 x  38    60932190
(98)  upsample        128 x  38 x  38     128 x  76 x  76        - 
(99)  route                  -            384 x  76 x  76    60932190
(100) conv-bn-leaky   384 x  76 x  76     128 x  76 x  76    60981854
(101) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61277790
(102) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61311070
(103) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61607006
(104) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61640286
(105) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61936222
(106) conv-linear     256 x  76 x  76     255 x  76 x  76    62001757
(107) yolo            255 x  76 x  76     255 x  76 x  76    62001757
Output blob names :
yolo_83
yolo_95
yolo_107
Total number of layers: 257
Total number of layers on DLA: 0
Building the TensorRT Engine...
0:00:06.723278720 18359 0x564d3cfd9860 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:01:00.928078798 18359 0x564d3cfd9860 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
Building complete!
0:01:02.121753741 18359 0x564d3cfd9860 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /home/optical/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo/model_b7_int8.engine
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:01:02.158866059 18359 0x564d3cfd9860 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:01:02.162443805 18359 0x564d3cfd9860 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF: FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	FPS 3 (Avg)	FPS 4 (Avg)	FPS 5 (Avg)	FPS 6 (Avg)	FPS 7 (Avg)	FPS 8 (Avg)	
**PERF: 0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

** INFO: <bus_callback:175>: Pipeline running

Creating LL OSD context new
ERROR from source: Could not write to resource.
Debug info: gstrtspsrc.c(7023): gst_rtspsrc_setup_streams_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin7/GstURIDecodeBin:src_elem/GstRTSPSrc:source:
Error (500): Internal Server Error
WARNING from source: Could not read from resource.
Debug info: gstrtspsrc.c(5280): gst_rtspsrc_loop_udp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin7/GstURIDecodeBin:src_elem/GstRTSPSrc:source:
The server closed the connection.
Quitting
ERROR from source: Could not write to resource.
Debug info: gstrtspsrc.c(7023): gst_rtspsrc_setup_streams_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin6/GstURIDecodeBin:src_elem/GstRTSPSrc:source:
Error (500): Internal Server Error
App run failed

This makes me confused that deepstream should not support so little in RTX2080Ti.

Thanks again for your reply,Looking forward to your more Suggestions.

8

Ok,when the deepstream run,here is “nvidia-smi dmon” result:

optical@optical-X10DRi:~$  nvidia-smi dmon
# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0   152    44     -    59    31     0    10  6800  1890
    0   144    45     -    60    33     0    10  6800  1890
    0   183    44     -    61    32     0    10  6800  1890
    0   145    44     -    58    31     0     9  6800  1890
    0   160    45     -    62    33     0    10  6800  1815
    0   174    44     -    59    31     0    10  6800  1890
    0   145    46     -    61    33     0    10  6800  1890
    0   188    44     -    61    31     0    10  6800  1890
    0   144    46     -    60    32     0    10  6800  1845
    0   159    45     -    61    32     0     9  6800  1875
    0   182    45     -    62    31     0    10  6800  1890
    0   145    46     -    60    32     0     9  6800  1815
    0   186    45     -    61    32     0    11  6800  1890
    0   150    45     -    60    31     0    10  6800  1890
    0   147    46     -    61    33     0    10  6800  1815
    0   150    46     -    60    31     0     9  6800  1875
    0   186    45     -    61    32     0    11  6800  1890
    0   147    47     -    60    31     0     9  6800  1890
    0   163    45     -    62    33     0    10  6800  1875
    0   162    46     -    59    31     0    10  6800  1890
    0   148    46     -    63    33     0     9  6800  1815
    0   186    45     -    62    32     0    10  6800  1890
    0   145    47     -    59    32     0    10  6800  1815
    0   169    46     -    61    32     0     9  6800  1890
    0   160    47     -    59    31     0    10  6800  1890
    0   145    47     -    59    32     0     9  6800  1815
    0   154    46     -    60    32     0    10  6800  1860
    0   188    46     -    62    31     0    10  6800  1890
    0   145    47     -    60    32     0    10  6800  1815
    0   172    46     -    61    33     0     9  6800  1890
    0   162    46     -    59    31     0    10  6800  1890
    0   152    47     -    58    31     0    10  6800  1860
    0   160    46     -    60    34     0    10  6800  1890
    0   153    47     -    60    31     0    10  6800  1815
    0   147    46     -    60    33     0    10  6800  1890
    0   189    48     -    61    32     0    10  6800  1890
    0   145    46     -    58    33     0    11  6800  1890

After it stops running,the result is:

^Coptical@optical-X10DRi:~nvidia-smi dmon
# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0    17    40     -     9     2     0     0   405   315
    0    21    40     -    12     3     0     0   405   300
    0    17    40     -     8     2     0     0   405   315
    0    16    40     -    18     3     0     0   405   300
    0    15    40     -     6     2     0     0   405   315
    0    16    40     -    11     3     0     0   405   300
    0    16    40     -     5     2     0     0   405   300
    0    16    39     -     4     2     0     0   405   300
    0    28    39     -     4     2     0     0   405   315
    0    18    39     -    15     3     0     0   405   300
    0    15    39     -     6     2     0     0   405   300
    0    23    39     -     7     2     0     0   405   300
    0    16    39     -    15     3     0     0   405   300
    0    15    39     -     7     2     0     0   405   300
    0    21    39     -     4     2     0     0   405   300
    0    18    39     -     6     2     0     0   405   300
    0    20    38     -    10     3     0     0   405   300
    0    15    38     -    14     3     0     0   405   300
    0    19    38     -     8     2     0     0   405   300
    0    16    38     -     7     2     0     0   405   300

thanks!

9

Check camera rtsp video fps 25 by vlc ?

10

Ok,it seems that i can’t send image,i use vlc to check camera rstp information.

I used two cameras,one is [source 0] num-sources=1 :

Information about what your media or stream is made of.
Muxer,Audio and Video Codes,Subtitles are shown.
stream0:
codec : H264-MPEG-4 AVC (part10)(h264)
type : Video
video resolution: 19201080
buffer dimensions: 19201088
decode formate:
orientation: Top left
color primaries : ITU-R BT709
color transfer function : ITU-R BT.709
color space: ITU-R BT.709 Range
chroma location: Left

another is [source 1] num-sources=6:

Information about what your media or stream is made of.
Muxer,Audio and Video Codes,Subtitles are shown.
stream0:
codec : H264-MPEG-4 AVC (part10)(h264)
type : Video
video resolution: 19201080
buffer dimensions: 19201088
frame rate : 25
decode formate:
orientation: Top left
chroma location: Left

well.[source 0 ] is not show frame rate…

thanks for you help!

11

Now，i set [source 0 ] enable = 0,and [source 1] num-sources =6,[streammux] batch-size = 6,[primary gie] batch size = 6.and run it again,output is :

optical@optical-X10DRi:~/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo$ deepstream-app -c deepstream_app_config_yoloV3.txt 
** WARN: <create_pipeline:930>: Num of Tiles less than number of sources, readjusting to 3 rows, 2 columns
Creating LL OSD context new
0:00:02.151233781 21428 0x55ed4299d750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:00:03.547420247 21428 0x55ed4299d750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:00:03.550125662 21428 0x55ed4299d750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:checkEngineParams(): Requested Max Batch Size is less than engine batch size
0:00:03.555532221 21428 0x55ed4299d750 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 62001757
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-leaky     3 x 608 x 608      32 x 608 x 608    992   
(2)   conv-bn-leaky    32 x 608 x 608      64 x 304 x 304    19680 
(3)   conv-bn-leaky    64 x 304 x 304      32 x 304 x 304    21856 
(4)   conv-bn-leaky    32 x 304 x 304      64 x 304 x 304    40544 
(5)   skip             64 x 304 x 304      64 x 304 x 304        - 
(6)   conv-bn-leaky    64 x 304 x 304     128 x 152 x 152    114784
(7)   conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    123232
(8)   conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    197472
(9)   skip            128 x 152 x 152     128 x 152 x 152        - 
(10)  conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    205920
(11)  conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    280160
(12)  skip            128 x 152 x 152     128 x 152 x 152        - 
(13)  conv-bn-leaky   128 x 152 x 152     256 x  76 x  76    576096
(14)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    609376
(15)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    905312
(16)  skip            256 x  76 x  76     256 x  76 x  76        - 
(17)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    938592
(18)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1234528
(19)  skip            256 x  76 x  76     256 x  76 x  76        - 
(20)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1267808
(21)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1563744
(22)  skip            256 x  76 x  76     256 x  76 x  76        - 
(23)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1597024
(24)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1892960
(25)  skip            256 x  76 x  76     256 x  76 x  76        - 
(26)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1926240
(27)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2222176
(28)  skip            256 x  76 x  76     256 x  76 x  76        - 
(29)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2255456
(30)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2551392
(31)  skip            256 x  76 x  76     256 x  76 x  76        - 
(32)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2584672
(33)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2880608
(34)  skip            256 x  76 x  76     256 x  76 x  76        - 
(35)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2913888
(36)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    3209824
(37)  skip            256 x  76 x  76     256 x  76 x  76        - 
(38)  conv-bn-leaky   256 x  76 x  76     512 x  38 x  38    4391520
(39)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    4523616
(40)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    5705312
(41)  skip            512 x  38 x  38     512 x  38 x  38        - 
(42)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    5837408
(43)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    7019104
(44)  skip            512 x  38 x  38     512 x  38 x  38        - 
(45)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    7151200
(46)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    8332896
(47)  skip            512 x  38 x  38     512 x  38 x  38        - 
(48)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    8464992
(49)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    9646688
(50)  skip            512 x  38 x  38     512 x  38 x  38        - 
(51)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    9778784
(52)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    10960480
(53)  skip            512 x  38 x  38     512 x  38 x  38        - 
(54)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    11092576
(55)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    12274272
(56)  skip            512 x  38 x  38     512 x  38 x  38        - 
(57)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    12406368
(58)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    13588064
(59)  skip            512 x  38 x  38     512 x  38 x  38        - 
(60)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    13720160
(61)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    14901856
(62)  skip            512 x  38 x  38     512 x  38 x  38        - 
(63)  conv-bn-leaky   512 x  38 x  38    1024 x  19 x  19    19624544
(64)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    20150880
(65)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    24873568
(66)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(67)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    25399904
(68)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    30122592
(69)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(70)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    30648928
(71)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    35371616
(72)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(73)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    35897952
(74)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    40620640
(75)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(76)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    41146976
(77)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    45869664
(78)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    46396000
(79)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    51118688
(80)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    51645024
(81)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    56367712
(82)  conv-linear    1024 x  19 x  19     255 x  19 x  19    56629087
(83)  yolo            255 x  19 x  19     255 x  19 x  19    56629087
(84)  route                  -            512 x  19 x  19    56629087
(85)  conv-bn-leaky   512 x  19 x  19     256 x  19 x  19    56761183
(86)  upsample        256 x  19 x  19     256 x  38 x  38        - 
(87)  route                  -            768 x  38 x  38    56761183
(88)  conv-bn-leaky   768 x  38 x  38     256 x  38 x  38    56958815
(89)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    58140511
(90)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    58272607
(91)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    59454303
(92)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    59586399
(93)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    60768095
(94)  conv-linear     512 x  38 x  38     255 x  38 x  38    60898910
(95)  yolo            255 x  38 x  38     255 x  38 x  38    60898910
(96)  route                  -            256 x  38 x  38    60898910
(97)  conv-bn-leaky   256 x  38 x  38     128 x  38 x  38    60932190
(98)  upsample        128 x  38 x  38     128 x  76 x  76        - 
(99)  route                  -            384 x  76 x  76    60932190
(100) conv-bn-leaky   384 x  76 x  76     128 x  76 x  76    60981854
(101) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61277790
(102) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61311070
(103) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61607006
(104) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61640286
(105) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61936222
(106) conv-linear     256 x  76 x  76     255 x  76 x  76    62001757
(107) yolo            255 x  76 x  76     255 x  76 x  76    62001757
Output blob names :
yolo_83
yolo_95
yolo_107
Total number of layers: 257
Total number of layers on DLA: 0
Building the TensorRT Engine...
0:00:07.046306559 21428 0x55ed4299d750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:01:00.843710405 21428 0x55ed4299d750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
Building complete!
0:01:02.041823271 21428 0x55ed4299d750 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /home/optical/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo/model_b6_int8.engine
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:01:02.080539904 21428 0x55ed4299d750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:01:02.084280677 21428 0x55ed4299d750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF: FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	FPS 3 (Avg)	FPS 4 (Avg)	FPS 5 (Avg)	
**PERF: 0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

Creating LL OSD context new
** INFO: <bus_callback:175>: Pipeline running

**PERF: 14.77 (14.77)	14.66 (14.66)	14.77 (14.77)	14.59 (14.59)	14.77 (14.77)	14.65 (14.65)	
**PERF: 14.83 (14.80)	14.83 (14.75)	14.83 (14.80)	14.83 (14.71)	14.83 (14.80)	14.83 (14.75)	
**PERF: 14.94 (14.85)	14.94 (14.81)	14.94 (14.85)	14.94 (14.79)	14.94 (14.85)	14.94 (14.81)	
**PERF: 16.86 (15.37)	16.86 (15.34)	16.86 (15.37)	16.86 (15.32)	16.86 (15.37)	16.86 (15.34)	
q
Quitting
App run successful

But the FPS are still very low.
What should I do? Thanks.

12

And when i replace the source from the camera with a file:

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://…/…/samples/streams/sample_1080p_h264.mp4
num-sources=6
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

and the output is :

optical@optical-X10DRi:~/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo$ deepstream-app -c deepstream_app_config_yoloV3.txt 
** WARN: <create_pipeline:930>: Num of Tiles less than number of sources, readjusting to 3 rows, 2 columns
Creating LL OSD context new
0:00:02.067178818 21904 0x55db9b533750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:00:03.418969141 21904 0x55db9b533750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:00:03.421423474 21904 0x55db9b533750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:checkEngineParams(): Requested Max Batch Size is less than engine batch size
0:00:03.426319435 21904 0x55db9b533750 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 62001757
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-leaky     3 x 608 x 608      32 x 608 x 608    992   
(2)   conv-bn-leaky    32 x 608 x 608      64 x 304 x 304    19680 
(3)   conv-bn-leaky    64 x 304 x 304      32 x 304 x 304    21856 
(4)   conv-bn-leaky    32 x 304 x 304      64 x 304 x 304    40544 
(5)   skip             64 x 304 x 304      64 x 304 x 304        - 
(6)   conv-bn-leaky    64 x 304 x 304     128 x 152 x 152    114784
(7)   conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    123232
(8)   conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    197472
(9)   skip            128 x 152 x 152     128 x 152 x 152        - 
(10)  conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    205920
(11)  conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    280160
(12)  skip            128 x 152 x 152     128 x 152 x 152        - 
(13)  conv-bn-leaky   128 x 152 x 152     256 x  76 x  76    576096
(14)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    609376
(15)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    905312
(16)  skip            256 x  76 x  76     256 x  76 x  76        - 
(17)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    938592
(18)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1234528
(19)  skip            256 x  76 x  76     256 x  76 x  76        - 
(20)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1267808
(21)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1563744
(22)  skip            256 x  76 x  76     256 x  76 x  76        - 
(23)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1597024
(24)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1892960
(25)  skip            256 x  76 x  76     256 x  76 x  76        - 
(26)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1926240
(27)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2222176
(28)  skip            256 x  76 x  76     256 x  76 x  76        - 
(29)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2255456
(30)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2551392
(31)  skip            256 x  76 x  76     256 x  76 x  76        - 
(32)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2584672
(33)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2880608
(34)  skip            256 x  76 x  76     256 x  76 x  76        - 
(35)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2913888
(36)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    3209824
(37)  skip            256 x  76 x  76     256 x  76 x  76        - 
(38)  conv-bn-leaky   256 x  76 x  76     512 x  38 x  38    4391520
(39)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    4523616
(40)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    5705312
(41)  skip            512 x  38 x  38     512 x  38 x  38        - 
(42)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    5837408
(43)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    7019104
(44)  skip            512 x  38 x  38     512 x  38 x  38        - 
(45)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    7151200
(46)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    8332896
(47)  skip            512 x  38 x  38     512 x  38 x  38        - 
(48)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    8464992
(49)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    9646688
(50)  skip            512 x  38 x  38     512 x  38 x  38        - 
(51)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    9778784
(52)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    10960480
(53)  skip            512 x  38 x  38     512 x  38 x  38        - 
(54)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    11092576
(55)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    12274272
(56)  skip            512 x  38 x  38     512 x  38 x  38        - 
(57)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    12406368
(58)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    13588064
(59)  skip            512 x  38 x  38     512 x  38 x  38        - 
(60)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    13720160
(61)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    14901856
(62)  skip            512 x  38 x  38     512 x  38 x  38        - 
(63)  conv-bn-leaky   512 x  38 x  38    1024 x  19 x  19    19624544
(64)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    20150880
(65)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    24873568
(66)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(67)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    25399904
(68)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    30122592
(69)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(70)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    30648928
(71)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    35371616
(72)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(73)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    35897952
(74)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    40620640
(75)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(76)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    41146976
(77)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    45869664
(78)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    46396000
(79)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    51118688
(80)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    51645024
(81)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    56367712
(82)  conv-linear    1024 x  19 x  19     255 x  19 x  19    56629087
(83)  yolo            255 x  19 x  19     255 x  19 x  19    56629087
(84)  route                  -            512 x  19 x  19    56629087
(85)  conv-bn-leaky   512 x  19 x  19     256 x  19 x  19    56761183
(86)  upsample        256 x  19 x  19     256 x  38 x  38        - 
(87)  route                  -            768 x  38 x  38    56761183
(88)  conv-bn-leaky   768 x  38 x  38     256 x  38 x  38    56958815
(89)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    58140511
(90)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    58272607
(91)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    59454303
(92)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    59586399
(93)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    60768095
(94)  conv-linear     512 x  38 x  38     255 x  38 x  38    60898910
(95)  yolo            255 x  38 x  38     255 x  38 x  38    60898910
(96)  route                  -            256 x  38 x  38    60898910
(97)  conv-bn-leaky   256 x  38 x  38     128 x  38 x  38    60932190
(98)  upsample        128 x  38 x  38     128 x  76 x  76        - 
(99)  route                  -            384 x  76 x  76    60932190
(100) conv-bn-leaky   384 x  76 x  76     128 x  76 x  76    60981854
(101) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61277790
(102) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61311070
(103) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61607006
(104) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61640286
(105) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61936222
(106) conv-linear     256 x  76 x  76     255 x  76 x  76    62001757
(107) yolo            255 x  76 x  76     255 x  76 x  76    62001757
Output blob names :
yolo_83
yolo_95
yolo_107
Total number of layers: 257
Total number of layers on DLA: 0
Building the TensorRT Engine...
0:00:06.851228099 21904 0x55db9b533750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:01:00.483574775 21904 0x55db9b533750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
Building complete!
0:01:01.691369812 21904 0x55db9b533750 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /home/optical/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo/model_b6_int8.engine
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:01:01.726832885 21904 0x55db9b533750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:01:01.730554182 21904 0x55db9b533750 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF: FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	FPS 3 (Avg)	FPS 4 (Avg)	FPS 5 (Avg)	
**PERF: 0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
** INFO: <bus_callback:189>: Pipeline ready

Creating LL OSD context new
** INFO: <bus_callback:175>: Pipeline running

**PERF: 17.64 (17.64)	17.64 (17.64)	17.70 (17.70)	17.64 (17.64)	17.64 (17.64)	17.64 (17.64)	
**PERF: 17.81 (17.73)	17.81 (17.73)	17.81 (17.76)	17.81 (17.73)	17.81 (17.73)	17.81 (17.73)	
**PERF: 17.81 (17.76)	17.81 (17.76)	17.81 (17.77)	17.81 (17.75)	17.81 (17.76)	17.81 (17.76)	
**PERF: 17.79 (17.76)	17.79 (17.76)	17.79 (17.78)	17.79 (17.76)	17.79 (17.76)	17.79 (17.76)	
q
Quitting
App run successful

when use mp4 file as input ,fps is still low.

13

Does your platform support int8?
Can you try to change yolov3.cfg width/height to be 416/416 ?

14

Thanks,after changed yoloV3.cfg width/height to 416/416,and retraining a new model (2 class),and run it ,the results showed that the FPS was lower,and how to determine whether the platform support int8?
I used a gpu of RTX2080ti .

optical@optical-X10DRi:~/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo$ deepstream-app -c deepstream_app_config_yoloV3.txt 

(deepstream-app:36960): GLib-GObject-WARNING **: 12:07:10.943: value "TRUE" of type 'gboolean' is invalid or out of range for property 'sync' of type 'gboolean'

(deepstream-app:36960): GLib-GObject-WARNING **: 12:07:10.943: value "TRUE" of type 'gboolean' is invalid or out of range for property 'qos' of type 'gboolean'
** WARN: <create_pipeline:930>: Num of Tiles less than number of sources, readjusting to 4 rows, 2 columns
Creating LL OSD context new
0:00:01.961856120 36960 0x55d332dfc690 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:00:03.362386107 36960 0x55d332dfc690 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:00:03.365010459 36960 0x55d332dfc690 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:checkEngineParams(): Requested Max Batch Size is less than engine batch size
0:00:03.370729913 36960 0x55d332dfc690 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 61581727
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-leaky     3 x 416 x 416      32 x 416 x 416    992   
(2)   conv-bn-leaky    32 x 416 x 416      64 x 208 x 208    19680 
(3)   conv-bn-leaky    64 x 208 x 208      32 x 208 x 208    21856 
(4)   conv-bn-leaky    32 x 208 x 208      64 x 208 x 208    40544 
(5)   skip             64 x 208 x 208      64 x 208 x 208        - 
(6)   conv-bn-leaky    64 x 208 x 208     128 x 104 x 104    114784
(7)   conv-bn-leaky   128 x 104 x 104      64 x 104 x 104    123232
(8)   conv-bn-leaky    64 x 104 x 104     128 x 104 x 104    197472
(9)   skip            128 x 104 x 104     128 x 104 x 104        - 
(10)  conv-bn-leaky   128 x 104 x 104      64 x 104 x 104    205920
(11)  conv-bn-leaky    64 x 104 x 104     128 x 104 x 104    280160
(12)  skip            128 x 104 x 104     128 x 104 x 104        - 
(13)  conv-bn-leaky   128 x 104 x 104     256 x  52 x  52    576096
(14)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    609376
(15)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    905312
(16)  skip            256 x  52 x  52     256 x  52 x  52        - 
(17)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    938592
(18)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    1234528
(19)  skip            256 x  52 x  52     256 x  52 x  52        - 
(20)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    1267808
(21)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    1563744
(22)  skip            256 x  52 x  52     256 x  52 x  52        - 
(23)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    1597024
(24)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    1892960
(25)  skip            256 x  52 x  52     256 x  52 x  52        - 
(26)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    1926240
(27)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    2222176
(28)  skip            256 x  52 x  52     256 x  52 x  52        - 
(29)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    2255456
(30)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    2551392
(31)  skip            256 x  52 x  52     256 x  52 x  52        - 
(32)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    2584672
(33)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    2880608
(34)  skip            256 x  52 x  52     256 x  52 x  52        - 
(35)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    2913888
(36)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    3209824
(37)  skip            256 x  52 x  52     256 x  52 x  52        - 
(38)  conv-bn-leaky   256 x  52 x  52     512 x  26 x  26    4391520
(39)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    4523616
(40)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    5705312
(41)  skip            512 x  26 x  26     512 x  26 x  26        - 
(42)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    5837408
(43)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    7019104
(44)  skip            512 x  26 x  26     512 x  26 x  26        - 
(45)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    7151200
(46)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    8332896
(47)  skip            512 x  26 x  26     512 x  26 x  26        - 
(48)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    8464992
(49)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    9646688
(50)  skip            512 x  26 x  26     512 x  26 x  26        - 
(51)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    9778784
(52)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    10960480
(53)  skip            512 x  26 x  26     512 x  26 x  26        - 
(54)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    11092576
(55)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    12274272
(56)  skip            512 x  26 x  26     512 x  26 x  26        - 
(57)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    12406368
(58)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    13588064
(59)  skip            512 x  26 x  26     512 x  26 x  26        - 
(60)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    13720160
(61)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    14901856
(62)  skip            512 x  26 x  26     512 x  26 x  26        - 
(63)  conv-bn-leaky   512 x  26 x  26    1024 x  13 x  13    19624544
(64)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    20150880
(65)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    24873568
(66)  skip           1024 x  13 x  13    1024 x  13 x  13        - 
(67)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    25399904
(68)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    30122592
(69)  skip           1024 x  13 x  13    1024 x  13 x  13        - 
(70)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    30648928
(71)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    35371616
(72)  skip           1024 x  13 x  13    1024 x  13 x  13        - 
(73)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    35897952
(74)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    40620640
(75)  skip           1024 x  13 x  13    1024 x  13 x  13        - 
(76)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    41146976
(77)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    45869664
(78)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    46396000
(79)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    51118688
(80)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    51645024
(81)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    56367712
(82)  conv-linear    1024 x  13 x  13      21 x  13 x  13    56389237
(83)  yolo             21 x  13 x  13      21 x  13 x  13    56389237
(84)  route                  -            512 x  13 x  13    56389237
(85)  conv-bn-leaky   512 x  13 x  13     256 x  13 x  13    56521333
(86)  upsample        256 x  13 x  13     256 x  26 x  26        - 
(87)  route                  -            768 x  26 x  26    56521333
(88)  conv-bn-leaky   768 x  26 x  26     256 x  26 x  26    56718965
(89)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    57900661
(90)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    58032757
(91)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    59214453
(92)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    59346549
(93)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    60528245
(94)  conv-linear     512 x  26 x  26      21 x  26 x  26    60539018
(95)  yolo             21 x  26 x  26      21 x  26 x  26    60539018
(96)  route                  -            256 x  26 x  26    60539018
(97)  conv-bn-leaky   256 x  26 x  26     128 x  26 x  26    60572298
(98)  upsample        128 x  26 x  26     128 x  52 x  52        - 
(99)  route                  -            384 x  52 x  52    60572298
(100) conv-bn-leaky   384 x  52 x  52     128 x  52 x  52    60621962
(101) conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    60917898
(102) conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    60951178
(103) conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    61247114
(104) conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    61280394
(105) conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    61576330
(106) conv-linear     256 x  52 x  52      21 x  52 x  52    61581727
(107) yolo             21 x  52 x  52      21 x  52 x  52    61581727
Output blob names :
yolo_83
yolo_95
yolo_107
Total number of layers: 257
Total number of layers on DLA: 0
Building the TensorRT Engine...
0:00:06.714983984 36960 0x55d332dfc690 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:00:57.884232697 36960 0x55d332dfc690 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
Building complete!
0:00:59.018334555 36960 0x55d332dfc690 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /home/optical/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo/model_b7_int8.engine
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:00:59.054707483 36960 0x55d332dfc690 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:00:59.058681256 36960 0x55d332dfc690 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF: FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	FPS 3 (Avg)	FPS 4 (Avg)	FPS 5 (Avg)	FPS 6 (Avg)	
**PERF: 0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

** INFO: <bus_callback:175>: Pipeline running

Creating LL OSD context new
** INFO: <bus_callback:175>: Pipeline running

**PERF: 5.15 (5.15)	5.06 (5.06)	5.06 (5.06)	5.06 (5.06)	5.06 (5.06)	5.06 (5.06)	5.06 (5.06)	
**PERF: 5.01 (5.07)	5.00 (5.03)	5.00 (5.03)	5.00 (5.03)	5.00 (5.03)	5.00 (5.03)	5.00 (5.03)	
**PERF: 4.99 (5.04)	5.00 (5.02)	5.00 (5.02)	5.00 (5.02)	5.00 (5.02)	4.98 (5.01)	4.98 (5.01)	
**PERF: 5.00 (5.03)	5.02 (5.02)	5.02 (5.02)	5.02 (5.02)	5.02 (5.02)	5.01 (5.01)	5.01 (5.01)	
**PERF: 5.02 (5.03)	4.99 (5.01)	4.99 (5.01)	4.99 (5.01)	4.99 (5.01)	5.03 (5.01)	5.03 (5.01)	
**PERF: 5.03 (5.03)	5.00 (5.01)	5.00 (5.01)	5.00 (5.01)	5.00 (5.01)	5.00 (5.01)	5.00 (5.01)	
**PERF: 4.97 (5.02)	5.02 (5.01)	5.02 (5.01)	5.02 (5.01)	5.02 (5.01)	4.98 (5.01)	4.98 (5.01)	
**PERF: 5.00 (5.02)	4.99 (5.01)	4.99 (5.01)	4.99 (5.01)	4.99 (5.01)	5.02 (5.01)	5.02 (5.01)	
**PERF: 5.00 (5.02)	5.02 (5.01)	5.02 (5.01)	5.02 (5.01)	5.02 (5.01)	4.99 (5.01)	4.99 (5.01)	
**PERF: 5.02 (5.02)	5.02 (5.01)	5.02 (5.01)	5.02 (5.01)	5.02 (5.01)	5.00 (5.01)	5.00 (5.01)	
q
Quitting
App run successful
15

I also changed [streammux] batchsize =7 and [primary] batch-size =7.

16

How about only one source?

17

Now there’s only one data source, But before we only used one source, FPS was 24.I haven’t done anything to my system during this time.

optical@optical-X10DRi:~/deepstream_sdk_v4.0.1_x86_64/sources/objectDetector_Yolo$ deepstream-app -c deepstream_app_config_yoloV3.txt 

(deepstream-app:39274): GLib-GObject-WARNING **: 13:01:40.273: value "TRUE" of type 'gboolean' is invalid or out of range for property 'sync' of type 'gboolean'

(deepstream-app:39274): GLib-GObject-WARNING **: 13:01:40.273: value "TRUE" of type 'gboolean' is invalid or out of range for property 'qos' of type 'gboolean'
Creating LL OSD context new
0:00:01.937944137 39274 0x55ca5ca67960 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:00:03.260202884 39274 0x55ca5ca67960 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
0:00:03.268780886 39274 0x55ca5ca67960 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was compiled against cuBLAS 10.2.0 but is linked against cuBLAS 10.1.0
cb_sourcesetup set 100 latency

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.

** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

** INFO: <bus_callback:175>: Pipeline running

Creating LL OSD context new

**PERF: FPS 0 (Avg)	
**PERF: 5.50 (5.50)	
**PERF: 4.97 (5.06)	
**PERF: 5.00 (5.03)	
**PERF: 5.00 (5.02)	
**PERF: 5.02 (5.02)	
**PERF: 4.99 (5.02)	
**PERF: 5.01 (5.02)	
**PERF: 5.01 (5.02)	
q
Quitting
App run successful
18

I restart my system ,and the fps is only 5…

19

Can you remove all engine files, and test one single stream firstly, double check config files and try to get 30fps on single source and then gradually increase the number of streams to see what’s the max limit.

Tracker also can increase the number of streams further.