Optimising a pipeline with a large amount of input sources

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
GPU RTX3090

• DeepStream Version
Deepstream 6.0

• JetPack Version (valid for Jetson only)
• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)
470.103.01

• Issue Type( questions, new requirements, bugs)
question

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi,

My use-case is to have:

  • YOLOv3 operate on 58 1080p RTSP camera streams.
  • The output to go to kafka
  • The streams to be as synchonised as possible to allow tracking between cameras

I currently have 58 test h265 files as follows:

mediainfo 192.168.250.100.h265
General
Complete name                            : 192.168.250.100.h265
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
File size                                : 5.40 MiB

Video
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main@L5@Main
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 15.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Color range                              : Full
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Looking at the specs of NVDEC it should be able to do 1500fps at 1080p for decoding, so I think I’m good for up to 100 cameras at 15fps from this perspective.

I have the following setup:

config_infer_primary_yoloV3.txt

################################################################################
# Copyright (c) 2019-2020, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

# Following properties are mandatory when engine files are not specified:
#   int8-calib-file(Only in INT8), model-file-format
#   Caffemodel mandatory properties: model-file, proto-file, output-blob-names
#   UFF: uff-file, input-dims, uff-input-blob-name, output-blob-names
#   ONNX: onnx-file
#
# Mandatory properties for detectors:
#   num-detected-classes
#
# Optional properties for detectors:
#   cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0)
#   custom-lib-path
#   parse-bbox-func-name
#
# Mandatory properties for classifiers:
#   classifier-threshold, is-classifier
#
# Optional properties for classifiers:
#   classifier-async-mode(Secondary mode only, Default=false)
#
# Optional properties in secondary mode:
#   operate-on-gie-id(Default=0), operate-on-class-ids(Defaults to all classes),
#   input-object-min-width, input-object-min-height, input-object-max-width,
#   input-object-max-height
#
# Following properties are always recommended:
#   batch-size(Default=1)
#
# Other optional properties:
#   net-scale-factor(Default=1), network-mode(Default=0 i.e FP32),
#   model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path,
#   mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary),
#   custom-lib-path, network-mode(Default=0 i.e FP32)
#
# The values in the config file are overridden by values set through GObject
# properties.

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
custom-network-config=/code/models/model/yolov3.cfg
model-file=/code/models/model/yolov3_best.weights
model-engine-file=model_b64_gpu0_fp16.engine
labelfile-path=/code/models/model/obj.names
int8-calib-file=yolov3-calibration.table.trt7.0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=1
gie-unique-id=1
network-type=0
is-classifier=0
## 0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)
cluster-mode=2
maintain-aspect-ratio=0
parse-bbox-func-name=NvDsInferParseCustomYoloV3
#custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
custom-lib-path=/code/sources/objectDetector_Yolo_ptl/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet
#scaling-filter=0
#scaling-compute-hw=0
batch-size=64

[class-attrs-all]
nms-iou-threshold=0.2
threshold=0.80

deepstream_app_config_yoloV3_kafka.txt

################################################################################
# Copyright (c) 2019-2020, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=8
columns=8
width=1800
height=1000
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.100.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.101.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source2]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.102.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source3]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.103.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source4]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.104.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source5]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.105.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source6]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.106.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source7]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.107.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source8]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.108.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source9]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.109.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source10]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.110.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source11]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.111.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source12]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.112.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source13]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.113.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source14]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.114.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source15]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.115.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source16]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.116.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source17]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.117.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source18]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.118.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source19]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.119.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source20]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.120.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source21]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.121.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source22]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.122.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source23]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.123.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source24]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.124.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source25]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.125.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source26]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.126.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source27]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.127.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source28]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.128.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source29]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.129.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source30]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.130.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source31]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.131.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source32]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.132.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source33]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.133.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source34]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.134.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source35]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.135.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source36]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.136.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source37]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.137.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source38]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.138.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source39]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.139.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source40]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.140.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source41]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.141.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source42]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.142.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source43]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.143.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source44]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.144.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source45]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.145.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source46]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.146.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source47]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.147.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source48]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.148.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source49]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.149.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source50]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.150.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source51]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.151.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source52]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.152.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source53]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.153.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source54]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.154.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source55]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.155.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source56]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.156.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0



[source57]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file:///code/example-videos/ten-second-video/192.168.250.157.h265
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0


[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=1
source-id=0
gpu-id=0
nvbuf-memory-type=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=UDPSink 5=nvoverlaysink 6=MsgConvBroker
type=6
msg-conv-config=dstest5_msgconv_sample_config.txt
#(0): PAYLOAD_DEEPSTREAM - Deepstream schema payload
#(1): PAYLOAD_DEEPSTREAM_MINIMAL - Deepstream schema payload minimal
#(256): PAYLOAD_RESERVED - Reserved type
#(257): PAYLOAD_CUSTOM   - Custom schema payload
msg-conv-payload-type=0
msg-broker-proto-lib=/opt/nvidia/deepstream/deepstream-5.1/lib/libnvds_kafka_proto.so
#Provide your msg-broker-conn-str here
msg-broker-conn-str=127.0.0.1;9092
topic=deepstream-coords
#Optional:
msg-broker-config=/code/sources/deepstream-test4/cfg_kafka.txt

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=16
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0
#sync-inputs=1

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
#model-engine-file=model_b1_gpu0_int8.engine
labelfile-path=../../models/model/obj.names
batch-size=16
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3.txt

[tracker]
enable=0
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.1/lib/libnvds_mot_klt.so

[tests]
file-loop=0

My problem is that I don’t seem to achieve 15fps when I run my example files, as shown by the perf output from deepstream. Note that htop shows that my CPU load is ok (medium load) during this:

**PERF:  14.25 (13.89)  14.25 (13.89)   14.25 (14.00)   14.25 (13.82)   14.25 (12.55)   14.25 (13.83)   14.25 (13.83)14.25 (13.89)   14.25 (12.94)   14.25 (13.89)   14.25 (11.75)   14.25 (9.17)    14.25 (13.35)   14.25 (12.71)   14.25 (8.94) 14.25 (13.89)   14.25 (13.03)   14.25 (13.88)   14.25 (10.22)   14.25 (6.54)    14.25 (13.83)   14.25 (10.54)14.25 (13.71)   14.25 (13.89)   14.25 (14.00)   14.25 (13.74)   14.25 (13.16)   14.25 (13.88)   14.25 (12.10)   14.25 (13.83)        14.25 (11.90)   14.25 (14.00)   14.25 (14.00)   14.25 (12.10)   14.25 (9.83)    14.25 (13.35)   14.25 (13.89)        14.25 (11.90)   14.25 (13.71)   14.25 (14.00)   14.25 (13.63)   14.25 (13.55)   14.25 (13.89)   14.25 (13.88)        14.25 (14.00)   14.25 (13.83)   14.25 (12.82)   14.25 (13.88)   14.25 (13.63)   14.25 (7.80)    14.25 (14.00)        14.25 (13.20)   14.25 (12.94)   14.25 (13.55)   14.25 (13.83)   14.25 (11.61)   14.25 (14.00)   14.25 (13.89)

Here is an example of the load on the system from nvidia-smi dmon during execution

# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0   110    30     -     7     2     0     4  9501  1695
    0   113    31     -     8     3     0    36  9501  1695
    0   152    34     -    27    13     0    85  9501  1875
    0   304    36     -    70    38     0    50  9501  1635
    0   303    36     -    70    36     0    48  9501  1695
    0   301    36     -    72    38     0    59  9501  1800
    0   304    38     -    72    38     0    49  9501  1635
    0   304    37     -    69    37     0    50  9501  1710
    0   306    38     -    75    38     0    54  9501  1785
    0   308    39     -    75    38     0    52  9501  1590
    0   310    39     -    75    37     0    46  9501  1605
    0   302    39     -    74    38     0    49  9501  1710
    0   291    39     -    67    36     0     7  9501  1710
    0   172    35     -    32     2     0     0  9501  1980
    0   135    35     -    10     0     0     0  9751  1980
    0   120    34     -     4     0     0     0  5001   825
    0    76    33     -    19     1     0     0  5001   585
    0    73    33     -     3     0     0     0  5001   420
    0    66    32     -     2     2     0     0   810   270

Is there anything obvious that I am doing wrong here?

Some unknowns I have despite trying to find answers:

  • How do I achieve 15fps on the system? As far as I can see I’m not saturating CPU, GPU or NVDEC but I’m not getting 15fps - where is the bottleneck?

  • What does the streammux do? I understand it batches frames, but for what reason? For the batch to then be run by the PGIE? But it has it’s own batch setting…

  • As above - what is the purpose of the batch-size in streammux and how do I choose an optimum number?

  • Why does streammux have a resolution?

  • what does it mean for the streammux live-source=1 - it says to set this for live sources, but what does it actually do?

  • How do I choose an optimum size for the PGIE batch? Is bigger better? Or are there tradeoffs? Does it need to match streammux? I don’t mind latency as long as all the cameras end up with the same latency.

  • What does the sync=1 setting do on [sink] blocks?

3

Please refer to Troubleshooting — DeepStream 6.1.1 Release documentation

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_FAQ.html#why-is-the-gst-nvstreammux-plugin-required-in-deepstream-4-0

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_FAQ.html#what-is-the-difference-between-batch-size-of-nvstreammux-and-nvinfer-what-are-the-recommended-values-for-nvstreammux-batch-size

You’d better to learn basic gstreamer knowledge before start with deepstream. And it is a necessary to read the whole user manual.

4

If you look at my post history you will see I have some experience over the last few years.

I will once again trawl the manuals in the hope that they tell you why choices are made. Here’s an example:

“ We recommend that the nvstreammux’s batch-size be set to either number of sources linked to it or the primary nvinfer’s batch-size.”

So basically “pick one of 2 values”. Well that’s not one value so some context here would be great to know so that I may pick the optimum value. I could try trial and error but surely knowing the theory is a better approach.

“ Multiple source components like decoder, camera, etc. are connected to the Gst-nvstreammux plugin to form a batch. This plugin is responsible for creating batch metadata, which is stored in the structure NvDsBatchMeta. This is the primary form of metadata in DeepStream 4.0.1. All plugins downstream from Gst-nvstreammux work on NvDsBatchMeta to access metadata and fill in the metadata they generate.”

So a metadata storage area? Then why the need for a resolution setting? It seems like it might transform all incoming frames to the same resolution (but not in the new stream mux maybe). I don’t think any amount of ( additional )hours of reading the docs will help me understand without more context here.

Maybe the gstreamer docs will explain the sync and live-source.

Is there an option to get other levels of support? It’s frustrating to spend £££ on NVIDIA hardware to then be told to RTM. A 2 min discussion with an engineer can save days of trawling and reading source code.

5

The concept of batch comes from TensorRT Quick Start Guide :: NVIDIA Deep Learning TensorRT Documentation

All the things done inside nvstreammux is to generate the batch(the batch information stored in NvDsBatchMeta which can be deliverred to downstream plugins with GstBuffer in gstreamer pipeline). So there are two ways to generate the batch. For multiple streams input, the nvstreammux batch size should be the same as the number of the streams, so the nvstreammux will try to pick up one frame from each stream to generate the batch(sometimes some frames will be missed, but nvstreammux will try the best with the timeout limitation). If there is only one stream, you can set nvstreammux batch size as the max batch size of the GIE(the model), so that nvstreammux will try to wait for enough frames from the stream to combine the batch, the latency will be longer.

For multiple stream case, the streams may have different resolutions, but the combine batch should be images with the same resolution. That is why there is width and height parameter for nvstreammux.

6

Thanks for the response.

So from this I think the original decoded frames are discarded once the batch is created and all subsequent inferences etc are done on the batch.

If this is correct, an additional question for the streammux - If I have two different resolutions, e.g. one stream at 4K and one at 1080p, I think it would make sense to set the streammux to 4K if I was using an SGIE so as not to lose resolution. E.g. the secondary would get higher resolution bounding boxes since they came from the 4K batch and not 1080p which would benefit the one stream that was originally 4K. Is this correct?

7

If the nvstreammux output resolution is different to the original video, the original frames are discarded.

The bbox resolution is decided by the detect model and the video. SGIE only takes images with fixed resolution. Even if your input image is very high resolution, it will be scaled to the resolution of the model input layer. Whether it will benefit depends on your model.

8

Thanks for the answer. In my case it’s better to scale down for the SGIE rather than scale up so I will stick with the 4K streammux.

I am now much clearer on the purpose of the streammux, thank you for taking the time to explain it.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.