Yolo V3 slow

I am having performance issue running Yolo V3 with Deepstream. When running my pipeline on a Tesla T4 I have about 1000 FPS with the model /opt/nvidia/deepstream/deepstream-6.0/samples/models/Primary_Detector/resnet10.caffemodel while when using YoloV3 I have only about 30 FPS. I know that YoloV3 is a bigger model and that I can’t expect 1000FPS but still 30 FPS seems too little for a T4.

Looking at nvidia-smi dmons:

# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0    70    75     -   100    27     0     0  5000   780
    0    69    75     -   100    37     0     0  5000   825
    0    70    75     -   100    42     0     0  5000   810
    0    70    75     -   100    39     0     1  5000   795
    0    69    75     -   100    42     0     0  5000   675
    0    71    75     -   100    45     0     2  5000   780
    0    67    76     -   100    54     0     0  5000   795
    0    65    76     -   100    44     0     3  5000   780
    0    67    76     -   100    44     0     0  5000   855
    0    67    76     -   100    32     0     9  5000   900
    0    70    76     -   100    37     0     0  5000   840
    0    67    76     -   100    18     0    26  5000   870
    0    71    76     -   100    34     0     0  5000   855

Is sm the cuda cores utilization?

Here’s how you can reproduce the issue. I am using the official deepstream docker container with python bindings installed.

Here’s how I set up YoloV3 into the container:

mkdir -p /src/models/yoloV3
cd /src/models/yoloV3
# Download data
wget https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg
wget https://pjreddie.com/media/files/yolov3.weights
# Copy calibration from sample
cp /opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo/yolov3-calibration.table.trt7.0 /src/models/yoloV3
# Build plugin
cd /opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo/
wget https://forums.developer.nvidia.com/uploads/short-url/oezjVVUIuYdfJ8BdTJWNmIBVawl.patch -O DS6.0GA_objectDetector_Yolo_perf_regression.patch
patch -p1 < DS6.0GA_objectDetector_Yolo_perf_regression.patch
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
mkdir /src/models/yoloV3/nvdsinfer_custom_impl_Yolo
cp /opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so /src/models/yoloV3/nvdsinfer_custom_impl_Yolo

My Yolo V3 configuration is the following (copied from the deepstream 6 container) :

################################################################################
# Copyright (c) 2019-2021, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

# Following properties are mandatory when engine files are not specified:
#   int8-calib-file(Only in INT8), model-file-format
#   Caffemodel mandatory properties: model-file, proto-file, output-blob-names
#   UFF: uff-file, input-dims, uff-input-blob-name, output-blob-names
#   ONNX: onnx-file
#
# Mandatory properties for detectors:
#   num-detected-classes
#
# Optional properties for detectors:
#   cluster-mode(Default=Group Rectangles), interval(Primary mode only, Default=0)
#   custom-lib-path
#   parse-bbox-func-name
#
# Mandatory properties for classifiers:
#   classifier-threshold, is-classifier
#
# Optional properties for classifiers:
#   classifier-async-mode(Secondary mode only, Default=false)
#
# Optional properties in secondary mode:
#   operate-on-gie-id(Default=0), operate-on-class-ids(Defaults to all classes),
#   input-object-min-width, input-object-min-height, input-object-max-width,
#   input-object-max-height
#
# Following properties are always recommended:
#   batch-size(Default=1)
#
# Other optional properties:
#   net-scale-factor(Default=1), network-mode(Default=0 i.e FP32),
#   model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path,
#   mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary),
#   custom-lib-path, network-mode(Default=0 i.e FP32)
#
# The values in the config file are overridden by values set through GObject
# properties.

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
custom-network-config=/src/models/yoloV3/yolov3.cfg
model-file=/src/models/yoloV3/yolov3.weights
#model-engine-file=yolov3_b1_gpu0_int8.engine
labelfile-path=/src/src_deepstream/components/models/yolov3/labels.txt
int8-calib-file=/src/models/yoloV3/yolov3-calibration.table.trt7.0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=1
num-detected-classes=80
gie-unique-id=1
network-type=0
is-classifier=0
## 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)
cluster-mode=2
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV3
custom-lib-path=/src/models/yoloV3/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet
#scaling-filter=0
#scaling-compute-hw=0

[class-attrs-all]
nms-iou-threshold=0.3
threshold=0.7

Here’s my pipeline. The code is inspired by the rtsp_in_rtsp_out example from the official python example. Function bus_call and class GETFPS can be found in the common folder of the Python bindings. For your convenience I copied their code below.
The pipeline is basically a video decoding + streammux + primary gpu inference engine. At the beginning of the code you can find some simple class to count the FPS. Again, most of the code is taken from the official example rtsp_in_rtsp_out. As you can see,
batch size is set to the number of sources.

import os
import sys
sys.path.append("../")
import pyds
from comoon.bus_call import bus_call
import math
import gi
gi.require_version("Gst", "1.0")
gi.require_version("GstRtspServer", "1.0")
from gi.repository import GObject, Gst, GstRtspServer

import argparse
import configparser

from common.FPS import GETFPS


from collections import defaultdict
import time
import multiprocessing
import threading


# Counter to count FPS
class Counter:
    lock = threading.Lock()
    count = defaultdict(int)
    started = multiprocessing.Value("d", time.time())

    @classmethod
    def print_fps_and_reset(cls):
        with cls.lock:
            # Compute fps
            tot_fps = round(sum(cls.count.values()) / (time.time() - cls.started.value), 2)
            avg_fps = round(tot_fps / 54) if cls.count else 0
            streams_up = len(cls.count)
            # Reset
            cls.count = defaultdict(int)
            cls.started.value = time.time()
            print(f"TOT FPS {tot_fps}   AVG FPS {avg_fps}   STREAMS UP {streams_up}")


# Start monitoring process to print FPS
def monitor(Counter):
    while True:
        Counter.print_fps_and_reset()
        time.sleep(5)
monitoring_thread = threading.Thread(target=monitor, daemon=True, args=(Counter,))
monitoring_thread.start()


# Manager for the Deepstream pipeline
class PipelineManager:

    def __init__(self):
        # Load cameras configuration
        url = "your rtsp camera"
        number_of_streams = 54
        self.fps_streams = dict()
        for index in range(number_of_streams):
            self.fps_streams["stream{0}".format(index)] = GETFPS(url)
        number_sources = len(self.fps_streams)
        print(f"Number of streams {number_sources}")

        # Standard GStreamer initialization
        GObject.threads_init()
        Gst.init(None)

        # Create Pipeline element that will form a connection of other elements
        print("Creating Pipeline \n")
        pipeline = Gst.Pipeline()
        if not pipeline:
            sys.stderr.write(" Unable to create Pipeline \n")

        # Create nvstreammux instance to form batches from one or more sources.
        print("Creating nvstreammux \n")
        streammux = Gst.ElementFactory.make("nvstreammux", "Stream-muxer")
        if not streammux:
            sys.stderr.write(" Unable to create NvStreamMux \n")
        pipeline.add(streammux)

        for index in range(number_of_streams):
            # sourcebin (source pad) -> (sinkpad) streammux ->
            # Source bin
            print("Creating source_bin ", index, " \n ")
            source_bin = self.create_source_bin(index, url)
            if not source_bin:
                sys.stderr.write("Unable to create source bin \n")
            pipeline.add(source_bin)
            # Sink pad
            padname = "sink_%u" % index
            sinkpad = streammux.get_request_pad(padname)
            if not sinkpad:
                sys.stderr.write("Unable to create sink pad bin \n")
            # Source pad
            srcpad = source_bin.get_static_pad("src")
            if not srcpad:
                sys.stderr.write("Unable to create src pad bin \n")
            srcpad.link(sinkpad)

        # Primary GPU Inference Engine (pgie)
        print("Creating Pgie \n ")
        pgie = Gst.ElementFactory.make("nvinfer", "primary-inference")
        if not pgie:
            sys.stderr.write(" Unable to create pgie \n")

        # Fakesink
        sink = Gst.ElementFactory.make("fakesink", "fakesink")

        batch_size = number_sources
        streammux.set_property("width", 1920)
        streammux.set_property("height", 1080)
        streammux.set_property("batch-size", batch_size)
        streammux.set_property("batched-push-timeout", 4000000)
        streammux.set_property("live-source", 1)  #  if is_live else 0)

        pgie.set_property("config-file-path", "src_deepstream/components/models/yolov3/config_infer_primary_yoloV3.txt")
        pgie.set_property("batch-size", batch_size)

        pipeline.add(pgie)
        pipeline.add(sink)

        streammux.link(pgie)
        pgie.link(sink)

        # create an event loop and feed gstreamer bus messages to it
        loop = GObject.MainLoop()
        bus = pipeline.get_bus()
        bus.add_signal_watch()
        bus.connect("message", bus_call, loop)

        # Probe to get metadata
        tiler_src_pad = pgie.get_static_pad("src")
        if not tiler_src_pad:
            sys.stderr.write(" Unable to get src pad \n")
        else:
            tiler_src_pad.add_probe(Gst.PadProbeType.BUFFER, self.tiler_src_pad_buffer_probe, 0)

        # Start play back and listen to events
        print("Starting pipeline \n")
        pipeline.set_state(Gst.State.PLAYING)

        try:
            loop.run()
        except BaseException:
            pass
        except Exception as e:
            print("Exception", str(e))

        # cleanup
        pipeline.set_state(Gst.State.NULL)

    def tiler_src_pad_buffer_probe(self, pad, info, u_data):
        """
        tiler_sink_pad_buffer_probe  will extract metadata received on OSD sink pad
        and update params for drawing rectangle, object information etc.
        """
        PGIE_CLASS_ID_VEHICLE = 0
        PGIE_CLASS_ID_BICYCLE = 1
        PGIE_CLASS_ID_PERSON = 2
        PGIE_CLASS_ID_ROADSIGN = 3

        frame_number = 0
        num_rects = 0
        gst_buffer = info.get_buffer()
        if not gst_buffer:
            print("Unable to get GstBuffer ")
            return

        # Retrieve batch metadata from the gst_buffer
        # Note that pyds.gst_buffer_get_nvds_batch_meta() expects the
        # C address of gst_buffer as input, which is obtained with hash(gst_buffer)
        batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
        l_frame = batch_meta.frame_meta_list
        while l_frame is not None:

            try:
                # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
                # The casting is done by pyds.NvDsFrameMeta.cast()
                # The casting also keeps ownership of the underlying memory
                # in the C code, so the Python garbage collector will leave
                # it alone.
                frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
            except StopIteration:
                break

            with Counter.lock:
                Counter.count[frame_meta.pad_index] += 1

            frame_number = frame_meta.frame_num
            l_obj = frame_meta.obj_meta_list
            num_rects = frame_meta.num_obj_meta
            obj_counter = {
                PGIE_CLASS_ID_VEHICLE: 0,
                PGIE_CLASS_ID_PERSON: 0,
                PGIE_CLASS_ID_BICYCLE: 0,
                PGIE_CLASS_ID_ROADSIGN: 0,
            }

            while l_obj is not None:
                try:
                    # Casting l_obj.data to pyds.NvDsObjectMeta
                    obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
                except StopIteration:
                    break
                obj_counter[obj_meta.class_id] += 1
                try:
                    l_obj = l_obj.next
                except StopIteration:
                    break

            # print(
            #     "Stream Number=",
            #     frame_meta.pad_index,
            #     "Frame Number=",
            #     frame_number,
            #     "Number of Objects=",
            #     num_rects,
            #     "Vehicle_count=",
            #     obj_counter[self.PGIE_CLASS_ID_VEHICLE],
            #     "Person_count=",
            #     obj_counter[self.PGIE_CLASS_ID_PERSON],
            # )

            # Get frame rate through this probe
            self.fps_streams["stream{0}".format(frame_meta.pad_index)].get_fps()
            try:
                l_frame = l_frame.next
            except StopIteration:
                break

        return Gst.PadProbeReturn.OK

    def cb_newpad(self, decodebin, decoder_src_pad, data):
        print("In cb_newpad\n")
        caps = decoder_src_pad.get_current_caps()
        gststruct = caps.get_structure(0)
        gstname = gststruct.get_name()
        source_bin = data
        features = caps.get_features(0)

        # Need to check if the pad created by the decodebin is for video and not
        # audio.
        print("gstname=", gstname)
        if gstname.find("video") != -1:
            # Link the decodebin pad only if decodebin has picked nvidia
            # decoder plugin nvdec_*. We do this by checking if the pad caps contain
            # NVMM memory features.
            print("features=", features)
            if features.contains("memory:NVMM"):
                # Get the source bin ghost pad
                bin_ghost_pad = source_bin.get_static_pad("src")
                if not bin_ghost_pad.set_target(decoder_src_pad):
                    sys.stderr.write(
                        "Failed to link decoder src pad to source bin ghost pad\n"
                    )
            else:
                sys.stderr.write(
                    " Error: Decodebin did not pick nvidia decoder plugin.\n")

    def decodebin_child_added(self, child_proxy, Object, name, user_data):
        print("Decodebin child added:", name, "\n")
        if name.find("decodebin") != -1:
            Object.connect("child-added", self.decodebin_child_added, user_data)

    def create_source_bin(self, id, uri):
        print("Creating source bin")

        # Create a source GstBin to abstract this bin's content from the rest of the
        # pipeline
        bin_name = "source-bin-%02d" % id
        print(bin_name)
        nbin = Gst.Bin.new(bin_name)
        if not nbin:
            sys.stderr.write(" Unable to create source bin \n")

        # Source element for reading from the uri.
        # We will use decodebin and let it figure out the container format of the
        # stream and the codec and plug the appropriate demux and decode plugins.
        uri_decode_bin = Gst.ElementFactory.make("uridecodebin", "uri-decode-bin")
        if not uri_decode_bin:
            sys.stderr.write(" Unable to create uri decode bin \n")
        # We set the input uri to the source element
        uri_decode_bin.set_property("uri", uri)
        # Connect to the "pad-added" signal of the decodebin which generates a
        # callback once a new pad for raw data has been created by the decodebin
        uri_decode_bin.connect("pad-added", self.cb_newpad, nbin)
        uri_decode_bin.connect("child-added", self.decodebin_child_added, nbin)

        # We need to create a ghost pad for the source bin which will act as a proxy
        # for the video decoder src pad. The ghost pad will not have a target right
        # now. Once the decode bin creates the video decoder and generates the
        # cb_newpad callback, we will set the ghost pad target to the video decoder
        # src pad.
        Gst.Bin.add(nbin, uri_decode_bin)
        bin_pad = nbin.add_pad(
            Gst.GhostPad.new_no_target(
                "src", Gst.PadDirection.SRC))
        if not bin_pad:
            sys.stderr.write(" Failed to add ghost pad in source bin \n")
            return None
        return nbin

bus_call.py (copied from deepstream examples)

################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2019-2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

import gi
import sys

gi.require_version('Gst', '1.0')

from gi.repository import GObject, Gst

from utils.debug import get_attributes, get_methods


def bus_call(bus, message, loop):
    t = message.type
    if t == Gst.MessageType.EOS:
        sys.stdout.write("End-of-stream\n")
        loop.quit()
    elif t == Gst.MessageType.WARNING:
        err, debug = message.parse_warning()
        sys.stderr.write("Warning: %s: %s\n" % (err, debug))
    elif t == Gst.MessageType.ERROR:
        err, debug = message.parse_error()
        sys.stderr.write("Error: %s: %s\n" % (err, debug))
        # print(type(err))
        # get_attributes(message.src.get_name())
        # get_methods(err)

        # Camera disconnected
        if str(err) == "gst-resource-error-quark: Could not read from resource. (9)":
            # TODO: when a camera is disconnected do something
            # Do not stop application
            return True

        loop.quit()
    return True

FPS.py (copied from deepstream examples)

################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2019-2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

import time
start_time=time.time()
frame_count=0

class GETFPS:
    def __init__(self,stream_id):
        global start_time
        self.start_time=start_time
        self.is_first=True
        global frame_count
        self.frame_count=frame_count
        self.stream_id=stream_id
    def get_fps(self):
        end_time=time.time()
        if(self.is_first):
            self.start_time=end_time
            self.is_first=False
        if(end_time-self.start_time>5):
            # print("**********************FPS*****************************************")
            # print("Fps of stream",self.stream_id,"is ", float(self.frame_count)/5.0)
            self.frame_count=0
            self.start_time=end_time
        else:
            self.frame_count=self.frame_count+1
    def print_data(self):
        print('frame_count=',self.frame_count)
        print('start_time=',self.start_time)

Note: using Yolov3-tiny or another model would not solve the issue since I believe the problem is not related to Yolo but rather to so wrong usage/implementation either on my side or in Deepstream.

You can reproduce the performance issue using the Python example deepstream_rtsp_in_rtsp_out.

I have two question:

  • Do you see anything wrong in the configuration? Any idea why I could have this bad performances? The weird part is that the GPU utilization is really high. Am I computing the FPS in a wrong way?
  • I don’t think this is related, but Yolo have an height and width input size of 608x608. Should I set these values for streammux too? How are the height and width of streammux related to the one of the model? Should they be equal?

Even though my primary focus remains Yolo, I’d like to point out that also using SSD via triton server as provided in the example from rtsp_in_rtsp_out produces only 50FPS despite GPU utilization being extremely low. I tried to increase the max_batch_size for the SSD in the settings to the value of 128 but I had no improvement in the FPS. Again, I don’t care that much about SSD, I’d rather use Yolo, but I wanted to point out this issue too. I’d like to know if you have the same issues with that example. Both Yolo and SSD are seems to be problematic. I have a Tesla T4, 16 cores and 64Gb of ram. I doubt the issue is the hardware here.

    0    49    77     -    89    64     0     1  5000  1440
    0    48    77     -    89    63     0     0  5000  1515
    0    45    77     -    90    58     0     7  5000  1560
    0    45    77     -    92    61     0     7  5000  1560
    0    46    77     -    88    59     0    13  5000  1560
    0    46    77     -    90    59     0    10  5000  1560
    0    46    77     -    89    58     0     9  5000  1560
    0    46    77     -    90    62     0     3  5000  1560
    0    47    77     -    91    63     0     3  5000  1560
    0    49    77     -    81    62     0     0  5000  1530

I created a gist so you can easily reproduce the issue on your machine.

You just need to follow the readme.txt instructions and a have an rtsp stream (you can try with a local file too). It should be really easy.

Did you get similar perf result for rtsp stream and local file?

Yes, same issues with both local files and rtsp stream.
You can try to run my git gist with url = "file:///opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_qHD.h264"

Is this patch already included in DS 6.0 Deepstream 6 YOLO performance issue - #22 by mchi ?

Not yet. you need to patch it into your code.

Ok. For comparison here’s Yolo V3 (git gist above).

Without patch (scroll to the bottom to see the FPS moving with a regular pattern between 21 and 32 FPS:

docker run   -it   --rm   --net=host   --runtime nvidia   -e DISPLAY=$DISPLAY   -v /tmp/.X11-unix/:/tmp/.X11-unix   --cap-add=SYS_PTRACE   --security-opt seccomp=unconfined   --device /dev/video0   --privileged   --expose 8554   deepstream-custom-slow
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
Number of streams 54
pipeline.py:75: PyGIDeprecationWarning: Since version 3.11, calling threads_init is no longer needed. See: https://wiki.gnome.org/PyGObject/Threading
  GObject.threads_init()

(gst-plugin-scanner:8): GStreamer-WARNING **: 03:49:19.911: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:8): GLib-GObject-WARNING **: 03:49:20.221: specified class size for type 'GstCompositor' is smaller than the parent type's 'GstVideoAggregator' class size

(gst-plugin-scanner:8): GLib-GObject-CRITICAL **: 03:49:20.221: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed

(gst-plugin-scanner:8): GLib-CRITICAL **: 03:49:20.221: g_once_init_leave: assertion 'result != 0' failed

(gst-plugin-scanner:8): GStreamer-CRITICAL **: 03:49:20.221: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
Creating Pipeline 

Creating nvstreammux 

Creating source_bin  0  
 
Creating source bin
source-bin-00
Creating source_bin  1  
 
Creating source bin
source-bin-01
Creating source_bin  2  
 
Creating source bin
source-bin-02
Creating source_bin  3  
 
Creating source bin
source-bin-03
Creating source_bin  4  
 
Creating source bin
source-bin-04
Creating source_bin  5  
 
Creating source bin
source-bin-05
Creating source_bin  6  
 
Creating source bin
source-bin-06
Creating source_bin  7  
 
Creating source bin
source-bin-07
Creating source_bin  8  
 
Creating source bin
source-bin-08
Creating source_bin  9  
 
Creating source bin
source-bin-09
Creating source_bin  10  
 
Creating source bin
source-bin-10
Creating source_bin  11  
 
Creating source bin
source-bin-11
Creating source_bin  12  
 
Creating source bin
source-bin-12
Creating source_bin  13  
 
Creating source bin
source-bin-13
Creating source_bin  14  
 
Creating source bin
source-bin-14
Creating source_bin  15  
 
Creating source bin
source-bin-15
Creating source_bin  16  
 
Creating source bin
source-bin-16
Creating source_bin  17  
 
Creating source bin
source-bin-17
Creating source_bin  18  
 
Creating source bin
source-bin-18
Creating source_bin  19  
 
Creating source bin
source-bin-19
Creating source_bin  20  
 
Creating source bin
source-bin-20
Creating source_bin  21  
 
Creating source bin
source-bin-21
Creating source_bin  22  
 
Creating source bin
source-bin-22
Creating source_bin  23  
 
Creating source bin
source-bin-23
Creating source_bin  24  
 
Creating source bin
source-bin-24
Creating source_bin  25  
 
Creating source bin
source-bin-25
Creating source_bin  26  
 
Creating source bin
source-bin-26
Creating source_bin  27  
 
Creating source bin
source-bin-27
Creating source_bin  28  
 
Creating source bin
source-bin-28
Creating source_bin  29  
 
Creating source bin
source-bin-29
Creating source_bin  30  
 
Creating source bin
source-bin-30
Creating source_bin  31  
 
Creating source bin
source-bin-31
Creating source_bin  32  
 
Creating source bin
source-bin-32
Creating source_bin  33  
 
Creating source bin
source-bin-33
Creating source_bin  34  
 
Creating source bin
source-bin-34
Creating source_bin  35  
 
Creating source bin
source-bin-35
Creating source_bin  36  
 
Creating source bin
source-bin-36
Creating source_bin  37  
 
Creating source bin
source-bin-37
Creating source_bin  38  
 
Creating source bin
source-bin-38
Creating source_bin  39  
 
Creating source bin
source-bin-39
Creating source_bin  40  
 
Creating source bin
source-bin-40
Creating source_bin  41  
 
Creating source bin
source-bin-41
Creating source_bin  42  
 
Creating source bin
source-bin-42
Creating source_bin  43  
 
Creating source bin
source-bin-43
Creating source_bin  44  
 
Creating source bin
source-bin-44
Creating source_bin  45  
 
Creating source bin
source-bin-45
Creating source_bin  46  
 
Creating source bin
source-bin-46
Creating source_bin  47  
 
Creating source bin
source-bin-47
Creating source_bin  48  
 
Creating source bin
source-bin-48
Creating source_bin  49  
 
Creating source bin
source-bin-49
Creating source_bin  50  
 
Creating source bin
source-bin-50
Creating source_bin  51  
 
Creating source bin
source-bin-51
Creating source_bin  52  
 
Creating source bin
source-bin-52
Creating source_bin  53  
 
Creating source bin
source-bin-53
Creating Pgie 
 
Unknown or legacy key specified 'is-classifier' for group [property]
Warn: 'threshold' parameter has been deprecated. Use 'pre-cluster-threshold' instead.
pipeline.py:136: PyGIDeprecationWarning: GObject.MainLoop is deprecated; use GLib.MainLoop instead
  loop = GObject.MainLoop()
Starting pipeline 

0:00:00.733563622     1      0x34dd430 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
Loading pre-trained weights...
Loading weights of yolov3 complete!
Total Number of weights read : 62001757
Loading pre-trained weights...
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
Loading weights of yolov3 complete!
Total Number of weights read : 62001757
Building Yolo network...
      layer               inp_size            out_size       weightPtr
(0)   conv-bn-leaky     3 x 608 x 608      32 x 608 x 608    992   
(1)   conv-bn-leaky    32 x 608 x 608      64 x 304 x 304    19680 
(2)   conv-bn-leaky    64 x 304 x 304      32 x 304 x 304    21856 
(3)   conv-bn-leaky    32 x 304 x 304      64 x 304 x 304    40544 
(4)   skip             64 x 304 x 304      64 x 304 x 304        - 
(5)   conv-bn-leaky    64 x 304 x 304     128 x 152 x 152    114784
(6)   conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    123232
(7)   conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    197472
(8)   skip            128 x 152 x 152     128 x 152 x 152        - 
(9)   conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    205920
(10)  conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    280160
(11)  skip            128 x 152 x 152     128 x 152 x 152        - 
(12)  conv-bn-leaky   128 x 152 x 152     256 x  76 x  76    576096
(13)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    609376
(14)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    905312
(15)  skip            256 x  76 x  76     256 x  76 x  76        - 
(16)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    938592
(17)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1234528
(18)  skip            256 x  76 x  76     256 x  76 x  76        - 
(19)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1267808
(20)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1563744
(21)  skip            256 x  76 x  76     256 x  76 x  76        - 
(22)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1597024
(23)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1892960
(24)  skip            256 x  76 x  76     256 x  76 x  76        - 
(25)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1926240
(26)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2222176
(27)  skip            256 x  76 x  76     256 x  76 x  76        - 
(28)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2255456
(29)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2551392
(30)  skip            256 x  76 x  76     256 x  76 x  76        - 
(31)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2584672
(32)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2880608
(33)  skip            256 x  76 x  76     256 x  76 x  76        - 
(34)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2913888
(35)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    3209824
(36)  skip            256 x  76 x  76     256 x  76 x  76        - 
(37)  conv-bn-leaky   256 x  76 x  76     512 x  38 x  38    4391520
(38)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    4523616
(39)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    5705312
(40)  skip            512 x  38 x  38     512 x  38 x  38        - 
(41)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    5837408
(42)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    7019104
(43)  skip            512 x  38 x  38     512 x  38 x  38        - 
(44)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    7151200
(45)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    8332896
(46)  skip            512 x  38 x  38     512 x  38 x  38        - 
(47)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    8464992
(48)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    9646688
(49)  skip            512 x  38 x  38     512 x  38 x  38        - 
(50)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    9778784
(51)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    10960480
(52)  skip            512 x  38 x  38     512 x  38 x  38        - 
(53)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    11092576
(54)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    12274272
(55)  skip            512 x  38 x  38     512 x  38 x  38        - 
(56)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    12406368
(57)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    13588064
(58)  skip            512 x  38 x  38     512 x  38 x  38        - 
(59)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    13720160
(60)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    14901856
(61)  skip            512 x  38 x  38     512 x  38 x  38        - 
(62)  conv-bn-leaky   512 x  38 x  38    1024 x  19 x  19    19624544
(63)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    20150880
(64)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    24873568
(65)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(66)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    25399904
(67)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    30122592
(68)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(69)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    30648928
(70)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    35371616
(71)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(72)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    35897952
(73)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    40620640
(74)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(75)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    41146976
(76)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    45869664
(77)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    46396000
(78)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    51118688
(79)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    51645024
(80)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    56367712
(81)  conv-linear    1024 x  19 x  19     255 x  19 x  19    56629087
(82)  yolo            255 x  19 x  19     255 x  19 x  19    56629087
(83)  route                  -            512 x  19 x  19    56629087
(84)  conv-bn-leaky   512 x  19 x  19     256 x  19 x  19    56761183
(85)  upsample        256 x  19 x  19     256 x  38 x  38        - 
(86)  route                  -            768 x  38 x  38    56761183
(87)  conv-bn-leaky   768 x  38 x  38     256 x  38 x  38    56958815
(88)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    58140511
(89)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    58272607
(90)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    59454303
(91)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    59586399
(92)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    60768095
(93)  conv-linear     512 x  38 x  38     255 x  38 x  38    60898910
(94)  yolo            255 x  38 x  38     255 x  38 x  38    60898910
(95)  route                  -            256 x  38 x  38    60898910
(96)  conv-bn-leaky   256 x  38 x  38     128 x  38 x  38    60932190
(97)  upsample        128 x  38 x  38     128 x  76 x  76        - 
(98)  route                  -            384 x  76 x  76    60932190
(99)  conv-bn-leaky   384 x  76 x  76     128 x  76 x  76    60981854
(100) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61277790
(101) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61311070
(102) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61607006
(103) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61640286
(104) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61936222
(105) conv-linear     256 x  76 x  76     255 x  76 x  76    62001757
(106) yolo            255 x  76 x  76     255 x  76 x  76    62001757
Output yolo blob names :
yolo_83
yolo_95
yolo_107
Total number of yolo layers: 257
Building yolo network complete!
Building the TensorRT Engine...
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
WARNING: [TRT]: Detected invalid timing cache, setup a local cache instead
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
WARNING: [TRT]: Try increasing the workspace size to 4194304 bytes to get better performance.
Building complete!
0:01:33.367041024     1      0x34dd430 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1947> [UID = 1]: serialize cuda engine to file: /src/model_b54_gpu0_int8.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 4
0   INPUT  kFLOAT data            3x608x608       
1   OUTPUT kFLOAT yolo_83         255x19x19       
2   OUTPUT kFLOAT yolo_95         255x38x38       
3   OUTPUT kFLOAT yolo_107        255x76x76       

0:01:33.796121765     1      0x34dd430 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:/src/config_infer_primary_yoloV3.txt sucessfully
Decodebin child added: source 

Decodebin child added: decodebin0 

Decodebin child added: source 

Decodebin child added: decodebin1 

Decodebin child added: source 

Decodebin child added: decodebin2 

Decodebin child added: source 

Decodebin child added: decodebin3 

Decodebin child added: source 

Decodebin child added: decodebin4 

Decodebin child added: source 

Decodebin child added: decodebin5 

Decodebin child added: source 

Decodebin child added: decodebin6 

Decodebin child added: source 

Decodebin child added: decodebin7 

Decodebin child added: source 

Decodebin child added: decodebin8 

Decodebin child added: source 

Decodebin child added: decodebin9 

Decodebin child added: source 

Decodebin child added: decodebin10 

Decodebin child added: source 

Decodebin child added: decodebin11 

Decodebin child added: source 

Decodebin child added: decodebin12 

Decodebin child added: source 

Decodebin child added: decodebin13 

Decodebin child added: source 

Decodebin child added: decodebin14 

Decodebin child added: source 

Decodebin child added: decodebin15 

Decodebin child added: source 

Decodebin child added: decodebin16 

Decodebin child added: source 

Decodebin child added: decodebin17 

Decodebin child added: source 

Decodebin child added: decodebin18 

Decodebin child added: source 

Decodebin child added: decodebin19 

Decodebin child added: source 

Decodebin child added: decodebin20 

Decodebin child added: source 

Decodebin child added: decodebin21 

Decodebin child added: source 

Decodebin child added: decodebin22 

Decodebin child added: source 

Decodebin child added: decodebin23 

Decodebin child added: source 

Decodebin child added: decodebin24 

Decodebin child added: source 

Decodebin child added: decodebin25 

Decodebin child added: source 

Decodebin child added: decodebin26 

Decodebin child added: source 

Decodebin child added: decodebin27 

Decodebin child added: source 

Decodebin child added: decodebin28 

Decodebin child added: source 

Decodebin child added: decodebin29 

Decodebin child added: source 

Decodebin child added: decodebin30 

Decodebin child added: source 

Decodebin child added: decodebin31 

Decodebin child added: source 

Decodebin child added: decodebin32 

Decodebin child added: source 

Decodebin child added: decodebin33 

Decodebin child added: source 

Decodebin child added: decodebin34 

Decodebin child added: source 

Decodebin child added: decodebin35 

Decodebin child added: source 

Decodebin child added: decodebin36 

Decodebin child added: source 

Decodebin child added: decodebin37 

Decodebin child added: source 

Decodebin child added: decodebin38 

Decodebin child added: source 

Decodebin child added: decodebin39 

Decodebin child added: source 

Decodebin child added: decodebin40 

Decodebin child added: source 

Decodebin child added: decodebin41 

Decodebin child added: source 

Decodebin child added: decodebin42 

Decodebin child added: source 

Decodebin child added: decodebin43 

Decodebin child added: source 

Decodebin child added: decodebin44 

Decodebin child added: source 

Decodebin child added: decodebin45 

Decodebin child added: source 

Decodebin child added: decodebin46 

Decodebin child added: source 

Decodebin child added: decodebin47 

Decodebin child added: source 

Decodebin child added: decodebin48 

Decodebin child added: source 

Decodebin child added: decodebin49 

Decodebin child added: source 

Decodebin child added: decodebin50 

Decodebin child added: source 

Decodebin child added: decodebin51 

Decodebin child added: source 

Decodebin child added: decodebin52 

Decodebin child added: source 

Decodebin child added: decodebin53 

Decodebin child added: h264parse0 

Decodebin child added: h264parse4 

Decodebin child added: h264parse2 

Decodebin child added: h264parse6 

Decodebin child added: h264parse7 
Decodebin child added: h264parse5 

Decodebin child added: capsfilter0 

Decodebin child added: h264parse1 

Decodebin child added: capsfilter2 

Decodebin child added: h264parse9 

Decodebin child added: h264parse8 


Decodebin child added: capsfilter1 
Decodebin child added: h264parse11 
Decodebin child added: h264parse10 
Decodebin child added: capsfilter3 

Decodebin child added: capsfilter8 

Decodebin child added: capsfilter7 

Decodebin child added: capsfilter5 
Decodebin child added: h264parse3 



Decodebin child added: capsfilter4 


Decodebin child added: capsfilter10 
Decodebin child added: h264parse13 
Decodebin child added: capsfilter6 


Decodebin child added: capsfilter9 

Decodebin child added: h264parse12 

Decodebin child added: h264parse15 

Decodebin child added: nvv4l2decoder2 

Decodebin child added: nvv4l2decoder1 


Decodebin child added: h264parse16 
Decodebin child added: capsfilter11 
Decodebin child added: capsfilter13 

Decodebin child added: h264parse14 

Decodebin child added: nvv4l2decoder5 
Decodebin child added: capsfilter14 

Decodebin child added: nvv4l2decoder6 

Decodebin child added: h264parse23 

Decodebin child added: nvv4l2decoder9 

Decodebin child added: nvv4l2decoder7 
Decodebin child added: h264parse20 

Decodebin child added: nvv4l2decoder0 

Decodebin child added: h264parse29 

Decodebin child added: h264parse24 

Decodebin child added: h264parse21 

Decodebin child added: h264parse26 

Decodebin child added: nvv4l2decoder4 

Decodebin child added: capsfilter12 


Decodebin child added: h264parse22 

Decodebin child added: nvv4l2decoder13 

Decodebin child added: h264parse28 
Decodebin child added: h264parse27 
Decodebin child added: h264parse33 
Decodebin child added: capsfilter15 


Decodebin child added: h264parse19 

Decodebin child added: h264parse32 

Decodebin child added: nvv4l2decoder11 

Decodebin child added: nvv4l2decoder8 


Decodebin child added: capsfilter16 

Decodebin child added: h264parse25 
Decodebin child added: capsfilter17 

Decodebin child added: nvv4l2decoder12 

Decodebin child added: h264parse37 

Decodebin child added: capsfilter21 


Decodebin child added: capsfilter23 


Decodebin child added: h264parse35 
Decodebin child added: h264parse34 

Decodebin child added: nvv4l2decoder3 

Decodebin child added: capsfilter18 

Decodebin child added: capsfilter22 
Decodebin child added: nvv4l2decoder10 


Decodebin child added: h264parse18 


Decodebin child added: capsfilter20 

Decodebin child added: h264parse36 
Decodebin child added: h264parse17 

Decodebin child added: h264parse30 
Decodebin child added: h264parse42 

Decodebin child added: capsfilter25 

Decodebin child added: h264parse41 

Decodebin child added: capsfilter19 

Decodebin child added: capsfilter24 

Decodebin child added: h264parse40 




Decodebin child added: h264parse45 
Decodebin child added: capsfilter26 

Decodebin child added: h264parse47 

Decodebin child added: h264parse44 

Decodebin child added: h264parse50 


Decodebin child added: h264parse38 

Decodebin child added: h264parse31 
Decodebin child added: h264parse39 


Decodebin child added: h264parse48 

Decodebin child added: capsfilter27 


Decodebin child added: capsfilter29 
Decodebin child added: nvv4l2decoder14 


Decodebin child added: h264parse49 

Decodebin child added: h264parse46 


Decodebin child added: capsfilter28 

Decodebin child added: h264parse43 


Decodebin child added: h264parse51 

Decodebin child added: capsfilter33 

Decodebin child added: capsfilter30 

Decodebin child added: capsfilter35 

Decodebin child added: h264parse52 

Decodebin child added: capsfilter36 

Decodebin child added: capsfilter44 

Decodebin child added: capsfilter42 

Decodebin child added: h264parse53 

Decodebin child added: capsfilter34 

Decodebin child added: capsfilter41 

Decodebin child added: capsfilter32 

Decodebin child added: capsfilter43 

Decodebin child added: capsfilter31 

Decodebin child added: capsfilter45 

Decodebin child added: capsfilter37 
Decodebin child added: capsfilter38 

Decodebin child added: capsfilter40 

Decodebin child added: capsfilter39 


Decodebin child added: capsfilter46 

Decodebin child added: capsfilter48 

Decodebin child added: capsfilter47 

Decodebin child added: capsfilter49 

Decodebin child added: capsfilter50 

Decodebin child added: capsfilter51 

Decodebin child added: nvv4l2decoder15 

Decodebin child added: nvv4l2decoder16 

Decodebin child added: nvv4l2decoder17 

Decodebin child added: nvv4l2decoder18 

Decodebin child added: nvv4l2decoder19 

Decodebin child added: nvv4l2decoder20 

Decodebin child added: capsfilter52 

Decodebin child added: nvv4l2decoder21 

Decodebin child added: nvv4l2decoder22 

Decodebin child added: nvv4l2decoder23 

Decodebin child added: nvv4l2decoder24 

Decodebin child added: nvv4l2decoder25 

Decodebin child added: nvv4l2decoder26 

Decodebin child added: nvv4l2decoder27 

Decodebin child added: nvv4l2decoder28 

Decodebin child added: nvv4l2decoder30 

Decodebin child added: nvv4l2decoder29 

Decodebin child added: capsfilter53 

Decodebin child added: nvv4l2decoder31 

Decodebin child added: nvv4l2decoder32 

Decodebin child added: nvv4l2decoder33 

Decodebin child added: nvv4l2decoder34 

Decodebin child added: nvv4l2decoder35 

Decodebin child added: nvv4l2decoder36 

Decodebin child added: nvv4l2decoder37 

Decodebin child added: nvv4l2decoder38 

Decodebin child added: nvv4l2decoder39 

Decodebin child added: nvv4l2decoder40 
Decodebin child added: nvv4l2decoder42 


Decodebin child added: nvv4l2decoder41 

Decodebin child added: nvv4l2decoder43 

Decodebin child added: nvv4l2decoder44 

Decodebin child added: nvv4l2decoder45 

Decodebin child added: nvv4l2decoder46 

Decodebin child added: nvv4l2decoder47 

Decodebin child added: nvv4l2decoder48 

Decodebin child added: nvv4l2decoder49 

Decodebin child added: nvv4l2decoder50 

Decodebin child added: nvv4l2decoder51 

Decodebin child added: nvv4l2decoder52 

Decodebin child added: nvv4l2decoder53 

In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5dec02c6e0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5dac02e0c0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5dc8029c40)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5dd802ca00)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5dcc001b20)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5dd002a4e0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d1402bc60)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5ddc02a9a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d2002e4a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d7402d840)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d34030060)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5cfc02cd00)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5dbc02c8a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5dc002a340)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d4802e5a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d7002b220)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5de802ef00)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5da002bf20)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5cf802e380)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5de00295a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5cf402caa0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d9c02c4e0)>
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5dc402d700)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5db002fcc0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d9402afa0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d5002b200)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d1802bf20)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d3002eec0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d2c02d300)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d5402d340)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d0c029e80)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d7802e180)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d9802b6a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d4c02af40)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d04031800)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5db402f8a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d6c02e220)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d1002aec0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5dd402b000)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d3802d4e0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5e04066cc0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d64029960)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d9002e100)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d08028c80)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d8c02a360)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d5802ec00)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d68030720)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d8002b960)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5e64036240)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d6002c0e0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d2802e640)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5da402e020)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066dde20 (GstCapsFeatures at 0x7f5d3c02c4c0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f62066ddd00 (GstCapsFeatures at 0x7f5d4402e860)>
TOT FPS 10.79   AVG FPS 0   STREAMS UP 54
TOT FPS 32.4   AVG FPS 1   STREAMS UP 54
TOT FPS 21.59   AVG FPS 0   STREAMS UP 54
TOT FPS 32.39   AVG FPS 1   STREAMS UP 54
TOT FPS 32.39   AVG FPS 1   STREAMS UP 54
TOT FPS 21.58   AVG FPS 0   STREAMS UP 54
TOT FPS 32.37   AVG FPS 1   STREAMS UP 54
TOT FPS 21.59   AVG FPS 0   STREAMS UP 54

nvidia-smi dmon output when the model is running:

# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0    71    77     -   100    39     0     0  5000   810
    0    72    77     -   100    39     0     0  5000   795
    0    73    77     -   100    46     0     0  5000   705
    0    70    77     -   100    40     0     0  5000   765
    0    73    77     -   100    45     0     0  5000   675
    0    70    77     -   100    43     0     0  5000   735
    0    69    77     -   100    52     0     0  5000   690
    0    65    78     -   100    43     0     0  5000   720
    0    70    77     -   100    48     0     0  5000   645
    0    69    78     -   100    42     0     0  5000   720
    0    63    78     -   100    51     0     0  5000   705
    0    64    78     -   100    41     0     0  5000   720
    0    70    78     -   100    52     0     0  5000   690
    0    69    78     -   100    41     0     0  5000   720
    0    66    78     -   100    47     0     0  5000   750
    0    69    78     -   100    40     0     0  5000   720
    0    64    78     -   100    51     0     0  5000   705
    0    69    79     -   100    41     0     0  5000   705
    0    67    79     -   100    43     0     0  5000   750
    0    67    79     -   100    41     0     0  5000   705
    0    64    79     -   100    49     0     0  5000   705

With the patch:

 docker run   -it   --rm   --net=host   --runtime nvidia   -e DISPLAY=$DISPLAY   -v /tmp/.X11-unix/:/tmp/.X11-unix   --cap-add=SYS_PTRACE   --security-opt seccomp=unconfined   --device /dev/video0   --privileged   --expose 8554   deepstream-custom-slow
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
Number of streams 54
pipeline.py:75: PyGIDeprecationWarning: Since version 3.11, calling threads_init is no longer needed. See: https://wiki.gnome.org/PyGObject/Threading
  GObject.threads_init()

(gst-plugin-scanner:8): GStreamer-WARNING **: 03:54:38.172: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:8): GLib-GObject-WARNING **: 03:54:38.482: specified class size for type 'GstCompositor' is smaller than the parent type's 'GstVideoAggregator' class size

(gst-plugin-scanner:8): GLib-GObject-CRITICAL **: 03:54:38.482: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed

(gst-plugin-scanner:8): GLib-CRITICAL **: 03:54:38.482: g_once_init_leave: assertion 'result != 0' failed

(gst-plugin-scanner:8): GStreamer-CRITICAL **: 03:54:38.482: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
Creating Pipeline 

Creating nvstreammux 

Creating source_bin  0  
 
Creating source bin
source-bin-00
Creating source_bin  1  
 
Creating source bin
source-bin-01
Creating source_bin  2  
 
Creating source bin
source-bin-02
Creating source_bin  3  
 
Creating source bin
source-bin-03
Creating source_bin  4  
 
Creating source bin
source-bin-04
Creating source_bin  5  
 
Creating source bin
source-bin-05
Creating source_bin  6  
 
Creating source bin
source-bin-06
Creating source_bin  7  
 
Creating source bin
source-bin-07
Creating source_bin  8  
 
Creating source bin
source-bin-08
Creating source_bin  9  
 
Creating source bin
source-bin-09
Creating source_bin  10  
 
Creating source bin
source-bin-10
Creating source_bin  11  
 
Creating source bin
source-bin-11
Creating source_bin  12  
 
Creating source bin
source-bin-12
Creating source_bin  13  
 
Creating source bin
source-bin-13
Creating source_bin  14  
 
Creating source bin
source-bin-14
Creating source_bin  15  
 
Creating source bin
source-bin-15
Creating source_bin  16  
 
Creating source bin
source-bin-16
Creating source_bin  17  
 
Creating source bin
source-bin-17
Creating source_bin  18  
 
Creating source bin
source-bin-18
Creating source_bin  19  
 
Creating source bin
source-bin-19
Creating source_bin  20  
 
Creating source bin
source-bin-20
Creating source_bin  21  
 
Creating source bin
source-bin-21
Creating source_bin  22  
 
Creating source bin
source-bin-22
Creating source_bin  23  
 
Creating source bin
source-bin-23
Creating source_bin  24  
 
Creating source bin
source-bin-24
Creating source_bin  25  
 
Creating source bin
source-bin-25
Creating source_bin  26  
 
Creating source bin
source-bin-26
Creating source_bin  27  
 
Creating source bin
source-bin-27
Creating source_bin  28  
 
Creating source bin
source-bin-28
Creating source_bin  29  
 
Creating source bin
source-bin-29
Creating source_bin  30  
 
Creating source bin
source-bin-30
Creating source_bin  31  
 
Creating source bin
source-bin-31
Creating source_bin  32  
 
Creating source bin
source-bin-32
Creating source_bin  33  
 
Creating source bin
source-bin-33
Creating source_bin  34  
 
Creating source bin
source-bin-34
Creating source_bin  35  
 
Creating source bin
source-bin-35
Creating source_bin  36  
 
Creating source bin
source-bin-36
Creating source_bin  37  
 
Creating source bin
source-bin-37
Creating source_bin  38  
 
Creating source bin
source-bin-38
Creating source_bin  39  
 
Creating source bin
source-bin-39
Creating source_bin  40  
 
Creating source bin
source-bin-40
Creating source_bin  41  
 
Creating source bin
source-bin-41
Creating source_bin  42  
 
Creating source bin
source-bin-42
Creating source_bin  43  
 
Creating source bin
source-bin-43
Creating source_bin  44  
 
Creating source bin
source-bin-44
Creating source_bin  45  
 
Creating source bin
source-bin-45
Creating source_bin  46  
 
Creating source bin
source-bin-46
Creating source_bin  47  
 
Creating source bin
source-bin-47
Creating source_bin  48  
 
Creating source bin
source-bin-48
Creating source_bin  49  
 
Creating source bin
source-bin-49
Creating source_bin  50  
 
Creating source bin
source-bin-50
Creating source_bin  51  
 
Creating source bin
source-bin-51
Creating source_bin  52  
 
Creating source bin
source-bin-52
Creating source_bin  53  
 
Creating source bin
source-bin-53
Creating Pgie 
 
Unknown or legacy key specified 'is-classifier' for group [property]
Warn: 'threshold' parameter has been deprecated. Use 'pre-cluster-threshold' instead.
pipeline.py:136: PyGIDeprecationWarning: GObject.MainLoop is deprecated; use GLib.MainLoop instead
  loop = GObject.MainLoop()
Starting pipeline 

0:00:00.719429631     1      0x3f90a30 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
Loading pre-trained weights...
Loading weights of yolov3 complete!
Total Number of weights read : 62001757
Loading pre-trained weights...
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
Loading weights of yolov3 complete!
Total Number of weights read : 62001757
Building Yolo network...
      layer               inp_size            out_size       weightPtr
(0)   conv-bn-leaky     3 x 608 x 608      32 x 608 x 608    992   
(1)   conv-bn-leaky    32 x 608 x 608      64 x 304 x 304    19680 
(2)   conv-bn-leaky    64 x 304 x 304      32 x 304 x 304    21856 
(3)   conv-bn-leaky    32 x 304 x 304      64 x 304 x 304    40544 
(4)   skip             64 x 304 x 304      64 x 304 x 304        - 
(5)   conv-bn-leaky    64 x 304 x 304     128 x 152 x 152    114784
(6)   conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    123232
(7)   conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    197472
(8)   skip            128 x 152 x 152     128 x 152 x 152        - 
(9)   conv-bn-leaky   128 x 152 x 152      64 x 152 x 152    205920
(10)  conv-bn-leaky    64 x 152 x 152     128 x 152 x 152    280160
(11)  skip            128 x 152 x 152     128 x 152 x 152        - 
(12)  conv-bn-leaky   128 x 152 x 152     256 x  76 x  76    576096
(13)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    609376
(14)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    905312
(15)  skip            256 x  76 x  76     256 x  76 x  76        - 
(16)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    938592
(17)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1234528
(18)  skip            256 x  76 x  76     256 x  76 x  76        - 
(19)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1267808
(20)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1563744
(21)  skip            256 x  76 x  76     256 x  76 x  76        - 
(22)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1597024
(23)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    1892960
(24)  skip            256 x  76 x  76     256 x  76 x  76        - 
(25)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    1926240
(26)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2222176
(27)  skip            256 x  76 x  76     256 x  76 x  76        - 
(28)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2255456
(29)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2551392
(30)  skip            256 x  76 x  76     256 x  76 x  76        - 
(31)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2584672
(32)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    2880608
(33)  skip            256 x  76 x  76     256 x  76 x  76        - 
(34)  conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    2913888
(35)  conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    3209824
(36)  skip            256 x  76 x  76     256 x  76 x  76        - 
(37)  conv-bn-leaky   256 x  76 x  76     512 x  38 x  38    4391520
(38)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    4523616
(39)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    5705312
(40)  skip            512 x  38 x  38     512 x  38 x  38        - 
(41)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    5837408
(42)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    7019104
(43)  skip            512 x  38 x  38     512 x  38 x  38        - 
(44)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    7151200
(45)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    8332896
(46)  skip            512 x  38 x  38     512 x  38 x  38        - 
(47)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    8464992
(48)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    9646688
(49)  skip            512 x  38 x  38     512 x  38 x  38        - 
(50)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    9778784
(51)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    10960480
(52)  skip            512 x  38 x  38     512 x  38 x  38        - 
(53)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    11092576
(54)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    12274272
(55)  skip            512 x  38 x  38     512 x  38 x  38        - 
(56)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    12406368
(57)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    13588064
(58)  skip            512 x  38 x  38     512 x  38 x  38        - 
(59)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    13720160
(60)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    14901856
(61)  skip            512 x  38 x  38     512 x  38 x  38        - 
(62)  conv-bn-leaky   512 x  38 x  38    1024 x  19 x  19    19624544
(63)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    20150880
(64)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    24873568
(65)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(66)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    25399904
(67)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    30122592
(68)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(69)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    30648928
(70)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    35371616
(71)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(72)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    35897952
(73)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    40620640
(74)  skip           1024 x  19 x  19    1024 x  19 x  19        - 
(75)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    41146976
(76)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    45869664
(77)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    46396000
(78)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    51118688
(79)  conv-bn-leaky  1024 x  19 x  19     512 x  19 x  19    51645024
(80)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    56367712
(81)  conv-linear    1024 x  19 x  19     255 x  19 x  19    56629087
(82)  yolo            255 x  19 x  19     255 x  19 x  19    56629087
(83)  route                  -            512 x  19 x  19    56629087
(84)  conv-bn-leaky   512 x  19 x  19     256 x  19 x  19    56761183
(85)  upsample        256 x  19 x  19     256 x  38 x  38        - 
(86)  route                  -            768 x  38 x  38    56761183
(87)  conv-bn-leaky   768 x  38 x  38     256 x  38 x  38    56958815
(88)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    58140511
(89)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    58272607
(90)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    59454303
(91)  conv-bn-leaky   512 x  38 x  38     256 x  38 x  38    59586399
(92)  conv-bn-leaky   256 x  38 x  38     512 x  38 x  38    60768095
(93)  conv-linear     512 x  38 x  38     255 x  38 x  38    60898910
(94)  yolo            255 x  38 x  38     255 x  38 x  38    60898910
(95)  route                  -            256 x  38 x  38    60898910
(96)  conv-bn-leaky   256 x  38 x  38     128 x  38 x  38    60932190
(97)  upsample        128 x  38 x  38     128 x  76 x  76        - 
(98)  route                  -            384 x  76 x  76    60932190
(99)  conv-bn-leaky   384 x  76 x  76     128 x  76 x  76    60981854
(100) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61277790
(101) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61311070
(102) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61607006
(103) conv-bn-leaky   256 x  76 x  76     128 x  76 x  76    61640286
(104) conv-bn-leaky   128 x  76 x  76     256 x  76 x  76    61936222
(105) conv-linear     256 x  76 x  76     255 x  76 x  76    62001757
(106) yolo            255 x  76 x  76     255 x  76 x  76    62001757
Output yolo blob names :
yolo_83
yolo_95
yolo_107
Total number of yolo layers: 257
Building yolo network complete!
Building the TensorRT Engine...
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
WARNING: [TRT]: Detected invalid timing cache, setup a local cache instead
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
Building complete!
0:04:51.293063848     1      0x3f90a30 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1947> [UID = 1]: serialize cuda engine to file: /src/model_b54_gpu0_int8.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 4
0   INPUT  kFLOAT data            3x608x608       
1   OUTPUT kFLOAT yolo_83         255x19x19       
2   OUTPUT kFLOAT yolo_95         255x38x38       
3   OUTPUT kFLOAT yolo_107        255x76x76       

0:04:51.747061392     1      0x3f90a30 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:/src/config_infer_primary_yoloV3.txt sucessfully
Decodebin child added: source 

Decodebin child added: decodebin0 

Decodebin child added: source 

Decodebin child added: decodebin1 

Decodebin child added: source 

Decodebin child added: decodebin2 

Decodebin child added: source 

Decodebin child added: decodebin3 

Decodebin child added: source 

Decodebin child added: decodebin4 

Decodebin child added: source 

Decodebin child added: decodebin5 

Decodebin child added: source 

Decodebin child added: decodebin6 

Decodebin child added: source 

Decodebin child added: decodebin7 

Decodebin child added: source 

Decodebin child added: decodebin8 

Decodebin child added: source 

Decodebin child added: decodebin9 

Decodebin child added: source 

Decodebin child added: decodebin10 

Decodebin child added: source 

Decodebin child added: decodebin11 

Decodebin child added: source 

Decodebin child added: decodebin12 

Decodebin child added: source 

Decodebin child added: decodebin13 

Decodebin child added: source 

Decodebin child added: decodebin14 

Decodebin child added: source 

Decodebin child added: decodebin15 

Decodebin child added: source 

Decodebin child added: decodebin16 

Decodebin child added: source 

Decodebin child added: decodebin17 

Decodebin child added: source 

Decodebin child added: decodebin18 

Decodebin child added: source 

Decodebin child added: decodebin19 

Decodebin child added: source 

Decodebin child added: decodebin20 

Decodebin child added: source 

Decodebin child added: decodebin21 

Decodebin child added: source 

Decodebin child added: decodebin22 

Decodebin child added: source 

Decodebin child added: decodebin23 

Decodebin child added: source 

Decodebin child added: decodebin24 

Decodebin child added: source 

Decodebin child added: decodebin25 

Decodebin child added: source 

Decodebin child added: decodebin26 

Decodebin child added: source 

Decodebin child added: decodebin27 

Decodebin child added: source 

Decodebin child added: decodebin28 

Decodebin child added: source 

Decodebin child added: decodebin29 

Decodebin child added: source 

Decodebin child added: decodebin30 

Decodebin child added: source 

Decodebin child added: decodebin31 

Decodebin child added: source 

Decodebin child added: decodebin32 

Decodebin child added: source 

Decodebin child added: decodebin33 

Decodebin child added: source 

Decodebin child added: decodebin34 

Decodebin child added: source 

Decodebin child added: decodebin35 

Decodebin child added: source 

Decodebin child added: decodebin36 

Decodebin child added: source 

Decodebin child added: decodebin37 

Decodebin child added: source 

Decodebin child added: decodebin38 

Decodebin child added: source 

Decodebin child added: decodebin39 

Decodebin child added: source 

Decodebin child added: decodebin40 

Decodebin child added: source 

Decodebin child added: decodebin41 

Decodebin child added: source 

Decodebin child added: decodebin42 

Decodebin child added: source 

Decodebin child added: decodebin43 

Decodebin child added: source 

Decodebin child added: decodebin44 

Decodebin child added: source 

Decodebin child added: decodebin45 

Decodebin child added: source 

Decodebin child added: decodebin46 

Decodebin child added: source 

Decodebin child added: decodebin47 

Decodebin child added: source 

Decodebin child added: decodebin48 

Decodebin child added: source 

Decodebin child added: decodebin49 

Decodebin child added: source 

Decodebin child added: decodebin50 

Decodebin child added: source 

Decodebin child added: decodebin51 

Decodebin child added: source 

Decodebin child added: decodebin52 

Decodebin child added: source 

Decodebin child added: decodebin53 

Decodebin child added: h264parse0 

Decodebin child added: capsfilter0 

Decodebin child added: h264parse1 

Decodebin child added: capsfilter1 

Decodebin child added: h264parse2 

Decodebin child added: h264parse3 

Decodebin child added: h264parse4 

Decodebin child added: h264parse5 

Decodebin child added: capsfilter2 

Decodebin child added: capsfilter3 

Decodebin child added: capsfilter4 

Decodebin child added: h264parse6 
Decodebin child added: nvv4l2decoder0 


Decodebin child added: nvv4l2decoder1 

Decodebin child added: capsfilter5 

Decodebin child added: h264parse7 

Decodebin child added: capsfilter7 

Decodebin child added: nvv4l2decoder3 

Decodebin child added: capsfilter6 

Decodebin child added: h264parse8 

Decodebin child added: nvv4l2decoder2 

Decodebin child added: capsfilter8 

Decodebin child added: nvv4l2decoder5 

Decodebin child added: nvv4l2decoder4 

Decodebin child added: nvv4l2decoder6 

Decodebin child added: nvv4l2decoder7 

Decodebin child added: h264parse9 

Decodebin child added: capsfilter9 

Decodebin child added: h264parse11 

Decodebin child added: h264parse10 

Decodebin child added: h264parse12 

Decodebin child added: h264parse13 

Decodebin child added: capsfilter10 

Decodebin child added: capsfilter11 

Decodebin child added: capsfilter12 

Decodebin child added: capsfilter13 

Decodebin child added: h264parse14 

Decodebin child added: capsfilter14 

Decodebin child added: h264parse15 

Decodebin child added: capsfilter15 

Decodebin child added: h264parse17 

Decodebin child added: h264parse16 

Decodebin child added: nvv4l2decoder8 

Decodebin child added: capsfilter16 

Decodebin child added: nvv4l2decoder9 

Decodebin child added: capsfilter17 

Decodebin child added: nvv4l2decoder10 

Decodebin child added: h264parse18 

Decodebin child added: nvv4l2decoder11 

Decodebin child added: h264parse19 

Decodebin child added: capsfilter18 

Decodebin child added: nvv4l2decoder12 

Decodebin child added: h264parse20 

Decodebin child added: nvv4l2decoder13 

Decodebin child added: capsfilter19 

Decodebin child added: nvv4l2decoder15 

Decodebin child added: nvv4l2decoder14 

Decodebin child added: h264parse21 

Decodebin child added: h264parse22 

Decodebin child added: h264parse23 

Decodebin child added: h264parse24 

Decodebin child added: capsfilter20 

Decodebin child added: nvv4l2decoder16 

Decodebin child added: capsfilter21 

Decodebin child added: capsfilter22 

Decodebin child added: capsfilter23 

Decodebin child added: nvv4l2decoder17 

Decodebin child added: h264parse25 

Decodebin child added: capsfilter24 

Decodebin child added: h264parse26 

Decodebin child added: h264parse28 

Decodebin child added: capsfilter25 

Decodebin child added: h264parse29 

Decodebin child added: nvv4l2decoder19 

Decodebin child added: nvv4l2decoder20 

Decodebin child added: h264parse27 

Decodebin child added: nvv4l2decoder18 

Decodebin child added: nvv4l2decoder21 

Decodebin child added: capsfilter26 

Decodebin child added: capsfilter27 

Decodebin child added: nvv4l2decoder22 
Decodebin child added: capsfilter28 

Decodebin child added: h264parse30 


Decodebin child added: h264parse31 

Decodebin child added: nvv4l2decoder23 
Decodebin child added: nvv4l2decoder24 

Decodebin child added: capsfilter29 


Decodebin child added: capsfilter31 

Decodebin child added: capsfilter30 

Decodebin child added: h264parse32 

Decodebin child added: capsfilter32 

Decodebin child added: h264parse33 

Decodebin child added: nvv4l2decoder25 

Decodebin child added: capsfilter33 

Decodebin child added: nvv4l2decoder26 

Decodebin child added: nvv4l2decoder27 

Decodebin child added: nvv4l2decoder28 

Decodebin child added: h264parse35 
Decodebin child added: h264parse36 

Decodebin child added: h264parse34 

Decodebin child added: nvv4l2decoder29 

Decodebin child added: h264parse43 

Decodebin child added: h264parse45 

Decodebin child added: nvv4l2decoder30 
Decodebin child added: h264parse48 
Decodebin child added: h264parse52 
Decodebin child added: h264parse46 

Decodebin child added: h264parse51 

Decodebin child added: h264parse41 

Decodebin child added: h264parse38 
Decodebin child added: h264parse50 

Decodebin child added: capsfilter36 

Decodebin child added: capsfilter34 

Decodebin child added: nvv4l2decoder31 

Decodebin child added: h264parse39 
Decodebin child added: h264parse40 

Decodebin child added: h264parse42 

Decodebin child added: h264parse44 



Decodebin child added: h264parse47 

Decodebin child added: h264parse37 

Decodebin child added: capsfilter38 

Decodebin child added: h264parse53 
Decodebin child added: capsfilter39 

Decodebin child added: capsfilter37 


Decodebin child added: capsfilter41 
Decodebin child added: h264parse49 


Decodebin child added: capsfilter42 


Decodebin child added: capsfilter40 

Decodebin child added: capsfilter48 
Decodebin child added: nvv4l2decoder34 



Decodebin child added: capsfilter46 

Decodebin child added: nvv4l2decoder33 
Decodebin child added: capsfilter43 


Decodebin child added: capsfilter49 

Decodebin child added: capsfilter50 
Decodebin child added: capsfilter45 



Decodebin child added: capsfilter35 

Decodebin child added: capsfilter47 

Decodebin child added: capsfilter52 

Decodebin child added: capsfilter44 

Decodebin child added: capsfilter51 

Decodebin child added: nvv4l2decoder32 


Decodebin child added: capsfilter53 

Decodebin child added: nvv4l2decoder35 

Decodebin child added: nvv4l2decoder36 

Decodebin child added: nvv4l2decoder37 

Decodebin child added: nvv4l2decoder38 

Decodebin child added: nvv4l2decoder39 

Decodebin child added: nvv4l2decoder40 

Decodebin child added: nvv4l2decoder41 

Decodebin child added: nvv4l2decoder42 

Decodebin child added: nvv4l2decoder43 

Decodebin child added: nvv4l2decoder44 

Decodebin child added: nvv4l2decoder45 

Decodebin child added: nvv4l2decoder47 

Decodebin child added: nvv4l2decoder46 

Decodebin child added: nvv4l2decoder48 

Decodebin child added: nvv4l2decoder49 

Decodebin child added: nvv4l2decoder50 

Decodebin child added: nvv4l2decoder53 
Decodebin child added: nvv4l2decoder51 

Decodebin child added: nvv4l2decoder52 


In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f2700009660)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f272c0037a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f269c001a40)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f267402f360)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f268802ba40)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f25cc02a340)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f26940290e0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f269802d960)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f261802bcc0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f26a402ac20)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x29cc0a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f2660031560)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f268c029cc0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f264402d340)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f264c029f00)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f262802c760)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f260c02fe60)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f25c802a8e0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f25d802eec0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f25b402a300)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f2730003b00)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f2740003b40)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f267c02ea60)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f25a402e600)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f25b002b980)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f261c02ae80)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f25e40301a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f261002a5a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f26200280e0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f268002dce0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f25a802d960)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f25e802bc20)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f2634029c40)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f25ac029c60)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f25bc02e020)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f261402d000)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f25c002f320)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f25d402dca0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f2624028b60)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f268402c700)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f26380306a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f258802bb20)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f25f002d560)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f25e002e640)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f25a002c400)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f259002e340)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f259802c0a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f26840034a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f2748011840)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f265002b1a0)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f264802bf00)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f264002e020)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44d60 (GstCapsFeatures at 0x7f260402a340)>
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f2985a44c40 (GstCapsFeatures at 0x7f26e8002f40)>
TOT FPS 0.0   AVG FPS 0   STREAMS UP 0
TOT FPS 75.55   AVG FPS 1   STREAMS UP 54
TOT FPS 64.74   AVG FPS 1   STREAMS UP 54
TOT FPS 75.54   AVG FPS 1   STREAMS UP 54
TOT FPS 64.78   AVG FPS 1   STREAMS UP 54
TOT FPS 75.54   AVG FPS 1   STREAMS UP 54
TOT FPS 64.8   AVG FPS 1   STREAMS UP 54
TOT FPS 64.73   AVG FPS 1   STREAMS UP 54
TOT FPS 75.57   AVG FPS 1   STREAMS UP 54
TOT FPS 64.78   AVG FPS 1   STREAMS UP 54
TOT FPS 75.55   AVG FPS 1   STREAMS UP 54
TOT FPS 64.73   AVG FPS 1   STREAMS UP 54
TOT FPS 75.52   AVG FPS 1   STREAMS UP 54
TOT FPS 64.79   AVG FPS 1   STREAMS UP 54

nvidia-smi dmon output when the model is running:

# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0    47    78     -    62    19     0     9  5000  1590
    0    47    79     -    37    17     0     0  5000  1410
    0    70    79     -    29    25     0     0  5000   930
    0    47    79     -   100    26     0    27  5000  1590
    0    47    79     -     0     0     0     0  5000  1590
    0    68    79     -    24     8     0     0  5000   810
    0    45    79     -   100    39     0    15  5000  1575
    0    47    79     -     0     0     0     0  5000  1590
    0   122    79     -    37    28     0     0  5000   615
    0    40    79     -    89    26     0    36  5000  1515
    0    47    79     -     0     0     0     0  5000  1590
    0    47    80     -    14    14     0     0  5000   750
    0    72    79     -   100    28     0    28  5000  1425
    0    47    79     -    27     2     0     5  5000  1590
    0    47    79     -     0     0     0     0  5000  1590
    0    71    80     -    61    35     0     0  5000   765
    0    41    79     -    81    22     0    13  5000  1560
    0    47    79     -     5     1     0     5  5000  1590
    0    47    80     -     5     5     0     0  5000  1155
    0    70    79     -   100    33     0    13  5000  1095
    0    47    79     -     9     0     0     0  5000  1590
    0    43    79     -     1     1     0     5  5000  1440
    0    69    80     -   100    37     0    13  5000   840

With the patch the frame rate doubled but 60-70fps are still a low number compared to what I used to see with Python + TensorRT. However, it seems that the gpu utilization is not 100%, so there should be room to improve the FPS. How can I optimize the git gist above to reach higher throughput? I connected 54 30fps streams so for sure there is more than enough data to process.

What value did you get using Python + TensorRT?
Using deepstream, you get a total, let’s say average frame fps 65 after you apply the patch, plus total sources 54, it’s 65*50=3250

@amycao the value I show above it the total number of frames (see my code in the gist). So if the total average is 65 and there are 50 streams, the fps per stream is 65/40=1.3.
With Python I get more than 200 fps (maybe it could even reach 400fps) for comparable model (Yolo V5) using FP16. For sure on a Tesla T4 Yolo V3 should do more than 60-70FPS especially if optimized with TensorRT and in INT8 mode.

@amycao

  • I noticed that when the pipeline is running there is one CPU core that reach 100% usage because of the Python script that start the pipeline.
  • None of the code in script is causing this 100% usage (I verified it using a Python profiler). So it must be something in the underlying deepstream python called by my script.
  • Shouldn’t the pipeline run on the GPU? And even if it uses the CPU, I am running the code on an Intel Xeon. It should not be a bottleneck. If it uses CPU, it should at least use multiple cores. But even then, it is strange. The CPU is a pretty powerful one.
  • Could you try to reproduce this issue using the gist above, please? The fact that CPU is a bottleneck sounds like an issue in the implementation to me.

I am reading online ( Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream | paulbridger.com ) that Deepstream does not support batch postprocessing, and that therefore each frame has to be processed individually. Is this still the case with DS 6.0? Post processing on the cpu (if that’s the case), frame by frame (if that’s the case) would explain the poor performance of the model.

Will try to repro your issue and update once progress.

You are using wrong way to calculate FPS for deepstream pipeline. your thread which calculate fps runing seperately, every 5 seconds period, caculate it as count got from frame counts sumed / current time - prev time.
it should be: frame counts * sources * 1000000 / total time running until the stream data enter into pad you probed.

@amycao what I am doing is the following, every 5 seconds I do:
(total number of frames processed during last 5 seconds) / (current time - prev time = 5 seconds). Note the Counter class keeps track of the frame counts for all streams in a dictionary cls.count where the key is the pad_index and the value is the count.
So it’s the total number of frames processed during the last 5 seconds, divided by 5 seconds, that is the frames per second processed by the pipeline. If you divide the results by the number of streams you get the FPS per stream.
Just print the current time - prev time and you will see it’s equal to 5. Also, the number of frame counted is reset every 5 seconds in the same function.
I think this is right. Your formula would be correct assuming you keep track of the time in milliseconds (Python uses seconds) and it’s just the same. The only different is that you keep track of the FPS since the application started and you divide by the time since the application started. It’s the same.

Hi @mfoglio ,
sorry, you are using yoloV3 model under /opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo/ , right?

Thanks!

Hi @mchi , yes I am ;) I have no idea how that Yolo is implemented but there is a CPU and at 100% utilization. I guess that’s the bottleneck.

are you using DeepStream 6.0GA ? what’s your platform? ===> Can you please provide the setup info as other topics to save our time to trigae the issue?

If you are using T4, I think it will be using along with a powerful x86 CPU, do you mean all the CPU cores of the powerful x86 CPU are 100% utilization?

I am using 6.0 container with triton. Following the gist here Yolo V3 slow - #4 by mfoglio you can reproduce the example. You’ll have to uncomment line 10 and 11 of setup.sh to apply the patch your support provided in this forum.
I am using an EC2 instance with T4 on AWS (instances G4). It has a T4, 64Gb of RAM and 16 Cores Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz. Note that using other custom Python + TensorRT implementations of Yolo model I get hundreds of FPS in FP16 (I don’t remember the exact number, it was between 250 and 600 FPS).
With this Yolo V3 implementation only one of the 16 cores reaches a 100% utilization. I suspect the code won’t run multicore and therefore causes the bottleneck.

About calculating FPS, you can not gurantee in the 5s period time, the frame count is accurate, right? so it should not be the way to do like this.