Deepstream-app with classification net as primary-gie won't show labels most of the time

Hello everyone.
I’m having some trouble with a deepstream-app implementation.
This comes from this topic, but diverged from the original question.

Problem is that the classification labels are only shown with this mp4 video, where tag is correctly deployed as aeropane.
The model fails tagging on other scenarios.

=================== tests i’ve done ====================
An rtsp camera as source, where I stand in front of the camera.
The output of the deepstream-app on the previous attemp.
An mp4 video of some online teacher.
A mp4 video of a dog.

I extracted frames from the three videos and input them on tlt, where they were correctly tagged, as below.

================= My config files ======================
So I think it has to do with the config files parameters instead of tlt.

# PR. GIE CON RED DE CLASIFICACION EN VEZ DE DETECCION
# SOURCES -> MUX -> OSD -> PR. GIE -> SINKS -> TILED

# Copyright (c) 2019-2020 NVIDIA Corporation.  All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1240
height=980
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:t3lco2019@192.168.100.64:554/ch1/main/av_stream
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source1]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
#uri=rtsp://<anothers cam ip>
uri=rtsp://<my cams ip>
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source2]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP 
type=3
#uri=file://../streams/outJP2.mp4
#uri=file://../streams/dog.mp4
#uri=file://../streams/maxwell.mp4
uri=file://../streams/planes.mp4
#uri=file://../streams/sample_1080p_h265.mp4
#uri=file://../streams/classification_test_video.mp4
num-sources=1
#drop-frame-interval=4
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source3]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP 
type=3
uri=file:/opt/nvidia/deepstream/deepstream/controlflow/videos/sample_1080p_h264.mp4
#uri=file: /opt/nvidia/deepstream/deepstream/controlflow/videos/streams/sample_720p.mp4
num-sources=1
#drop-frame-interval=4
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0


[sink0]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File
type=5
sync=1
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=outJP.mp4
source-id=0

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=1
bitrate=3000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
# set below properties in case of RTSPStreaming  
rtsp-port=8555
udp-port=5401

[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=1
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=8
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0
## If set to TRUE, system timestamp will be attached as ntp timestamp
## If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached
# attach-sys-ts-as-ntp=1
# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.

[primary-gie]
enable=1
gpu-id=0
model-engine-file=../models/Controlflow_tlt_classification_full/final_model_full_labels.etlt.engine
batch-size=1
#Required by the app for OSD, not a plugin property
## 0=FP32, 1=INT8, 2=FP16 mode
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_secondary_controlflow_as_primary.txt

[tests]
file-loop=1

this is the config file for the primary-gie.

# Copyright (c) 2020 NVIDIA Corporation.  All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.

[property]
gpu-id=0
net-scale-factor=1.0
tlt-model-key=<my private key>
tlt-encoded-model=../models/Controlflow_tlt_classification_full/final_model_full_labels.etlt
labelfile-path=../models/Controlflow_tlt_classification_full/labels.txt
#int8-calib-file=../models/Controlflow_tlt/dashcamnet_int8.txt
model-engine-file=../models/Controlflow_tlt_classification_full/final_model_full_labels.etlt.engine
#input-dims=3;544;960;0
input-dims=3;224;224;0
uff-input-blob-name=input_1
batch-size= 1 #8 #3
process-mode=1
model-color-format=0
#model-color-format=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
network-type=1
#num-detected-classes=7
interval=2
gie-unique-id=1
output-blob-names=predictions/Softmax
threshold=0.05
classifier-threshold=0.2
offsets=123.67;116.28;103.53

The offset values were recomended from the other answer, but I don’t actually know how to get them.

Thanks for your help!

Do you mean, if the input is RTSP stream, objects can’t be tagged correctly. if input is frames (jpeg file?), objects can’t be tagged correctly. What are the configure file and other changes between these two methods?

Thanks!

I’m closing this topic due to there is no update from you for a period, assuming this issue was resolved.
If still need the support, please open a new topic. Thanks

Hi @ai12
Did you get this issue solved?

Thanks!