Hello everyone.
I just made a config file for deepstream-app to run with a classifier as primary (and only) gie.
The problem is that everything is listed as background
Any help on this will be apreciated!
=================== Things I tried ======================
- Using a rtsp camera and capturing myself, a bottle, among others
- Using a rtsp camera and capturing an internet photo of a plane shown on my cell
- Using a rtsp camera and capturing an image on validation dataset shown on my computer
- Using a rtsp camera and capturing an image on test dataset shown on my computer
- Giving this video as input instead of a rtsp camera: https://www.youtube.com/watch?v=G0usamnRrMM
================== Some Outputs ========================
=================== About the model =======================
I trained the model using the classification example on tlt docker. I ran the full 80 epochs, pruned, and retrained again 80 epochs.
I’m exporting on FP32 and deepstream-app is generating the engine file.
I dont know where the engine file is generated, it is not on the folder I ask it to be.
Here it is the confusion matrix of the model and the classification report.
precision recall f1-score support
aeroplane 0.74 0.74 0.74 134
bicycle 0.36 0.57 0.44 111
bird 0.74 0.56 0.64 153
boat 0.52 0.76 0.62 102
bottle 0.20 0.19 0.19 142
bus 0.47 0.49 0.48 85
car 0.53 0.45 0.48 233
cat 0.67 0.64 0.66 216
chair 0.20 0.17 0.18 224
cow 0.62 0.48 0.54 61
diningtable 0.14 0.18 0.16 108
dog 0.57 0.46 0.51 258
horse 0.45 0.40 0.43 97
motorbike 0.42 0.36 0.39 106
person 0.37 0.35 0.36 818
pottedplant 0.22 0.23 0.22 106
sheep 0.47 0.71 0.57 65
sofa 0.18 0.20 0.19 102
train 0.58 0.85 0.69 109
tvmonitor 0.32 0.37 0.34 115
micro avg 0.43 0.43 0.43 3345
macro avg 0.44 0.46 0.44 3345
weighted avg 0.43 0.43 0.43 3345
=============== Labels ================
labels.txt looks like this:
aeroplane
bicycle
bird
boat
bottle
bus
car
cat
chair
cow
diningtable
dog
horse
motorbike
person
pottedplant
sheep
sofa
train
tvmonitor
background is not even included here.
=============== Config Files ================
Deepstream-app Config File
# PR. GIE CON RED DE CLASIFICACION EN VEZ DE DETECCION
# SOURCES -> MUX -> OSD -> PR. GIE -> TRCKER -> SINKS -> TILED
# Copyright (c) 2019-2020 NVIDIA Corporation. All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl
[tiled-display]
enable=1
rows=1
columns=1
width=1240
height=980
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0
[source0]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://<My Camera>
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device - Memory type Device
# (1): memtype_pinned - Memory type Host Pinned
# (2): memtype_unified - Memory type Unified
cudadec-memtype=0
[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=file://../streams/planes.mp4
#uri=file://../streams/sample_1080p_h265.mp4
#uri=file://../streams/sample_720p.mp4
num-sources=1
#drop-frame-interval=4
gpu-id=0
# (0): memtype_device - Memory type Device
# (1): memtype_pinned - Memory type Host Pinned
# (2): memtype_unified - Memory type Unified
cudadec-memtype=0
[sink0]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File
type=5
sync=1
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1
[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=out.mp4
source-id=0
[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=1
bitrate=3000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
# set below properties in case of RTSPStreaming
rtsp-port=8555
udp-port=5401
[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=1
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0
[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=8
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0
## If set to TRUE, system timestamp will be attached as ntp timestamp
## If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached
# attach-sys-ts-as-ntp=1
# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=../models/Controlflow_tlt_classification_full/final_model_full_labels.etlt.engine
batch-size=1
#Required by the app for OSD, not a plugin property
## 0=FP32, 1=INT8, 2=FP16 mode
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_secondary_controlflow_as_primary.txt
[tests]
file-loop=0
Model’s Config File
# Copyright (c) 2020 NVIDIA Corporation. All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
tlt-model-key=<My Key>
tlt-encoded-model=../models/Controlflow_tlt_classification_full/final_model_full_labels.etlt
labelfile-path=../models/Controlflow_tlt_classification/labels.txt
#int8-calib-file=../models/Controlflow_tlt/dashcamnet_int8.txt
model-engine-file=../models/Controlflow_tlt_classification_full/final_model_full_labels.etlt.engine
#input-dims=3;544;960;0
input-dims=3;224;224;0
uff-input-blob-name=input_1
batch-size= 1 #8 #3
process-mode=1
model-color-format=0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
network-type=1
#num-detected-classes=7
interval=2
gie-unique-id=1
output-blob-names=predictions/Softmax
classifier-threshold=0.2
Thanks!