Hello,
having now created a model for age/gender based on classification and also generated the engine on Jetson, I have used the poeplenet deepstream model and added a secondary-gie0 into the config.
This works for another model which i have trained and loaded to Jetson e.g. detectnet, but when I replace the secondary-gie0 config from detectnet model to the generated classification model and when i start deepstream, the secondary classification model gets loaded but it is not somehow visible (I am running the deepstream with a webcam as input source)
–>What I would expect is, that peoplenet recognizes the face and with the face as input, the classification (age-gender) should recognize the related class and show it as bbox with the class-name.
–>But how to configure this?
below my log from jetson and configs:
log
jetson@nano01:~/tlt-racket/nano-jetson/tlt-configs$ deepstream-app -c my_deepstream_app_source1_peoplenet.txt
2020-09-19 10:14:22.966787: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2(deepstream-app:10238): GStreamer-CRITICAL **: 10:14:23.790: passed ‘0’ as denominator for `GstFraction’
(deepstream-app:10238): GStreamer-CRITICAL **: 10:14:23.790: passed ‘0’ as denominator for `GstFraction’
Warning: ‘input-dims’ parameter has been deprecated. Use ‘infer-dims’ instead.
Warning: ‘input-dims’ parameter has been deprecated. Use ‘infer-dims’ instead.Using winsys: x11
0:00:06.902412681 10238 0x2685430 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 4]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1701> [UID = 4]: deserialized trt engine from :/home/jetson/tlt-racket/nano-jetson/tlt-engines/classification_age_gender.engine
INFO: [Implicit Engine Info]: layers num: 2
0 INPUT kFLOAT input_1 3x224x224
1 OUTPUT kFLOAT predictions/Softmax 37x1x10:00:06.902697167 10238 0x2685430 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 4]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1805> [UID = 4]: Use deserialized engine model: /home/jetson/tlt-racket/nano-jetson/tlt-engines/classification_age_gender.engine
0:00:07.090150585 10238 0x2685430 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary_gie_0> [UID 4]: Load new model:/home/jetson/tlt-racket/nano-jetson/tlt-configs/my_config_infer_secondary_age_gender.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
0:00:07.610666230 10238 0x2685430 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1701> [UID = 1]: deserialized trt engine from :/home/jetson/tlt-racket/nano-jetson/tlt-models/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x544x960
1 OUTPUT kFLOAT output_bbox/BiasAdd 12x34x60
2 OUTPUT kFLOAT output_cov/Sigmoid 3x34x600:00:07.610824932 10238 0x2685430 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1805> [UID = 1]: Use deserialized engine model: /home/jetson/tlt-racket/nano-jetson/tlt-models/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine
0:00:07.623889711 10238 0x2685430 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/home/jetson/tlt-racket/nano-jetson/tlt-configs/config_infer_primary_peoplenet.txt sucessfullyRuntime commands:
h: Print this help
q: Quitp: Pause
r: ResumeNOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.**PERF: FPS 0 (Avg)
**PERF: 0.00 (0.00)
** INFO: <bus_callback:181>: Pipeline ready** INFO: <bus_callback:167>: Pipeline running
**PERF: 0.00 (0.00)
KLT Tracker Init
**PERF: 5.73 (4.03)
**PERF: 5.04 (4.67)
**PERF: 4.96 (4.80)
**PERF: 4.98 (4.86)
**PERF: 4.98 (4.89)
**PERF: 4.97 (4.91)
**PERF: 4.98 (4.92)
**PERF: 4.98 (4.93)
**PERF: 4.98 (4.94)
**PERF: 4.98 (4.95)
**PERF: 4.98 (4.95)
**PERF: 4.99 (4.95)
**PERF: 4.98 (4.96)
Quitting
App run successful
my_deepstream_app_source1_peoplenet
################################################################################
Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the “Software”),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
################################################################################
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1[tiled-display]
enable=1
rows=1
columns=1
width=1920
height=1080
gpu-id=0[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=1
num-sources=1#uri=file://…/…/streams/sample_1080p_h265.mp4
gpu-id=0
camera-width=1920
camera-height=1080
camera-fps-n=24
camera-v4l2-dev-node=0
[streammux]
gpu-id=0
batch-size=1
batched-push-timeout=40000Set muxer output width and height
width=1920
height=1080[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0[osd]
enable=1
gpu-id=0
border-width=3
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Arial[primary-gie]
enable=1
gpu-id=0Modify as necessary
model-engine-file=…/…/models/tlt_pretrained_models/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
config-file=config_infer_primary_peoplenet.txt[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
codec=1
encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=out.mp4
source-id=0[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=4
#1=h264 2=h265
codec=1
encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400[tracker]
enable=1
tracker-width=640
tracker-height=384
#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_iou.so
#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_nvdcf.so
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
#ll-config-file required for DCF/IOU only
ll-config-file=…/deepstream-app/tracker_config.yml
#ll-config-file=iou_config.txt
gpu-id=0
#enable-batch-process applicable to DCF only
enable-batch-process=1[secondary-gie0]
enable=1
gpu-id=0
batch-size=4
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=my_config_infer_secondary_age_gender.txt[tests]
file-loop=0
my_config_infer_secondary_age_gender
################################################################################
Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the “Software”),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
################################################################################
[property]
gpu-id=0
net-scale-factor=1
offsets=124;117;104
tlt-model-key=tlt_encode
tlt-encoded-model=…/tlt-models/classification/export/age_gender_final_model.etlt
labelfile-path=…/tlt-labels/classification_age_gender.txt
model-engine-file=…/tlt-engines/classification_age_gender.engine
input-dims=3;224;224;0
uff-input-blob-name=input_1
batch-size=4
process-mode=2
model-color-format=0network-mode=2
network-type=1
num-detected-classes=37
interval=0
gie-unique-id=1
output-blob-names=predictions/Softmax
classifier-threshold=0.2