Deepstream5.0.1+yolov5+resnet50 No output classifier result

yolov5l is used as the primary-gie detector and Resnet50 is used as the secondary0 and secondary1 classifier model,but only the detection results of yolov5l can be displayed.

**• Hardware Platform (Jetson / GPU) Tesla T4
**• DeepStream Version 5.0.1
**• JetPack Version (valid for Jetson only)
**• TensorRT Version 7.0.0
**• NVIDIA GPU Driver Version (valid for GPU only) 440.118.02

Here is my config file:
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=0
rows=2
columns=3
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
#uri=rtsp://admin:a1234567@192.168.10.216:554/Streaming/Channels/101
uri=file:/home/user/video/madiliang/helmet1.mp4
#uri=file:/home/nvidia/Documents/5-Materials/Videos/0825.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

drop-frame-interval=0
cudadec-memtype=0
rtsp-reconnect-interval-sec=30
latency=200
select-rtp-protocol=4

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=4
uri=rtsp://admin:admin123@192.168.10.80:554/Streaming/Channels/101
#uri=file:/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_1080p_h264.mp4
#uri=file:/home/nvidia/Documents/5-Materials/Videos/0825.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0
rtsp-reconnect-interval-sec=30
latency=200
select-rtp-protocol=4
[source2]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=4
uri=rtsp://admin:a1234567@192.168.10.11:554/Streaming/Channels/101
#uri=file:/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_1080p_h264.mp4
#uri=file:/home/nvidia/Documents/5-Materials/Videos/0825.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0
rtsp-reconnect-interval-sec=30
latency=200
select-rtp-protocol=4

[source3]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=4
uri=rtsp://admin:admin123@192.168.10.198:554/Streaming/Channels/101
#uri=file:/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_1080p_h264.mp4
#uri=file:/home/nvidia/Documents/5-Materials/Videos/0825.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0
rtsp-reconnect-interval-sec=30
latency=200
select-rtp-protocol=4

[source4]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=4
uri=rtsp://admin:admin123@192.168.10.138:554/Streaming/Channels/101
#uri=file:/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_1080p_h264.mp4
#uri=file:/home/nvidia/Documents/5-Materials/Videos/0825.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0
rtsp-reconnect-interval-sec=30
latency=200
select-rtp-protocol=4

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
output-file=yolov4.mp4
bitrate=2000000
rtsp-port=11240
udp-port=21240
#link-to-demux=1
#qos=1
iframeinterval=25
profile=2
#udp-buffer-size=100000

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0
source-id=1
gpu-id=0
nvbuf-memory-type=0
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#enc-type=1
width=1280
height=720
output-file=yolov5.mp4
bitrate=2000000
rtsp-port=12241
udp-port=22241
#link-to-demux=1

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0
source-id=2
gpu-id=0
nvbuf-memory-type=0
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
output-file=yolov4.mp4
bitrate=2000000
rtsp-port=13242
udp-port=23242
#enc-type=1
#link-to-demux=1

[sink3]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0
source-id=3
gpu-id=0
nvbuf-memory-type=0
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
output-file=yolov4.mp4
bitrate=2000000
rtsp-port=14243
udp-port=24243
#enc-type=1
#link-to-demux=1

[sink4]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0
source-id=4
gpu-id=0
nvbuf-memory-type=0
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
output-file=yolov4.mp4
bitrate=2000000
rtsp-port=15244
udp-port=25244
#enc-type=1
#link-to-demux=1

[osd]
enable=1
gpu-id=0
border-width=1
text-size=12
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=5
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000

Set muxer output width and height

width=1280
height=720
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]
enable=1
gpu-id=0
model-engine-file=…/engine/yolov5l_helmet.engine
labelfile-path=…/label/helmet_labels.txt
batch-size=5
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=5
gie-unique-id=1
#operate-on-gie-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV5_Helmet.txt

[secondary-gie0]
enable=1
gpu-id=0
model-engine-file=…/engine/resnet50_secondary0_helmet.engine
labelfile-path=…/label/helmet_secondary0_labels.txt
gie-unique-id=2
operate-on-gie-id=1
config-file=config_infer_secondary0_resnet50_Helmet.txt
operate-on-class-ids=0;1
batch-size=5

[secondary-gie1]
enable=1
gpu-id=0
model-engine-file=…/engine/resnet50_secondary1_helmet.engine
labelfile-path=…/label/helmet_secondary1_labels.txt
gie-unique-id=3
operate-on-gie-id=2
config-file=config_infer_secondary1_resnet50_Helmet.txt
operate-on-class-ids=0;1;2
batch-size=5

[tracker]
enable=1
gpu-id=0
tracker-width=512
tracker-height=320
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so

[tests]
file-loop=1

Here is my secondary0-gie confi file
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
model-engine-file=…/engine/resnet50_secondary0_helmet.engine
labelfile-path=…/label/helmet_secondary0_labels.txt
process-mode=2

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=3
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0;1
batch-size=5
#0: Detector,1: Classifier,2: Segmentation
network-type=1
#is-classifier=1
#output-blob-names=predictions/Softmax

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

cluster-mode=2
maintain-aspect-ratio=1
#scaling-filter=0
#scaling-compute-hw=0

[class-attrs-all]
nms-iou-threshold=0.5
pre-cluster-threshold=0.4

Here is my secondary1-gie confi file
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
model-engine-file=…/engine/resnet50_secondary1_helmet.engine
labelfile-path=…/label/helmet_secondary1_labels.txt
process-mode=2

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=2
gie-unique-id=3
operate-on-gie-id=2
operate-on-class-ids=0;1;2
batch-size=5
network-type=1
#is-classifier=1
#output-blob-names=predictions/Softmax

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

cluster-mode=2
maintain-aspect-ratio=1
#scaling-filter=0
#scaling-compute-hw=0

[class-attrs-all]
nms-iou-threshold=0.5
pre-cluster-threshold=0.4

You SGIE configurations are not correct. We have samples of secondary GIE configuration in the sample code. Please refer to /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_secondary_carmake.txt or /opt/nvidia/deepstream/deepstream/samples/configs/tlt_pretrained_models/config_infer_secondary_vehiclemakenet.txt

The meaning of fields are explained in Gst-nvinfer — DeepStream 5.1 Release documentation

thanks your reply.I can only output the inference result of sgie0, and the label will display all the results of sig0. This is not what I expected. I cannot get the inference result of sgie1. The following is my latest configuration and result picture .
Current Primary config file:
[primary-gie]
enable=1
gpu-id=0
model-engine-file=…/engine/yolov5l_helmet.engine
labelfile-path=…/label/helmet_labels.txt
batch-size=5
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=10
gie-unique-id=1
#operate-on-gie-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV5_Helmet.txt

[secondary-gie0]
enable=1
gpu-id=0
model-engine-file=…/engine/resnet50_secondary0_helmet.engine
labelfile-path=…/label/helmet_secondary0_labels.txt
gie-unique-id=2
operate-on-gie-id=1
config-file=config_infer_secondary0_resnet50_Helmet.txt
operate-on-class-ids=0
batch-size=5

[secondary-gie1]
enable=1
gpu-id=0
model-engine-file=…/engine/resnet50_secondary1_helmet.engine
labelfile-path=…/label/helmet_secondary1_labels.txt
gie-unique-id=3
operate-on-gie-id=1
config-file=config_infer_secondary1_resnet50_Helmet.txt
operate-on-class-ids=0
batch-size=5

SGIE0 config file:

[property]
gpu-id=0
net-scale-factor=1
offsets=124;117;104
#net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
model-engine-file=…/engine/resnet50_secondary0_helmet.engine
labelfile-path=…/label/helmet_secondary0_labels.txt
#1:pgie 2:sgie
process-mode=2

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=3
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0
batch-size=5
#0: Detector,1: Classifier,2: Segmentation
network-type=1
#is-classifier=1
input-object-min-width=64
input-object-min-height=64
output-blob-names=output1;output2
classifier-async-mode=1
classifier-threshold=0.51

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

cluster-mode=2
maintain-aspect-ratio=1
#scaling-filter=0
#scaling-compute-hw=0

[class-attrs-all]
nms-iou-threshold=0.5
pre-cluster-threshold=0.4
eps=0.2
group-threshold=1

SGIE1 config file:
[property]
gpu-id=0
#net-scale-factor=0.0039215697906911373
net-scale-factor=1
offsets=124;117;104
#0=RGB, 1=BGR
model-color-format=0
model-engine-file=…/engine/resnet50_secondary1_helmet.engine
labelfile-path=…/label/helmet_secondary1_labels.txt
process-mode=2

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=2
gie-unique-id=3
operate-on-gie-id=2
operate-on-class-ids=0
batch-size=5
network-type=1
#is-classifier=1
input-object-min-width=64
input-object-min-height=64
output-blob-names=output1;output2
classifier-async-mode=1
classifier-threshold=0.51

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

cluster-mode=2
maintain-aspect-ratio=1
#scaling-filter=0
#scaling-compute-hw=0

[class-attrs-all]
nms-iou-threshold=0.5
pre-cluster-threshold=0.4
eps=0.2
group-threshold=1

GIE workflow image:

test result image:

What are your sgie1 labels?

sgie1 labels represent the value of the label corresponding to secondary-gie1, which is rightBelt_helmet and wrongBelt_helmet. I have some new questions. I will have different results by changing the value of the net-scale-factor field under the property group. The label will display all categories of objects in sgie0 or all categories of objects in sgie1. This is abnormal. I need to implement NvDsInferParseCustomResnet50 function by myself?