Deepstream imagedata multi-stream sample got crashed after integration of reid(onnx) model

[Pritam]

Dear Team,

I am using deepstream-6.1 container and I have integrated peoplenet and reid(onnx based) model but I have observed that the code base get stuck after some time. Can you please help me to resolve the issue? I have added reid model as sgie and my observation is that when the crowd is more (Crowd means when more people are detected in frame, if 1-2 people in frame than code works fine) in the frame then this issue arise.

Below is my configuration for peopleNet model and reid model.

PeopleNet config:

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373


tlt-model-key=tlt_encode
tlt-encoded-model=../models/people_detection/resnet34_peoplenet_pruned.etlt
int8-calib-file=../models/people_detection/resnet34_peoplenet_int8.txt
model-engine-file= ../models/people_detection/resnet34_peoplenet_pruned.etlt_b2_gpu0_fp16.engine
labelfile-path=../models/people_detection/peoplenet_label.txt

input-dims=3;544;960;0
uff-input-blob-name=input_1

cluster-mode=1
batch-size=16
network-mode=2
process-mode=1
model-color-format=0
num-detected-classes=3
interval=0
gie-unique-id=1

#output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid
output-blob-names=output_bbox/BiasAdd;output_cov/Sigmoid
#output-blob-names=output_bbox;output_cov



[class-attrs-0]
detected-min-w=100
detected-min-h=200
pre-cluster-threshold=0.30
eps=0.7
minBoxes=1
group-threshold=1
#bbox-border-color=0;1;0;0.3

[class-attrs-1]
pre-cluster-threshold=1.0
eps=0.7
minBoxes=1
group-threshold=1
bbox-border-color=0;1;0;0.3

[class-attrs-2]
pre-cluster-threshold=1.0
eps=0.7
minBoxes=1
group-threshold=1
bbox-border-color=0;1;0;0.3

Reid Config (Sgie configuration)

[property]
gpu-id=0
net-scale-factor=0.01735207357279195
offsets=123.675;116.28;103.53
model-color-format=0

# Set the model file path
#onnx-file=../models/reid_osnet_ibn/model.onnx
# Set the model input layer name
#model-engine-file=../models/reid_osnet_ibn/model.onnx_b1_gpu0_fp32.engine
model-engine-file=/root/data/Pritam/Script/deep-person-reid/log/osnet_ibn_x1_0/model/model.pth.onnx.engine
input-dims=3;256;128;0
input-object-min-width=30
input-object-min-height=50
force-implicit-batch-dim=1

process-mode=2
output-blob-names=output
model-color-format=0
operate-on-gie-id=1
gie-unique-id=2

classifier-threshold=0
output-tensor-meta=1
maintain-aspect-ratio=0
## Extract embedding on "people" class only
operate-on-class-ids=0
network-type=100

Please suggest what is wrong here. Why the OSD output got freeze after some time and only **PERF: {} is visible.

Thanks

[Fiona.Chen]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)

[Pritam]

Sure below is the configuration.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.1
• JetPack Version (valid for Jetson only) NA
• TensorRT Version nv-tensorrt-repo-ubuntu1804-cuda11.1-trt7.2.2.3-ga-20201211 1-1
• NVIDIA GPU Driver Version (valid for GPU only) Nvidia-Driver-535

Problem Statement:

I am using deepstream-6.1 container and I have integrated peoplenet and reid(onnx based) model but I have observed that the code base get stuck after some time. Can you please help me to resolve the issue? I have added reid model as sgie and my observation is that when the crowd is more (Crowd means when more people are detected in frame, if 1-2 people in frame than code works fine) in the frame then this issue arise.

Below is my configuration for peopleNet model and reid model.

PeopleNet config:

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373


tlt-model-key=tlt_encode
tlt-encoded-model=../models/people_detection/resnet34_peoplenet_pruned.etlt
int8-calib-file=../models/people_detection/resnet34_peoplenet_int8.txt
model-engine-file= ../models/people_detection/resnet34_peoplenet_pruned.etlt_b2_gpu0_fp16.engine
labelfile-path=../models/people_detection/peoplenet_label.txt

input-dims=3;544;960;0
uff-input-blob-name=input_1

cluster-mode=1
batch-size=16
network-mode=2
process-mode=1
model-color-format=0
num-detected-classes=3
interval=0
gie-unique-id=1

#output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid
output-blob-names=output_bbox/BiasAdd;output_cov/Sigmoid
#output-blob-names=output_bbox;output_cov



[class-attrs-0]
detected-min-w=100
detected-min-h=200
pre-cluster-threshold=0.30
eps=0.7
minBoxes=1
group-threshold=1
#bbox-border-color=0;1;0;0.3

[class-attrs-1]
pre-cluster-threshold=1.0
eps=0.7
minBoxes=1
group-threshold=1
bbox-border-color=0;1;0;0.3

[class-attrs-2]
pre-cluster-threshold=1.0
eps=0.7
minBoxes=1
group-threshold=1
bbox-border-color=0;1;0;0.3

Reid Config (Sgie configuration)

[property]
gpu-id=0
net-scale-factor=0.01735207357279195
offsets=123.675;116.28;103.53
model-color-format=0

# Set the model file path
#onnx-file=../models/reid_osnet_ibn/model.onnx
# Set the model input layer name
#model-engine-file=../models/reid_osnet_ibn/model.onnx_b1_gpu0_fp32.engine
model-engine-file=/root/data/Pritam/Script/deep-person-reid/log/osnet_ibn_x1_0/model/model.pth.onnx.engine
input-dims=3;256;128;0
input-object-min-width=30
input-object-min-height=50
force-implicit-batch-dim=1

process-mode=2
output-blob-names=output
model-color-format=0
operate-on-gie-id=1
gie-unique-id=2

classifier-threshold=0
output-tensor-meta=1
maintain-aspect-ratio=0
## Extract embedding on "people" class only
operate-on-class-ids=0
network-type=100

Please suggest what is wrong here. Why the OSD output got freeze after some time and only **PERF: {} is visible.

Thanks

[Fiona.Chen]

DeepStream 6.1 is too old.

Please use the latest DeepStream 7.0 GA version.

With DeepStream 7.0 GA, you can add “tensor-meta-pool-size” in the SGIE nvinfer configuration file. The large value, the better according to the objects number detected by PGIE…

[Pritam]

Thanks @Fiona.Chen

I will try the same things over DS-7.0 and will check the issue is still there or not.

Below container I am going to use.

nvcr.io/nvidia/deepstream:7.0-samples-multiarch

Thanks.

[Pritam]

Hi @Fiona.Chen

Can you please explain the meaning of GA (DeepStream 7.0 GA) I mean I am not able to get it on DeepStream | NVIDIA NGC

Thanks.

[Fiona.Chen]

GA means “Global Available”. You can use the “nvcr.io/nvidia/deepstream:7.0-triton-multiarch” docker. Before you use it, please make sure the driver version should be * NVIDIA driver 535.161.08 Installation — DeepStream documentation 6.4 documentation

and please follow the instruction Docker Containers — DeepStream documentation 6.4 documentation

[Pritam]

Sure.

Thanks.

[Pritam]

Thanks @Fiona.Chen

I have tested the same thing on nvcr.io/nvidia/deepstream:7.0-samples-multiarch container and I am able to run the code on DS-7 without any issue.

Thanks a lot.