Hello NVIDIA,
Scenario1:- In both GPU_0 and GPU_1, Iam loading the same head detection model. In this case memory is leaking.
Scenario2:- The same head model loading each separately as head1 and head2.
In GPU_0 , Iam loading head1 detection model. In GPU_1 , Iam loading head2 detection model. In this case also memory leak is happening.
Observation:
====>>That means, the moment I try to load same detection model in mutiple GPU’s, memory leak is happening.
=====>> If I load different detection models in multiple GPU’s, It is absolutely good and with single GPU also It is good.
Please check below for nvinfer config of head1 and head2 which Iam using for head detection model in both GPU’s.
nvinfer config head1
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
custom-network-config=/opt/nvidia/deepstream/deepstream/nvodin24/models/assets/normalHead1/yolov4-tiny.cfg
model-file=/opt/nvidia/deepstream/deepstream/nvodin24/models/assets/normalHead1/yolov4-tiny_best.weights
model-engine-file=/opt/nvidia/deepstream/deepstream/engine/normalHead1.engine
labelfile-path=/opt/nvidia/deepstream/deepstream/nvodin24/models/assets/normalHead1/head.txt
#int8-calib-file=/opt/nvidia/deepstream/deepstream/nvodin24/models/assets/normalHead1/calib.table
force-implicit-batch-dim=1
batch-size=1
network-mode=2
process-mode=1
model-color-format=0
num-detected-classes=1
interval=3
gie-unique-id=1
output-blob-names=num_detections;detection_boxes;detection_scores;detection_classes
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=/opt/nvidia/deepstream/deepstream/DeepStream-Yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet
[class-attrs-0]
pre-cluster-threshold=0.3
eps=0.2
group-threshold=1
nvinfer config head2
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
custom-network-config=/opt/nvidia/deepstream/deepstream/nvodin24/models/assets/normalHead2/yolov4-tiny.cfg
model-file=/opt/nvidia/deepstream/deepstream/nvodin24/models/assets/normalHead2/yolov4-tiny_best.weights
model-engine-file=/opt/nvidia/deepstream/deepstream/engine/normalHead2.engine
labelfile-path=/opt/nvidia/deepstream/deepstream/nvodin24/models/assets/normalHead2/head.txt
#int8-calib-file=/opt/nvidia/deepstream/deepstream/nvodin24/models/assets/normalHead2/calib.table
force-implicit-batch-dim=1
batch-size=1
network-mode=2
process-mode=1
model-color-format=0
num-detected-classes=1
interval=3
gie-unique-id=2
output-blob-names=num_detections;detection_boxes;detection_scores;detection_classes
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=/opt/nvidia/deepstream/deepstream/DeepStream-Yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet
[class-attrs-0]
pre-cluster-threshold=0.3
eps=0.2
group-threshold=1
In infer plugin element, Iam setting gpu-id property, But Iam not setting in nvinfer config accordingly. I hope it is not affecting??
Thanks.