Ok, I think you may need to check the properties in sgie config file, especially the properties marked with bold below.
The meaning of these properties can refer to Gst-nvinfer — DeepStream 6.1.1 Release documentation
• config_infer_secondary_mcu.txt
[property]
net-scale-factor=1
offsets=…
model-engine-file=models/resnet18_model.onnx_b1_gpu0_fp16.engine
labelfile-path=models/resnet18_labels.txt
onnx-file=models/resnet18_model.onnx
#force-implicit-batch-dim=1
batch-size=1
model-color-format=0
process-mode=2
infer-dims=3;224;224
#0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
maintain-aspect-ratio=…
is-classifier=1
classifier-async-mode=1
classifier-threshold=0.2
input-object-min-width=128
input-object-min-height=128
#scaling-filter=0
#scaling-compute-hw=0
operate-on-gie-id=1 // equals to GIE id of PGIE