How to correlate between secondary and primary gie object meta

• DeepStream Version 6.0

I am using back to back detector. Below are the config files related to this deepstream application.

deepstream_app_config.txt

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=4
uri=rtsp://164.52.15.153:8556/test
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
type=1
sync=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=8
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[secondary-gie0]
enable=1
gpu-id=0
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary.txt

[tracker]
enable=1
gpu-id=0
tracker-width=960
tracker-height=544
ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
ll-config-file=/home/Development/projects/app/build/analytic_server/server_data/oconfigs/config_tracker_NvDCF_perf.yml
enable-batch-process=0

[tests]
file-loop=0

config_infer_primary.txt (Primary Gie)

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
custom-network-config=yolov4-tiny.cfg
model-file=yolov4-tiny.weights
labelfile-path=labels.txt
batch-size=8
network-mode=0
network-type=0
num-detected-classes=80
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=0
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.6
pre-cluster-threshold=0.25

config_infer_secondary.txt (Secondary Gie)

[property]
gpu-id=0
process-mode=2
net-scale-factor=0.0039215697906911373
model-file=fd_lpd.caffemodel
proto-file=fd_lpd.prototxt
labelfile-path=fp_lpd_labels.txt

force-implicit-batch-dim=1
batch-size=1
network-mode=0
num-detected-classes=2
interval=0
gie-unique-id=2
output-blob-names=output_bbox;output_cov
input-object-min-width=64
input-object-min-height=64
maintain-aspect-ratio=1
# Person has class-id 2 for the primary detector. This ensures that this secondary
# detector only works on persons.
#operate-on-class-ids=0
cluster-mode=2
network-type=0

[class-attrs-all]
pre-cluster-threshold=0.2
topk=20
nms-iou-threshold=0.5

When I am trying to process object metadata. Below are the things that I observed.

  1. Objects which have unique_component_id equal to 1 (primary gie), have unique object_id. Using object_id I can distinguish between detections by primary gie.
  2. Objects which have unique_component_id equal to 2 (secondary gie), have object_id -1 and the parent is also set to a null pointer.

So, my query is how can I correlate between secondary and primary gie object meta (If both gie’s are detectors).

Edit 1. Added network-type=0 to both gie config.

@bcao Can you help me with this?

From your config only the first one acts as detector (process-mode=1)

Thank you for your response @dmitry.skorokhod. I update the process-mode to 1 in secondary config but problem still persist.

deepstream-app does not support back-to-back, please refer to deepstream_reference_apps/back-to-back-detectors at master · NVIDIA-AI-IOT/deepstream_reference_apps · GitHub

@Fiona.Chen I used back-to-back detectors in deepstream 5.0. During deepstream 5.0 parent of secondary gie object meta was not null.

@dmitry.skorokhod According to this Gst-nvinfer — DeepStream 6.1.1 Release documentation process-mode should be 2 for secondary config.

Yes. You need to set “process-mode=2” for SGIE

@Fiona.Chen process-mode is 2 for secondary config.

The problem is that means your SGIE works on objects detected by PGIE (i.e works as a classifier rather than second detector) if I understand it right after reading the link above.
Is that what you are trying to achieve? Based on this statement

it’s not.

@dmitry.skorokhod Got your point. But when I have set process-mode to 2 and network-type to 0, then secondary gie should work as a secondary detector. Right?
The problem with secondary gie is that it is running as a primary detector instead of a secondary detector even when process-mode is set to 2.
I just want to know which secondary gie’s object meta belongs to which primary gie’s object meta.

Your PGIE has 80 classes. So you need to config “operate-on-gie-id” and “operate-on-class-ids” in your SGIE config file. Gst-nvinfer — DeepStream 6.1.1 Release documentation

Please refer to NVIDIA-AI-IOT/deepstream_lpr_app: Sample app code for LPR deployment on DeepStream (github.com)

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.