NvDsObjectMeta parent is None on Secondary GIE

[Levi_Pereira]

• Hardware Platform (Jetson / GPU)
GPU
• DeepStream Version
docker: nvcr.io/nvidia/deepstream:6.1.1-triton
• TensorRT Version
docker: nvcr.io/nvidia/deepstream:6.1.1-triton
• NVIDIA GPU Driver Version (valid for GPU only)
Driver Version: 520.56.06 CUDA Version: 11.8
• Issue Type( questions, new requirements, bugs)
Secondary GIE does not have Parent data. I’m trying associate the car that was detected (PGIE) with the detected LPD (SGIE) using YOLOv7 model
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Python APP code snippet

pgie = Gst.ElementFactory.make("nvinferserver", "primary-inference")
sgie1 = Gst.ElementFactory.make("nvinferserver", "secondary1-nvinference-engine")

pgie.set_property("config-file-path", "/apps/configs/config_infer_primary_yolov7_triton-server.txt")
sgie1.set_property("config-file-path", "/apps/configs/config_infer_primary_yolov7_lpd_triton-server.txt")

pgie.set_property("process-mode", 1)
sgie1.set_property("process-mode", 2)

streammux.link(queue1)
queue1.link(pgie)
pgie.link(queue2)
queue2.link(tracker)
tracker.link(queue3)
queue3.link(nvanalytics)
nvanalytics.link(queue4)
queue4.link(sgie1)



## function
try: 
	# Note that l_obj.data needs a cast to pyds.NvDsObjectMeta
	# The casting is done by pyds.NvDsObjectMeta.cast()
	obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
except StopIteration:
	break

ndom_confidence = obj_meta.confidence
ndom_unique_component_id = obj_meta.unique_component_id
ndom_parent = obj_meta.parent  
ndom_object_id = obj_meta.object_id
ndom_class_id = obj_meta.class_id
print(ndom_object_id,ndom_class_id, ndom_unique_component_id,ndom_parent) 

Triton Primary GIE

infer_config {
  unique_id: 1
  gpu_ids: [0]
  max_batch_size: 6 
  
  backend {
    triton {
      model_name: "yolov4-640"
      version: -1
      grpc {
        url: "0.0.0.0:8001"
      }
    }
  }
.
.
.
input_control {
  process_mode : PROCESS_MODE_FULL_FRAME
  interval : 0
}

Triton Secondary GIE

infer_config {
  unique_id: 2
  max_batch_size: 30 

  backend {
    triton {
      model_name: "yolo-lpr"
      version: -1
      grpc {
        url: "0.0.0.0:8001"
      }
    }
  }
.
.
.

input_control {
  process_mode: PROCESS_MODE_CLIP_OBJECTS
  operate_on_gie_id: 1
  operate_on_class_ids: [0, 1, 2, 3, 4]
  interval: 0
  async_mode: false
  object_control {
    bbox_filter {
      min_width: 128
      min_height: 128
    }
  }
}

Output of Parent is None for Secondary GIE

GIE_ID  > OBJ_TRACKER_ID > CLASS_ID  > Parent
2 18446744073709551615 0 None
1 14451367522747809832 0 None
1 14451367522747809833 1 None
2 18446744073709551615 0 None
2 18446744073709551615 0 None
1 14451367522747809832 0 None
1 14451367522747809833 1 None
2 18446744073709551615 0 None

Also, I don’t understand why Tracker is operating on Secondary GIE.

[fanzh]

please refer to Can't get the object_id of sgie parent - #10 by MingGatsby.

[Levi_Pereira]

It worked, but I have strange behaviour.

I don’t know if this behaviour is normal but works only if add probe function on nvanalytics src, but if add probe function on filter1 src parent is None.

My Pipeline

    streammux.link(pgie)
    pgie.link(tracker)
    tracker.link(sgie1)
    sgie1.link(nvanalytics)
    nvanalytics.link(nvvidconv1)
    # Add nvvidconv1 and filter1 to convert the frames to RGBA
    # which is easier to work with in Python.
    nvvidconv1.link(filter1)
    filter1.link(queue7)

 nvanalytics_src_pad=nvanalytics.get_static_pad("src")  ## return parent with data
 nvanalytics_src_pad=filter1.get_static_pad("src")  ## return parent with None

[fanzh]

As we know, capsfilter is gstreamer opensource plugin, it will not modify the metadata.
please narrow down this issue by adding more probe.

[Levi_Pereira]

This is confusing because only NvDsObjectMeta.parent data is missing, all other data are ok in capsfilter src.
But ok … will add more probe