Pipeline has a detector as primary gie and another detector as secondary gie

Hi. I am using this environment:

• Hardware Platform (Jetson / GPU) : GPU
• DeepStream Version : 4.0.1
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) : 418.43

Here are some snippets of the pipeline:

Primary gie:

Secondary gie (0 as detector and 1 as classifier):

Configuration Snippet:

[primary-gie]
  enable = 1
  gie-unique-id = 1
  batch-size = 1
  config-file = 7-class.conf

[secondary-gie0]
  enable = 1
  gie-unique-id = 4
  batch-size = 1
  config-file = person-parts.conf
  operate-on-gie-id = 1

[tracker]
  enable = 1
  tracker-width = 960
  tracker-height = 540
  ll-lib-file = /opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_nvdcf.so

What I want to achieve:

{
  "frame_id" : 2,
  "detections" : [
      {
      "bounding_box" : [
        0,
        324,
        56,
        466
      ],
      "class" : "person",
      "confidence" : 0.9,
      "track_id" : 9,
      "attributes" : {
        "subclass" : [
          "upper", "lower"
        ],
        "bbox" : [
          {"lefttopx": 0, "lefttopy": 300 ,"rightbottomx": 50, "rightbottomy":400},
          {"lefttopx": 10, "lefttopy": 333 ,"rightbottomx": 60, "rightbottomy":300}
        ],
        "confidence" : [
          0.54759711027145386, 0.64759711027145386
        ]
      }
    }
  ]
}

What I am getting from the current APIs:

{
  "frame_id" : 2,
  "detections" : [
    {
      "bounding_box" : [
        385,
        328,
        385,
        330
      ],
      "class" : "person",
      "confidence" : 0.0,
      "track_id" : -1,
      "attributes" : {
        "subclass" : [
          "head"
        ],
        "confidence" : [
        ],
        "colour" : [
        ],
        "score" : [
        ],
        "bbox" : [
          {
            "topleftx" : 385,
            "toplefty" : 328,
            "rightbottomx" : 385,
            "rightbottomy" : 330
          }
        ]
      }
    },
 ]
}

The field’s data is accessed from NvDsObjectMeta and NvDsFrameMeta struct from nvdsmeta.h.

As you can see the track_id is not giving a valid tracking id. Since the subclass is head, the bbox is same data as bounding_box.

So I am just wondering if there is an existing support for a detector as a secondary gie? If there is, probably point me in the correct direction or resource or possible work around?

If there isn’t an existing support, may I know when the support for detector as a secondary gie will be made available?

Thanks.

Have you checked deepstream_reference_apps/back-to-back-detectors at master · NVIDIA-AI-IOT/deepstream_reference_apps · GitHub

BTW, we cannot see your first screenshot clearly.