The obj_meta->parent is empty (=NULL) on sgie object on downstream elements

[pramod4]

PC: RTX 2070 Super
Deepstream: 6.2
Driver Version: 525.105.17
Docker image: deepstream:6.2-devel

Hi, I have a deepstream pipeline in c++, with pgie, sgie and a custom dsexample plugin (dsexample plugin similiar to deepstream-opencv-test sample app). In my use case, I do my processing inside the custom dsexample plugin. Thus when I go through the sgie objects obj_meta->parent , it returns NULL. The process-mode is configured correctly for pgie and sgie.

I have found similiar posts that suggest this is a deepstream 6.0 problem. It seems parent is accessible from the src pad of sgie, but not the downstream elements. since my dsexample plugin is after sgie.

In deepstream-6.0‘s deepstream_lpr_app, obj_meta->parent is empty (=NULL)

The quick fix described here was to probe in sgie src pad, however I would like to access the parent information from inside my custom plugin. Considering this was not a problem in DS 5.0 as suggested in the above post, I believe this is DS 6.0 related.

[yuweiw]

Could you attach your whole piepline?

[pramod4]

Sure, I’ll try to attach code snippets of the pipeline.

Initialize elements

  streammux = gst_element_factory_make ("nvstreammux", "stream-muxer"); 
  primary_detector = gst_element_factory_make ("nvinfer", "primary-nvinference-engine");
  secondary_detector = gst_element_factory_make ("nvinfer", "secondary-nvinference-engine"); 
  tracker = gst_element_factory_make ("nvtracker", "nvtracker"); 
  fake_sink = gst_element_factory_make ("fakesink", "fake-renderer");    
  nvvidconv_analytic = gst_element_factory_make ("nvvideoconvert", "nvvideo-converter");
  caps_filter_analytic = gst_element_factory_make ("capsfilter", "analytic-caps-filter");
  caps_analytic = gst_caps_from_string("video/x-raw(memory:NVMM), format=RGBA");
  analytic = gst_element_factory_make ("dsexample", "example-plugin");
  queue1 = gst_element_factory_make ("queue", "queue1");
  queue2 = gst_element_factory_make ("queue", "queue2");
  queue3 = gst_element_factory_make ("queue", "queue3");

Configure elements

g_object_set (G_OBJECT (streammux), "width", MUXER_OUTPUT_WIDTH, "height",
      MUXER_OUTPUT_HEIGHT, "batch-size", num_sources,
      "batched-push-timeout", MUXER_BATCH_TIMEOUT_USEC, NULL);

  g_object_set (G_OBJECT (primary_detector), "config-file-path", pgie_config_path,
          "unique-id", 1, "interval", 0, NULL);

  g_object_set (G_OBJECT (secondary_detector), "config-file-path", sgie_config_path,
        "unique-id", 2, "interval", 0, NULL);
  g_object_set (G_OBJECT (nvvidconv_analytic), "nvbuf-memory-type", 3, NULL);
  g_object_set (G_OBJECT (caps_filter_analytic), "caps", caps_analytic, NULL);
  g_object_set (G_OBJECT (analytic), "full-frame", FALSE, NULL);
  g_object_set (G_OBJECT (analytic), "blur-objects", TRUE, NULL);  
  set_tracker_properties(tracker, tracker_config_path);

Next I create source elements (uridecodebin) and link to streammux.
Finally add everything and link as shown below.

gst_bin_add_many (GST_BIN (pipeline), primary_detector, tracker, secondary_detector, nvvidconv_analytic, caps_filter_analytic, analytic, fake_sink, queue1, queue2, queue3, NULL);

gst_element_link_many (streammux, queue1, primary_detector, queue2, tracker, queue3, secondary_detector, nvvidconv_analytic, caps_filter_analytic, analytic, fake_sink, NULL)

I am trying to access meta data from the gst_dsexample_transform_ip function in gstdsexample.cpp.
When I loop through obj meta and filter out objects with sgie ID, the parent of it is always NULL.
Everything else works fine, pgie and sgie works fine and the detections are correct. (sgie has correctly operated on objects inside pgie object, as i configured in their respective configs).

[yuweiw]

Some DeepStream plugin , like nvvideoconvert will generate new NvDsMeta for the output, the parent is not reserved in the new NvDsMeta.

[pramod4]

Thank you for the clarification.

Right now I have found a workaround for this problem, where I would probe the sgie src and filter out sgie elements and then make the object_id of the sgie element as the object_id of its parent, as below.

          if (obj_meta->unique_component_id == 2) {
            obj_meta->object_id = obj_meta->parent->object_id;
          }

Now in my custom plugin when I loop through sgie elements, and the object_id, is actually the object_id of its parent, thus this would be a way of linking the pgie elements and sgie elements.