About update object_meta in attach_metadata_detector in gst-nvinfer plugin

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hello !
I have a problem, I am using a model that detects license plates, returns bounding boxes and landmarks. This model infers on objects detected from the Primary model. I want to update landmarks in obj_user_meta_list in object_meta and I do it like this:

if (obj.label && !nvinfer->alignment_type){
      g_strlcpy (obj_meta->obj_label, obj.label, MAX_LABEL_SIZE);
      text_params.display_text = g_strdup (obj.label);
    } else if (obj.label && nvinfer->alignment_type){
      std::vector<std::string> landmarks;
      std::stringstream ss(obj.label);
      std::string ele;

      while (std::getline(ss, ele, ',')) {
        ele.erase(std::remove_if(ele.begin(), ele.end(), isspace), ele.end());
        landmarks.push_back(ele);
      }
      g_strlcpy (obj_meta->obj_label, landmarks[0].c_str(), MAX_LABEL_SIZE);
      text_params.display_text = g_strdup (landmarks[0].c_str());

      // Update to user_meta
      NvDsUserMeta *user_meta = NULL;
      NvDsMetaType user_meta_type = NVDS_USER_OBJECT_META_EXAMPLE;

      NvDsBatchMeta *obj_batch_meta = obj_meta->base_meta.batch_meta;
      user_meta = nvds_acquire_user_meta_from_pool(obj_batch_meta);

      user_meta->user_meta_data = (void *)set_metadata_ptr(landmarks, nvinfer, parent_obj_meta, frame);
      user_meta->base_meta.meta_type = user_meta_type;
      user_meta->base_meta.copy_func = (NvDsMetaCopyFunc)copy_user_meta;
      user_meta->base_meta.release_func = (NvDsMetaReleaseFunc)release_user_meta;

      nvds_add_user_meta_to_obj(obj_meta, user_meta);
    }

If I check obj_user_meta_list of object _meta right in attach_metadata_detector, it is updated. But if I check obj_user_meta_list of object_meta in gst_nvinfer_process_objects or other function of gstnvinfer.cpp it is Null.

if (nvinfer->alignment_type){
        NvDsMetaList * l_user_meta = NULL;
        NvDsUserMeta *user_meta = NULL;
        gint *user_meta_data = NULL;
        for (l_user_meta = object_meta->obj_user_meta_list; l_user_meta != NULL; l_user_meta = l_user_meta->next) {
          user_meta = (NvDsUserMeta *) (l_user_meta->data);
          user_meta_data = (gint *)user_meta->user_meta_data;
          for (unsigned int i=0; i < 10; i++) {
            std::cout << "CHECK: " << user_meta_data[i] << std::endl;
          }
        }
      }

Please help me

3

Why can’t you add that in the attach_metadata_detector? gst_nvinfer_process_objects cannot get the result after inference. It’s used to send the raw data.

4

Oh sorry about my misunderstanding, so where can I check obj_user_meta_list of object _meta after I
updated in attach_metadata_detector.

And one more thing I don’t quite understand. When I check the label of object_meta in gst_nvinfer_process_objects, it still shows the label of the object after the first detect model, so can you explain more about what you said: " gst_nvinfer_process_objects cannot get the result after inference. It’s used to send the raw data "?

5

In theory, we don’t recommend that you change the plugin code directly, but add these code in your app. You can get this user meta in the probe function in your app code, like our demo code sources\apps\sample_apps\deepstream-image-meta-test.

6

My goal is that after updating the landmarks for the license plate object_meta, I can use it to align the license plate image before putting it into the license plate recognition model. Therefore, I need to access the license plate object_meta after it is updated to get the landmarks and align the image in the plugin. Please help me !

7

Could you attach your whole pipeline and illustrate that?

8

This is my pipeline:
1 → Input
2 → PGIE_1 detector detect vehicles (like car, bus, motorbike, …)
3 → PGIE_2 detector detect license plates on crop images of objects were detected from PGIE_1. The output of
this model are bounding boxes and landmarks of license plates.
4 → SGIE_2 classifier recognize the character of license plates.

9

So you get the landmarks through the PGIE_2. How do you use the landmarks data next, for SGIE_2 or others?

10

I updated the landmarks for the license plate object_meta in attach_metadata_detector. Currently I have not used those landmarks for SGIE_2 but I want to use them to align the license plate images then these aligned images are inputs of SGIE_2 to recognize the character of license plates.

More specifically, I want to write code in the nvinfer plugin functions to retrieve updated landmarks and use them to align the input image of SGIE_2. Can I write this code in any of the available functions of nvinfer plugin?

11

In theory, you add that in the PGIE_1 plugin, you can get that from the gst_nvinfer_submit_input_buffer in the PGIE_2 or SGIE_2 plugin. Could you print the name of the plugin to check which plugin is printed the infomation?
You can print that with gst_element_get_name(<plugin>) API.

12

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.