Sgie's detecting properly but classifier_meta_list is None

Setup:
• Hardware Platform: Jetson Xavier NX
• DeepStream Version: 5.0.1
• Language: Python

I built a “mask/not mask” sgie model to run on traditional pgie-4 classes model (To run only on class “person”).
Although my model is predicting poorly, I can see the wrong predictions on screen, however I can’t get the metadada: classifier_meta_list is always None.

I’m trying to get the metadata in classifier_meta_list by:

   while l_obj is not None:
        try: 
            obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
        except StopIteration:
            break
        l_class = obj_meta.classifier_meta_list
        
        while l_class is not None:
            print("There's SOME metadata!")
            try:
                class_meta = pyds.NvDsClassifierMeta.cast(l_class.data)
            except StopIteration:
                break
            l_label = class_meta.label_info_list
            while l_label is not None:
                    try:
                        label_info = pyds.NvDsLabelInfo.cast(l_label.data)
                    except StopIteration:
                        break
                print("Result:", label_info.result_label)

I can see my sgie BBs in the display, but message “There’s SOME metadata!” is never printed on console. (i.e. classifier_meta_list is always None)
I’ve already checked and I’m getting obj_meta metadata correctly, but for every detection, classifier_meta_list is always None!

I’m attaching an image to show what I mean with more detail: As I said the model is behaving poorly so I changed the operate-on-class_id to VEHICLE to get a proper shot to show you, but please keep in mind this is not the issue. The issue is that BB (Although wrong) is shown on display (“con_tapabocas”), but metadata is not recovered from classifier_meta_list.

Can you share your sgie config with us?

Sure, here it is.
I tend to keep the original lines commented, just in case.
Uncommented lines are my own TLT trained model:
sgie_config_epp.txt (3.5 KB)

Could you also set following config items to 0 and see if issue persist?

input-object-min-height=30
input-object-min-width=40

If you still see the issue, you can try to add some debug info inside gstnvinfer.cpp → should_infer_object to check where is wrong

When applying this change, video wasn’t even displayed and got this error:

/dvs/git/dirty/git-master_linux/nvutils/nvbufsurftransform/nvbufsurftransform.cpp:3391: => VIC Configuration failed image scale factor exceeds 16, use GPU for Transformation
0:00:41.090910157 15716      0x280f8f0 WARN                 nvinfer gstnvinfer.cpp:1277:convert_batch_and_push_to_input_thread:<secondary1-nvinference-engine> error: NvBufSurfTransform failed with error -2 while converting buffer
Error: gst-stream-error-quark: NvBufSurfTransform failed with error -2 while converting buffer (1): /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(1277): convert_batch_and_push_to_input_thread (): /GstPipeline:pipeline0/GstNvInfer:secondary1-nvinference-engine
Exiting app

Sorry… I understand Python code very well, but have little experience with C++.
Can you guide me on: Where to look and change the script you mention, or any documentation on the relationship between the scripts on both languages?

Ok, so I think I managed to add some code in gstnvinfer.cpp to write some debugging to a file but now:
I understand that C++, unlike Python, needs to be compiled so here’s what I’ve tried:

  • Compiling my own way, adding the library paths of the .h files everytime it failed:
    g++ -I../../includes -I../../libs/nvdsinfer -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 ...... gstvninfer.cpp
    I got error:
    error: DS_VERSION was not declared in this scope
  • Since I didn’t wanted to change anything else in the .cpp file, I read in a forum that I needed to build the whole app again:
    $ make
    Makefile:25: *** "CUDA_VER is not set". Stop

So I really don’t understand how to compile my new gstnvinfer.cpp in order to run with my debug instructions and I don’t want to ruin anything from the rest of the app…

Thanks in advance

Oh, that’s a basic compile issue.
You can try to use “export DS_VERSION=5.0” “export CUDA_VER=10.2”, assume your DS version is 5.0 and cuda version is 10.2.

Setting the DS_VERSION =5.1 in my case led to a new error:

deepstream_app.c:23:10: fatal error: gst/gst.h: No such file or directory
#include <gst/gst.h>

But in general, my question is: is this the correct way to compile gstvninfer.cpp?
What is the correct way to add debug into gstvninfer.cpp, in order to fix my original problem?
Thanks in advance!

Oh, you need to cd the same directory where the gstnvinfer.cpp locate and run make and make install

Ok, I think I did it:
I added a log entry in gstnvinfer.cpp → should_infer_object everytime it returns a TRUE/FALSE value to see how its behaving, like this:

static inline gboolean
should_infer_object (GstNvInfer * nvinfer, GstBuffer * inbuf,
    NvDsObjectMeta * obj_meta, gulong frame_num,
    GstNvInferObjectHistory * history)
{
  if (nvinfer->operate_on_gie_id > -1 &&
      obj_meta->unique_component_id != nvinfer->operate_on_gie_id) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 1: FALSE!");
    return FALSE;
    }
  if (obj_meta->rect_params.width < nvinfer->min_input_object_width) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 2: FALSE!");
    return FALSE;
  }
  if (obj_meta->rect_params.height < nvinfer->min_input_object_height) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 3: FALSE!");
    return FALSE;
  }
  if (nvinfer->max_input_object_width > 0 &&
      obj_meta->rect_params.width > nvinfer->max_input_object_width) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 4: FALSE!");
    return FALSE;
  }
  if (nvinfer->max_input_object_height > 0 &&
      obj_meta->rect_params.height > nvinfer->max_input_object_height) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 5: FALSE!");
    return FALSE;
  }
  /* Infer on object if the operate_on_class_ids list is empty or if
   * the flag at index  class_id is TRUE. */
  if (!nvinfer->operate_on_class_ids->empty () &&
      ((int) nvinfer->operate_on_class_ids->size () <= obj_meta->class_id ||
          nvinfer->operate_on_class_ids->at (obj_meta->class_id) == FALSE)) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 6: FALSE!");
    return FALSE;
  }

  /* History is irrevelavant for detectors. */
  if (history && IS_CLASSIFIER_INSTANCE (nvinfer)) {
    gboolean should_reinfer = FALSE;

    /* Do not reinfer if the object area has not grown by the reinference area
     * threshold and reinfer interval criteria is not met. */
    if ((history->last_inferred_coords.width *
          history->last_inferred_coords.height * (1 +
            REINFER_AREA_THRESHOLD)) <
        (obj_meta->rect_params.width * obj_meta->rect_params.height)) {
      should_reinfer = TRUE;
      GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 7: TRUE!");
    }

    if (frame_num - history->last_inferred_frame_num >
         nvinfer->secondary_reinfer_interval) {
      GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 8: TRUE!");
      should_reinfer = TRUE;
    }
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 9:", should_reinfer);
    return should_reinfer;
  }
  GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 10: outside all ifs: TRUE");
  return TRUE;
}

So now, I can see the process enters this function a lot of times (This makes sense, because the sgie is configured to run on class ‘vehicle’), but, as you can see in the following image, it shows the detection of the sgie (CON_TAPABOCAS) in the display, but it doesn’t print the metadata from the probe.