Sgie's detecting properly but classifier_meta_list is None

• Hardware Platform: Jetson Xavier NX
• DeepStream Version: 5.0.1
• Language: Python

I built a “mask/not mask” sgie model to run on traditional pgie-4 classes model (To run only on class “person”).
Although my model is predicting poorly, I can see the wrong predictions on screen, however I can’t get the metadada: classifier_meta_list is always None.

I’m trying to get the metadata in classifier_meta_list by:

   while l_obj is not None:
        except StopIteration:
        l_class = obj_meta.classifier_meta_list
        while l_class is not None:
            print("There's SOME metadata!")
                class_meta = pyds.NvDsClassifierMeta.cast(
            except StopIteration:
            l_label = class_meta.label_info_list
            while l_label is not None:
                        label_info = pyds.NvDsLabelInfo.cast(
                    except StopIteration:
                print("Result:", label_info.result_label)

I can see my sgie BBs in the display, but message “There’s SOME metadata!” is never printed on console. (i.e. classifier_meta_list is always None)
I’ve already checked and I’m getting obj_meta metadata correctly, but for every detection, classifier_meta_list is always None!

I’m attaching an image to show what I mean with more detail: As I said the model is behaving poorly so I changed the operate-on-class_id to VEHICLE to get a proper shot to show you, but please keep in mind this is not the issue. The issue is that BB (Although wrong) is shown on display (“con_tapabocas”), but metadata is not recovered from classifier_meta_list.

1 Like

Can you share your sgie config with us?

Sure, here it is.
I tend to keep the original lines commented, just in case.
Uncommented lines are my own TLT trained model:
sgie_config_epp.txt (3.5 KB)

Could you also set following config items to 0 and see if issue persist?


If you still see the issue, you can try to add some debug info inside gstnvinfer.cpp → should_infer_object to check where is wrong

When applying this change, video wasn’t even displayed and got this error:

/dvs/git/dirty/git-master_linux/nvutils/nvbufsurftransform/nvbufsurftransform.cpp:3391: => VIC Configuration failed image scale factor exceeds 16, use GPU for Transformation
0:00:41.090910157 15716      0x280f8f0 WARN                 nvinfer gstnvinfer.cpp:1277:convert_batch_and_push_to_input_thread:<secondary1-nvinference-engine> error: NvBufSurfTransform failed with error -2 while converting buffer
Error: gst-stream-error-quark: NvBufSurfTransform failed with error -2 while converting buffer (1): /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(1277): convert_batch_and_push_to_input_thread (): /GstPipeline:pipeline0/GstNvInfer:secondary1-nvinference-engine
Exiting app

Sorry… I understand Python code very well, but have little experience with C++.
Can you guide me on: Where to look and change the script you mention, or any documentation on the relationship between the scripts on both languages?

Ok, so I think I managed to add some code in gstnvinfer.cpp to write some debugging to a file but now:
I understand that C++, unlike Python, needs to be compiled so here’s what I’ve tried:

  • Compiling my own way, adding the library paths of the .h files everytime it failed:
    g++ -I../../includes -I../../libs/nvdsinfer -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 ...... gstvninfer.cpp
    I got error:
    error: DS_VERSION was not declared in this scope
  • Since I didn’t wanted to change anything else in the .cpp file, I read in a forum that I needed to build the whole app again:
    $ make
    Makefile:25: *** "CUDA_VER is not set". Stop

So I really don’t understand how to compile my new gstnvinfer.cpp in order to run with my debug instructions and I don’t want to ruin anything from the rest of the app…

Thanks in advance

Oh, that’s a basic compile issue.
You can try to use “export DS_VERSION=5.0” “export CUDA_VER=10.2”, assume your DS version is 5.0 and cuda version is 10.2.

Setting the DS_VERSION =5.1 in my case led to a new error:

deepstream_app.c:23:10: fatal error: gst/gst.h: No such file or directory
#include <gst/gst.h>

But in general, my question is: is this the correct way to compile gstvninfer.cpp?
What is the correct way to add debug into gstvninfer.cpp, in order to fix my original problem?
Thanks in advance!

Oh, you need to cd the same directory where the gstnvinfer.cpp locate and run make and make install

Ok, I think I did it:
I added a log entry in gstnvinfer.cpp → should_infer_object everytime it returns a TRUE/FALSE value to see how its behaving, like this:

static inline gboolean
should_infer_object (GstNvInfer * nvinfer, GstBuffer * inbuf,
    NvDsObjectMeta * obj_meta, gulong frame_num,
    GstNvInferObjectHistory * history)
  if (nvinfer->operate_on_gie_id > -1 &&
      obj_meta->unique_component_id != nvinfer->operate_on_gie_id) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 1: FALSE!");
    return FALSE;
  if (obj_meta->rect_params.width < nvinfer->min_input_object_width) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 2: FALSE!");
    return FALSE;
  if (obj_meta->rect_params.height < nvinfer->min_input_object_height) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 3: FALSE!");
    return FALSE;
  if (nvinfer->max_input_object_width > 0 &&
      obj_meta->rect_params.width > nvinfer->max_input_object_width) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 4: FALSE!");
    return FALSE;
  if (nvinfer->max_input_object_height > 0 &&
      obj_meta->rect_params.height > nvinfer->max_input_object_height) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 5: FALSE!");
    return FALSE;
  /* Infer on object if the operate_on_class_ids list is empty or if
   * the flag at index  class_id is TRUE. */
  if (!nvinfer->operate_on_class_ids->empty () &&
      ((int) nvinfer->operate_on_class_ids->size () <= obj_meta->class_id ||
          nvinfer->operate_on_class_ids->at (obj_meta->class_id) == FALSE)) {
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 6: FALSE!");
    return FALSE;

  /* History is irrevelavant for detectors. */
  if (history && IS_CLASSIFIER_INSTANCE (nvinfer)) {
    gboolean should_reinfer = FALSE;

    /* Do not reinfer if the object area has not grown by the reinference area
     * threshold and reinfer interval criteria is not met. */
    if ((history->last_inferred_coords.width *
          history->last_inferred_coords.height * (1 +
        (obj_meta->rect_params.width * obj_meta->rect_params.height)) {
      should_reinfer = TRUE;
      GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 7: TRUE!");

    if (frame_num - history->last_inferred_frame_num >
         nvinfer->secondary_reinfer_interval) {
      GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 8: TRUE!");
      should_reinfer = TRUE;
    GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 9:", should_reinfer);
    return should_reinfer;
  GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 10: outside all ifs: TRUE");
  return TRUE;

So now, I can see the process enters this function a lot of times (This makes sense, because the sgie is configured to run on class ‘vehicle’), but, as you can see in the following image, it shows the detection of the sgie (CON_TAPABOCAS) in the display, but it doesn’t print the metadata from the probe.

Have you disabled the async mode for SGIE?

Yes, async mode is off. Although now that I think about it, does it make sense, given my sgie is a detector, and not a classifier?

operate-on-class-ids=0 # Vehicles for testing. Person=2

Would it make a difference if turn async mode on?

Although my model is a detector, I still tested with both values you mention:
Both with the same results: classifier_meta_list is always None, and Metadata from the probe can’t be recovered

Hmmm, can you also add some log or via gdb in the gstnvinfer_meta_utils.cpp ->merge_classification_output to see what happened, all the source code is open-sourced, you can track the call flow to find the root cause.

Ok, so here’s something I found:
I added some debug on gstnvinfer_meta_utils.cpp → attach_metadata_detector and found out it’s never calling this function.
So I also added debug in gstnvinfer → gst_nvinfer_process_objects in the place where I think the attach_meta_detector function should be called and this what I found (Please check comments on each line):

      GST_INFO_OBJECT(nvinfer, "========= Check nvinfer->classifier_async_mode:", nvinfer->classifier_async_mode);  // This gets printed many times but classifier_async_mode is empty
      if (obj_history && nvinfer->classifier_async_mode) { // classifier_async_mode is empty so this condition is not true
        GstNvInferFrame frame;
        frame.obj_meta = object_meta;
        GST_INFO_OBJECT(nvinfer, "========= About to call function"); // This sentence never gets written
        attach_metadata_classifier (nvinfer, nullptr, frame, obj_history->cached_info); // This function is never called

You can see in the image below why I’m concluding that the value is empty, because it’s not printing its value:

This happens for both classifier-async-mode = 0 or 1 (Although my model is a detector)
Why is this happening? Could this be the source of why classifier_meta_list is always None? How can this be solved?

I’m confused, what’s your whole pipeline, why you expect the attach_meta_detector get called if your sgie is a classifier?

No, my sgie is a detector. My pipeline is:

streamux → pgie (detector: 4 classes) → Tracker → sgie (detector on class: Person) → tiler

Sgie detects if a person has: mask, helmet, boots.
I’ve already checked and attach_meta_detector is working fine for pgie and I can handle probe’s metadata as well.
However, when the detection of the sgie is made, I can’t get the metadata from the probe. This is: classifier_meta_list is always None. (Or is there another structure for this?)
So, I was trying to add some debugging to the .cpp files to check where is failing, but maybe I’m checking in the wrong place? In which .cpp file or function does the sgie results get stored?

IF your sgie is detector, then classifier_meta_list is null.