I built a “mask/not mask” sgie model to run on traditional pgie-4 classes model (To run only on class “person”).
Although my model is predicting poorly, I can see the wrong predictions on screen, however I can’t get the metadada: classifier_meta_list is always None.
I’m trying to get the metadata in classifier_meta_list by:
while l_obj is not None:
try:
obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
except StopIteration:
break
l_class = obj_meta.classifier_meta_list
while l_class is not None:
print("There's SOME metadata!")
try:
class_meta = pyds.NvDsClassifierMeta.cast(l_class.data)
except StopIteration:
break
l_label = class_meta.label_info_list
while l_label is not None:
try:
label_info = pyds.NvDsLabelInfo.cast(l_label.data)
except StopIteration:
break
print("Result:", label_info.result_label)
I can see my sgie BBs in the display, but message “There’s SOME metadata!” is never printed on console. (i.e. classifier_meta_list is always None)
I’ve already checked and I’m getting obj_meta metadata correctly, but for every detection, classifier_meta_list is always None!
I’m attaching an image to show what I mean with more detail: As I said the model is behaving poorly so I changed the operate-on-class_id to VEHICLE to get a proper shot to show you, but please keep in mind this is not the issue. The issue is that BB (Although wrong) is shown on display (“con_tapabocas”), but metadata is not recovered from classifier_meta_list.
Sure, here it is.
I tend to keep the original lines commented, just in case.
Uncommented lines are my own TLT trained model: sgie_config_epp.txt (3.5 KB)
Sorry… I understand Python code very well, but have little experience with C++.
Can you guide me on: Where to look and change the script you mention, or any documentation on the relationship between the scripts on both languages?
Ok, so I think I managed to add some code in gstnvinfer.cpp to write some debugging to a file but now:
I understand that C++, unlike Python, needs to be compiled so here’s what I’ve tried:
Compiling my own way, adding the library paths of the .h files everytime it failed: g++ -I../../includes -I../../libs/nvdsinfer -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 ...... gstvninfer.cpp
I got error: error: DS_VERSION was not declared in this scope
Since I didn’t wanted to change anything else in the .cpp file, I read in a forum that I needed to build the whole app again: $ make Makefile:25: *** "CUDA_VER is not set". Stop
So I really don’t understand how to compile my new gstnvinfer.cpp in order to run with my debug instructions and I don’t want to ruin anything from the rest of the app…
Oh, that’s a basic compile issue.
You can try to use “export DS_VERSION=5.0” “export CUDA_VER=10.2”, assume your DS version is 5.0 and cuda version is 10.2.
Setting the DS_VERSION =5.1 in my case led to a new error:
deepstream_app.c:23:10: fatal error: gst/gst.h: No such file or directory
#include <gst/gst.h>
But in general, my question is: is this the correct way to compile gstvninfer.cpp?
What is the correct way to add debug into gstvninfer.cpp, in order to fix my original problem?
Thanks in advance!
Ok, I think I did it:
I added a log entry in gstnvinfer.cpp → should_infer_object everytime it returns a TRUE/FALSE value to see how its behaving, like this:
static inline gboolean
should_infer_object (GstNvInfer * nvinfer, GstBuffer * inbuf,
NvDsObjectMeta * obj_meta, gulong frame_num,
GstNvInferObjectHistory * history)
{
if (nvinfer->operate_on_gie_id > -1 &&
obj_meta->unique_component_id != nvinfer->operate_on_gie_id) {
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 1: FALSE!");
return FALSE;
}
if (obj_meta->rect_params.width < nvinfer->min_input_object_width) {
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 2: FALSE!");
return FALSE;
}
if (obj_meta->rect_params.height < nvinfer->min_input_object_height) {
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 3: FALSE!");
return FALSE;
}
if (nvinfer->max_input_object_width > 0 &&
obj_meta->rect_params.width > nvinfer->max_input_object_width) {
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 4: FALSE!");
return FALSE;
}
if (nvinfer->max_input_object_height > 0 &&
obj_meta->rect_params.height > nvinfer->max_input_object_height) {
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 5: FALSE!");
return FALSE;
}
/* Infer on object if the operate_on_class_ids list is empty or if
* the flag at index class_id is TRUE. */
if (!nvinfer->operate_on_class_ids->empty () &&
((int) nvinfer->operate_on_class_ids->size () <= obj_meta->class_id ||
nvinfer->operate_on_class_ids->at (obj_meta->class_id) == FALSE)) {
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 6: FALSE!");
return FALSE;
}
/* History is irrevelavant for detectors. */
if (history && IS_CLASSIFIER_INSTANCE (nvinfer)) {
gboolean should_reinfer = FALSE;
/* Do not reinfer if the object area has not grown by the reinference area
* threshold and reinfer interval criteria is not met. */
if ((history->last_inferred_coords.width *
history->last_inferred_coords.height * (1 +
REINFER_AREA_THRESHOLD)) <
(obj_meta->rect_params.width * obj_meta->rect_params.height)) {
should_reinfer = TRUE;
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 7: TRUE!");
}
if (frame_num - history->last_inferred_frame_num >
nvinfer->secondary_reinfer_interval) {
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 8: TRUE!");
should_reinfer = TRUE;
}
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 9:", should_reinfer);
return should_reinfer;
}
GST_INFO_OBJECT(nvinfer, "========= Should_reinfer 10: outside all ifs: TRUE");
return TRUE;
}
So now, I can see the process enters this function a lot of times (This makes sense, because the sgie is configured to run on class ‘vehicle’), but, as you can see in the following image, it shows the detection of the sgie (CON_TAPABOCAS) in the display, but it doesn’t print the metadata from the probe.
Although my model is a detector, I still tested with both values you mention: classifier-async-mode=0
and classifier-async-mode=1
Both with the same results: classifier_meta_list is always None, and Metadata from the probe can’t be recovered
Hmmm, can you also add some log or via gdb in the gstnvinfer_meta_utils.cpp ->merge_classification_output to see what happened, all the source code is open-sourced, you can track the call flow to find the root cause.
Ok, so here’s something I found:
I added some debug on gstnvinfer_meta_utils.cpp → attach_metadata_detector and found out it’s never calling this function.
So I also added debug in gstnvinfer → gst_nvinfer_process_objects in the place where I think the attach_meta_detector function should be called and this what I found (Please check comments on each line):
GST_INFO_OBJECT(nvinfer, "========= Check nvinfer->classifier_async_mode:", nvinfer->classifier_async_mode); // This gets printed many times but classifier_async_mode is empty
if (obj_history && nvinfer->classifier_async_mode) { // classifier_async_mode is empty so this condition is not true
GstNvInferFrame frame;
frame.obj_meta = object_meta;
GST_INFO_OBJECT(nvinfer, "========= About to call function"); // This sentence never gets written
attach_metadata_classifier (nvinfer, nullptr, frame, obj_history->cached_info); // This function is never called
You can see in the image below why I’m concluding that the value is empty, because it’s not printing its value:
This happens for both classifier-async-mode = 0 or 1 (Although my model is a detector)
Why is this happening? Could this be the source of why classifier_meta_list is always None? How can this be solved?
Sgie detects if a person has: mask, helmet, boots.
I’ve already checked and attach_meta_detector is working fine for pgie and I can handle probe’s metadata as well.
However, when the detection of the sgie is made, I can’t get the metadata from the probe. This is: classifier_meta_list is always None. (Or is there another structure for this?)
So, I was trying to add some debugging to the .cpp files to check where is failing, but maybe I’m checking in the wrong place? In which .cpp file or function does the sgie results get stored?
