• Hardware Platform (Jetson / GPU) Jetson Nano
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only) 4.4
• TensorRT Version 7.0
Hi
I am using two nvinfer elements in my pipeline. The primary one is a 1-class detector, and it works well. The class_id for the one-class is naturally class_id = 0. The secondary one is a 20-class classifier. You can say it’s similar to the car color secondary classifier. Additionally, I append user metadata to the gst buffers upstream of the primary nvinfer element using a custom gstreamer plugin A. The custom metadata, in this case, is bounding box coordinates.
In between the two nvinfer elements, I have another custom gstreamer plugin B which:
- Fetches the custom metadata appended by custom plugin A.
- Depending on if there are any detections through the primary
nvinferelement, resizes the bounding box, and adds it to the frame as NvDsObjectMeta using a class_id = 1. The code for that is taken from some of the sample apps in deepstream and is listed below.
The problem is that the secondary nvinfer element only operates on the class_id = 0 NvDsObjectMeta. The classifier does not work on the manually added NvDsObjectMeta using plugin B. I have specified both classes in the secondary nvinfer config in operate-on-class-ids=0;1. I can verify using a plugin downstream of the secondary nvinfer that the object metadata for class_id = 1 is indeed appended and can be accessed. However, the classifier metadata, as well as the NvDsInferTensorMeta is not present for class_id = 1. Both the NvDsClassifierMeta as well as NvDsInferTensorMeta can be accessed, however, for class_id = 0. I am kind of lost here on why this does not work. Any tips to resolve this would be great.
Thanks
static GstFlowReturn
gst_example_transform_ip (GstBaseTransform * btrans, GstBuffer * buffer)
{
GstExampleIOI * extractioi = GST_EXAMPLE(btrans);
NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buffer);
NvCustomUserMeta *custom_user_meta = NULL;
for (NvDsMetaList *l_frame = batch_meta->frame_meta_list; l_frame != NULL; l_frame = l_frame->next)
{
NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
for (NvDsMetaList *l_user_meta = frame_meta->frame_user_meta_list; l_user_meta != NULL; l_user_meta = l_user_meta->next)
{
NvDsUserMeta *user_meta = (NvDsUserMeta *) (l_user_meta->data);
if(user_meta->base_meta.meta_type == NVDS_CUSTOM_USER_META)
{
custom_user_meta = (NvCustomUserMeta *) user_meta->user_meta_data;
}
}
if (custom_user_meta != NULL)
{
for (NvDsMetaList *l_obj = frame_meta->obj_meta_list; l_obj != NULL; l_obj = l_obj->next)
{
NvDsObjectMeta *obj_meta = (NvDsObjectMeta *) (l_obj->data);
if (obj_meta->class_id == 0)
{
custom_user_meta->top = custom_user_meta->top + 0.2 * custom_user_meta->height;
custom_user_meta->height = 0.8 * custom_user_meta->height;
}
}
NvDsObjectMeta *object_meta = nvds_acquire_obj_meta_from_pool(batch_meta);
NvOSD_RectParams & rect_params = object_meta->rect_params;
NvOSD_TextParams & text_params = object_meta->text_params;
rect_params.left = custom_user_meta->left;
rect_params.top = custom_user_meta->top;
rect_params.width = custom_user_meta->width;
rect_params.height = custom_user_meta->height;
object_meta->class_id = 1;
object_meta->object_id = UNTRACKED_OBJECT_ID;
rect_params.has_bg_color = 0;
rect_params.bg_color = (NvOSD_ColorParams) {1, 1, 0, 0.4};
rect_params.border_width = 3;
rect_params.border_color = (NvOSD_ColorParams) {0, 1, 0, 1};
g_strlcpy (object_meta->obj_label, "label1", MAX_LABEL_SIZE);
text_params.display_text = g_strdup ("label1");
text_params.x_offset = rect_params.left;
text_params.y_offset = rect_params.top - 10;
text_params.set_bg_clr = 1;
text_params.text_bg_clr = (NvOSD_ColorParams) {0, 1, 0, 1};
text_params.font_params.font_name = (char *)"Serif";
text_params.font_params.font_size = 11;
text_params.font_params.font_color = (NvOSD_ColorParams) {1, 1, 1, 1};
nvds_add_obj_meta_to_frame(frame_meta, object_meta, NULL);
frame_meta->bInferDone = TRUE;
}
}
}
return GST_FLOW_OK;
}