Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) :- Jetson Xavier
• DeepStream Version :- 6.0
I have this pipeline :- pgie(car detector)->sgie1(LPD)->sgie2(LPR).
In a probe function (attached on src pad of SGIE2) I’m accessing the metadata and trying to create a JSON structure that contains, information about each car, for eg :- car_bounding_box, plate_bounding_box, LP_Number, confidence_score etc. , which are attached by different models. I’m trying to extract this metadata and put in a JSON.
my question is, how can I make sure that I’m extracting the metadata properly without mismatching between cars.
eg :- suppose there are two cars in a picture, the JSON gets created with car_bbox_info and plate_bbox_info of the car_1 but licence_plate_number of car_2.
I want to make sure that the way I’m doing it is correct.
GstPadProbeReturn sgie2_src_pad_buffer_probe(
GstPad *pad, GstPadProbeInfo *info, gpointer u_data) {
for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
l_frame = l_frame->next) {
NvDsFrameMeta *frame_meta = (NvDsFrameMeta *)(l_frame->data);
int offset = 0;
if (!frame_meta) {
continue;
}
for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;
l_obj = l_obj->next) {
obj_meta = (NvDsObjectMeta *)(l_obj->data);
if (!obj_meta) continue;
if (obj_meta->unique_component_id == SECONDARY_DETECTOR_UID) {
if (obj_meta->parent) {
/*obtaining the vehicle (PGIE) metadata*/
pleft = obj_meta->parent->detector_bbox_info.org_bbox_coords
.left;
ptop = obj_meta->parent->detector_bbox_info.org_bbox_coords
.top;
pright = obj_meta->parent->detector_bbox_info
.org_bbox_coords.left +
obj_meta->parent->detector_bbox_info
.org_bbox_coords.width;
pbottom = obj_meta->parent->detector_bbox_info
.org_bbox_coords.top +
obj_meta->parent->detector_bbox_info
.org_bbox_coords.height;
float car_conf = obj_meta->parent->confidence;
id = obj_meta->parent->object_id;
/*obtaining the LPD (SGIE1)metadata*/
sleft = obj_meta->detector_bbox_info.org_bbox_coords.left;
stop = obj_meta->detector_bbox_info.org_bbox_coords.top;
sright = obj_meta->detector_bbox_info.org_bbox_coords.left +
obj_meta->detector_bbox_info.org_bbox_coords.width;
sbottom =
obj_meta->detector_bbox_info.org_bbox_coords.top +
obj_meta->detector_bbox_info.org_bbox_coords.height;
plate_conf = obj_meta->confidence;
class_id = obj_meta->class_id;
valid_plate = is_inside(pcoords, scoords); /*checking if plate bbox is inside car bbox */
}
}
/*obtaining the LPR (SGIE2) metadata*/
for (l_class = obj_meta->classifier_meta_list; l_class != NULL;
l_class = l_class->next) {
class_meta = (NvDsClassifierMeta *)(l_class->data);
if (!class_meta) continue;
if (class_meta->unique_component_id ==
SECONDARY_CLASSIFIER_UID) {
for (label_i = 0, l_label = class_meta->label_info_list;
label_i < class_meta->num_labels && l_label;
label_i++, l_label = l_label->next) {
label_info = (NvDsLabelInfo *)(l_label->data);
if (label_info) {
if (label_info->label_id == 0 &&
label_info->result_class_id == 1) {
g_print("Plate License %s\n",
label_info->result_label);
label_id = label_info->result_label;
number = label_id;
plate_count = plate_count + 1;
number_conf = label_info->result_prob;
label_id = label_info->result_label;
}
}
}
}
}
if (/*checking some conditions here, and creating the JSON*/ )
{
j[key].push_back( ***/*key here is object ID */***
{
{"vehicle_box", PGIE_bbox_info},
{"object_id", id},
{"lpr_confidence", number_conf},
{"veh_number", number},
{"lpd_confidence", plate_conf},
{"num_plate_box", sgie1_bbox_info},
});
}
}
}
/*writing the JSON to a file*/
return GST_PAD_PROBE_OK;
}
so, with this approach, what I feel is LPR is not getting properly mapped with LPD and car detector metadata. however this is the same way it is done in the NVIDIA LPR app. But there they were not trying to match the metadata from different models. Is this not the correct way ? How can I improve it ?