Hope get the output tensor of SGIE. But the obj_meta->obj_user_meta_list is NUL

[yangyi]

Please provide complete information as applicable to your setup.

**• Hardware Platform (GPU)
**• DeepStream Version6.1
**• TensorRT Version8.2E
**• NVIDIA GPU Driver Version 515
Thank you for reading this topic. Yestoday, I got help from this fourm to check SGIE output. Now it can work.
But, I meet another problem
I use the default pipeline, and hope to get the raw output tensor of SGIE, So I reference the “deepstream-infer-tensor-meta-test”
I add a “gst_pad_probe” in deepstream-app.c → process_meta as below

if (appCtx->pipeline.common_elements.secondary_gie_bin.sub_bins[0].secondary_gie)
{
GstPad *my_sgie_pad = gst_element_get_static_pad (
appCtx->pipeline.common_elements.secondary_gie_bin.sub_bins[0].secondary_gie, “src”);
if (!my_sgie_pad)
NVGSTDS_ERR_MSG_V (“Unable to get SGIE src pad\n”);
else
{
gst_pad_add_probe (my_sgie_pad, GST_PAD_PROBE_TYPE_BUFFER,
sgie_pad_buffer_probe, NULL, NULL);
gst_object_unref (my_sgie_pad);
}
}

I write the “sgie_pad_buffer_probe” in deepstream-app.c as below:

static GstPadProbeReturn
sgie_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data)
{
NvDsBatchMeta batch_meta = gst_buffer_get_nvds_batch_meta (GST_BUFFER (info->data));
/ Iterate each frame metadata in batch */
for(NvDsMetaList * l_frame = batch_meta->frame_meta_list; l_frame != NULL; l_frame = l_frame->next)
{
printf(“========================frame list========================\n”);
NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) l_frame->data;
for(NvDsMetaList * l_obj = frame_meta->obj_meta_list; l_obj != NULL; l_obj = l_obj->next)
{
printf(“---------------------list obj--------------------------\n”);
NvDsObjectMeta *obj_meta = (NvDsObjectMeta *) l_obj->data;
printf(“l_obj id = %d\n”, obj_meta->class_id);

  //---- show the classifer result label-------//
  for(NvDsMetaList *l = obj_meta->classifier_meta_list; l != NULL; l = l->next)
  {
     printf("-------classifer_list-----\n");
     NvDsClassifierMeta * classifierMeta = (NvDsClassifierMeta *) (l->data);
     for (NvDsMetaList * n = classifierMeta->label_info_list; n != NULL; n = n->next)
     {
         NvDsLabelInfo *labelInfo = (NvDsLabelInfo*) (n->data);
         printf("label: %s \n ", &labelInfo->result_label[0]);
     }
  }
  //------------ show the raw data  --------//
  /* Iterate user metadata in object to search SGIE's tensor data */
  for (NvDsMetaList * l_user = obj_meta->obj_user_meta_list; l_user != NULL;  l_user = l_user->next)
  {
    printf("*****user_meta_list*****\n");
    NvDsUserMeta *user_meta = (NvDsUserMeta *) l_user->data;
    if (user_meta->base_meta.meta_type != NVDSINFER_TENSOR_OUTPUT_META)
      continue;
  }
}

}
return GST_PAD_PROBE_OK;
}

The SGIE configure as below:

[property]
gpu-id=0
net-scale-factor=1

model-file=./Secondary_CarColor/resnet18.caffemodel
proto-file=./Secondary_CarColor/resnet18.prototxt
model-engine-file=./Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine
int8-calib-file=./Secondary_CarColor/cal_trt.bin
mean-file=./Secondary_CarColor/mean.ppm
labelfile-path=./Secondary_CarColor/labels.txt
force-implicit-batch-dim=1
batch-size=1
model-color-format=1
process-mode=2

0=FP32, 1=INT8, 2=FP16 mode

network-mode=1
output-blob-names=predictions/Softmax
classifier-async-mode=1
classifier-threshold=0.41
input-object-min-width = 10
input-object-min-height = 10
operate-on-gie-id=1
operate-on-class-ids=5
classifier-type=carcolor
network-type=1
output-tensor-meta=1
#scaling-filter=0
#scaling-compute-hw=0

The result is

In this example, l_obj id = 5, SGIE works, I can get the output label “white” . But I think the “obj_meta->obj_user_meta_list” is NULL, so there is no output of “user_meta_list”.

I check the similar topics in this forum, but I can not get the clear answer. If I hope get the output tensor of SGIE, what can I do?

Thank you very much.

[yangyi]

I checked the "gst-nvinfer - > gstnvinfer_meta_utils.cpp ->attach_tensor_output_meta ".

at the end of the function, I find there are three types to attach the user_meta as below:

if (nvinfer->input_tensor_from_meta) {
cout<<“----nvds_add_user_meta_to_roi” <<endl;
nvds_add_user_meta_to_roi (frame.roi_meta, user_meta);
} else if (nvinfer->process_full_frame) {
cout<<“-----process_full_frame” <<endl;
nvds_add_user_meta_to_frame (frame.frame_meta, user_meta);
} else {
cout<<“-----nvds_add_user_meta_to_obj” <<endl;
nvds_add_user_meta_to_obj (frame.obj_meta, user_meta);
}

the print is " ----process_full_frame".
My question is:
(1) Is the above code determined attachment type? Am I right?
(2) If my guess is correct, how do I change the attachment type?
(3) If the output means the user_data does not attach to the object? How Can I do. I just want to get the raw tensor output of sgie.
waiting for your help. Thank you very much

[fanzh]

1. what do you mean about “attachment type”? that code is used to add user_meta, if input_tensor_from_meta == true, this meta will add to roi_meta, and others have the similar meaning.
2. you need to set output-tensor-meta=1 in sgie 's configuration, please refer to deepstream-infer-tensor-meta-test.

[yangyi]

@yangyi

out<<“-----process_full_frame” <<endl;
nvds_add_user_meta_to_frame (

Thank you very much.
My primary aim is to find the output tensor of SGIE. So I reference the “deepstream-infer-tensor-meta-test”. I found the under two “for loop”: (frame loop and object loop), I find the “for loop” of l_user:
for (NvDsMetaList * l_user = obj_meta->obj_user_meta_list; l_user != NULL; l_user = l_user->next)
{
printf(“user_meta_list\n”);
NvDsUserMeta *user_meta = (NvDsUserMeta *) l_user->data;
if (user_meta->base_meta.meta_type != NVDSINFER_TENSOR_OUTPUT_META)
continue;
}
}

I have set the “output-tensor-meta=1” in the SGIE configure txt. But it does not work. The “printf(“user_meta_list\n”)” can not show.

Please kindly help me.

[fanzh]

sgie’s network-type should be 100 if want to dump output tensor, please refer to deepstream-infer-tensor-meta-test, and Gst-nvinfer — DeepStream 6.1 Release documentation

[yangyi]

Thank you very much.! it works