DS can get the raw classification probability, but can not show the label

Please provide complete information as applicable to your setup.

**• Hardware Platform:GPU
**• DeepStream Version 6.1
**• TensorRT Version8.2E
**• NVIDIA GPU Driver Version 515

Dear professor:
Thank you for reading my message. For my root problem, I have created two topics, thanks to the technical experts help me.

I add a new object in dsexample, and I hope to use SGIE to classify it.
(1) The property of the new added object is :
my_obj_meta->class_id = 5;
my_obj_meta->unique_component_id = 1;
my_obj_meta->object_id = 100;
set_rect_params.width = 1000;
set_rect_params.height = 200;

(2) My configure txt is
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
offsets=97.6980; 99.9760; 88.8259

labelfile-path=./SGIElabels.txt
onnx-file=./ResNet50_with_softmax_500_100.onnx
model-engine-file=./ResNet50_with_softmax_500_100.onnx_b1_gpu0_fp16.engine
batch-size=1
force-implicit-batch-dim=0

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
input-object-min-width = 50
input-object-min-height = 50

process-mode=2

#network-type: 100 output the tensor; 1:classifier
network-type=1
model-color-format=1
gie-unique-id=3
operate-on-gie-id=1
operate-on-class-ids=5
is-classifier=1
output-blob-names=predictions/Softmax
classifier-async-mode=0
classifier-threshold=0.01
output-tensor-meta=0

#scaling-filter=0
#scaling-compute-hw=0

(3) I can get the raw probability meta of SGIE as below

(4) I use the following code in deepstream_app_main.c to get the label of SGIE. But failed
//===========This function is added here======//
static GstPadProbeReturn
sgie_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data)
{
static guint use_device_mem = 0;
NvDsBatchMeta batch_meta = gst_buffer_get_nvds_batch_meta (GST_BUFFER (info->data));
/
Iterate each frame metadata in batch */
for(NvDsMetaList * l_frame = batch_meta->frame_meta_list; l_frame != NULL; l_frame = l_frame->next)
{
// printf(“========================frame list========================\n”);
NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) l_frame->data;

for(NvDsMetaList * l_obj = frame_meta->obj_meta_list; l_obj != NULL;  l_obj = l_obj->next)
{

// printf(“---------------------list obj--------------------------\n”);
NvDsObjectMeta *obj_meta = (NvDsObjectMeta *) l_obj->data;
printf(“class_id = %d, obj_id = %d \n”, obj_meta->class_id, obj_meta->object_id);
printf(“width = %f hight = %f confidence = %f \n\n”, obj_meta->rect_params.width,
obj_meta->rect_params.height, obj_meta->confidence);

  for(NvDsMetaList *l = obj_meta->classifier_meta_list; l != NULL; l = l->next)
  {
     printf("-----------SGIE classification---------\n");
     NvDsClassifierMeta * classifierMeta = (NvDsClassifierMeta *) (l->data);
     for (NvDsMetaList * n = classifierMeta->label_info_list; n != NULL; n = n->next)
     {
         NvDsLabelInfo *labelInfo = (NvDsLabelInfo*) (n->data);
         printf("class_id = %d, obj_id = %d  SGIE_label = %s \n", obj_meta->class_id, obj_meta->object_id, &labelInfo->result_label[0]);
         printf("width = %f  hight = %f  confidence = %f\n\n", obj_meta->rect_params.width,
                                            obj_meta->rect_params.height, obj_meta->confidence);
     }
  }
  /* Iterate user metadata in object to search SGIE's tensor data */
  for (NvDsMetaList * l_user = obj_meta->obj_user_meta_list; l_user != NULL;  l_user = l_user->next)
  {
    printf("Raw meta output of obj_id = %d \n", obj_meta->class_id);
    NvDsUserMeta *user_meta = (NvDsUserMeta *) l_user->data;
    if (user_meta->base_meta.meta_type != NVDSINFER_TENSOR_OUTPUT_META)
      continue;

    NvDsInferTensorMeta *meta = (NvDsInferTensorMeta *) user_meta->user_meta_data;
    for (unsigned int i = 0; i < meta->num_output_layers; i++)
    {
      NvDsInferLayerInfo *info = &meta->output_layers_info[i];
      info->buffer = meta->out_buf_ptrs_host[i];
      if (use_device_mem && meta->out_buf_ptrs_dev[i])
      {
        cudaMemcpy (meta->out_buf_ptrs_host[i], meta->out_buf_ptrs_dev[i],
                    info->inferDims.numElements * 4, cudaMemcpyDeviceToHost);
      }
    }
    NvDsInferDimsCHW dims;
    getDimsCHWFromDims (dims, meta->output_layers_info[0].inferDims);
    unsigned int numClasses = dims.c;
    float *outputCoverageBuffer = (float *) meta->output_layers_info[0].buffer;
    for (unsigned int c = 0; c < numClasses; c++)
    {
      float probability = outputCoverageBuffer[c];
      printf("%f   ", probability);
    }
    printf("\n\n");
  }


}
printf("============  end frame=====================\n");

}
return GST_PAD_PROBE_OK;
}
//==============================================================//

That means the SGIE can work and output probability for my added object, but can not show the SGIE label. I change the “operate-on-class-ids” to the other object( class_id =0, it is detect by PGIE), the label can be found, as below


In this picture, under the message “-------SGIE classifcation-------” “SGIE_label= low”

So I think there some parameters should be set. Could you kindly help me, thank you very much.

You can try to set output-tensor-meta=1.

Thank you for your responce. I have tried " output-tensor-meta=1", and I can get the classification probability of SGIE.
But I hope to get the classification label of SGIE. But I can not get it.

Thank you very much.

You can run our demo code to see if you can get the label.
Also you can refer our demo code to get the label:

sources\apps\sample_apps\deepstream-infer-tensor-meta-test\deepstream_infer_tensor_meta_test.cpp

sgie_pad_buffer_probe

Thank you very much. I find the way to show the classification probability.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.