Classifier_meta_list is always null for classification model in deepstream-app

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson
• DeepStream Version
6.0

Hi, I am working on deepstream-app (deepstream_app_main.c in the nvdsanalytics-test app). I have the default pipeline with primary detector, tracker, secondary VehicleType classifier, and the nvds-analytics plugin enabled. I need to access the meta data generated by the classifier, but for each object, the classifier_meta_list is always NULL. My code for the nvds-analytics probe (from where I am accessing all meta data for each object detected) looks like this:

static GstPadProbeReturn
nvdsanalytics_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data)
{
  GstBuffer *buf = (GstBuffer *) info->data;
  NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);
  guint index=0;

  for (NvDsMetaList * l_frame = batch_meta->frame_meta_list; l_frame != NULL; l_frame = l_frame->next) {
    guint num_objects_det_in_curr_frame = 0;
    NvDsFrameMeta *frame_meta = (NvDsFrameMeta*)l_frame->data;

    for (NvDsMetaList * l_obj = frame_meta->obj_meta_list; l_obj != NULL; l_obj = l_obj->next) {
      NvDsObjectMeta *obj = (NvDsObjectMeta *) l_obj->data;
      //g_print("class_id:%d\n", obj->class_id);
      num_objects_det_in_curr_frame++;

      if(obj->class_id==0) {
          if(obj->classifier_meta_list==NULL)
              g_print("NULL DETECTED\n");
          else
              g_print("NOT NULL\n");
      }

......

The output is always “NULL DETECTED”.
My configurations looks like this:

[primary-gie]
enable=1
gpu-id=0
model-engine-file=/opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet10.caffemodel_b4_gpu0_fp16.engine
batch-size=4
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt

[tracker]
enable=1
# For NvDCF and DeepSORT tracker, tracker-width and tracker-height must be a multiple of 32, respectively
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
# ll-config-file required to set different tracker types
# ll-config-file=config_tracker_IOU.yml
ll-config-file=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml
# ll-config-file=config_tracker_NvDCF_accuracy.yml
# ll-config-file=config_tracker_DeepSORT.yml
gpu-id=0
enable-batch-process=1
enable-past-frame=1
display-tracking-id=1

[secondary-gie0]
enable=1
model-engine-file=/opt/nvidia/deepstream/deepstream/samples/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_fp16.engine
gpu-id=0
batch-size=16
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_secondary_vehicletypes.txt

the config_infer_secondary_vehicletypes.txt looks like this:

[property]
gpu-id=0
net-scale-factor=1
network-type=100
model-file=../../models/Secondary_VehicleTypes/resnet18.caffemodel
proto-file=../../models/Secondary_VehicleTypes/resnet18.prototxt
model-engine-file=../../models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine
int8-calib-file=../../models/Secondary_VehicleTypes/cal_trt.bin
mean-file=../../models/Secondary_VehicleTypes/mean.ppm
labelfile-path=../../models/Secondary_VehicleTypes/labels.txt
force-implicit-batch-dim=1
batch-size=16
model-color-format=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
is-classifier=1
process-mode=2
output-tensor-meta=1
output-blob-names=predictions/Softmax
classifier-async-mode=0
classifier-threshold=0.01
input-object-min-width=0
input-object-min-height=0
operate-on-gie-id=1
operate-on-class-ids=0
classifier-type=vehicletype
#scaling-filter=0
#scaling-compute-hw=0

please refer to create_common_elements of opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-app\deepstream_app.c, the pipeline should be pgie + tracker + analytics + sgie, nvdsanalytics_src_pad_buffer_probe is a probe function on nvdsanalytics plugin, after nvdsanalytics there is no objects classification meta, only after sgie, there are objects classification meta, you might add log in all_bbox_generated.

1 Like

Hi, thanks for the reply. I can access the sgie meta data from the all_bbox_generated callback function.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.