How to get multiple predictions for each frame when using a classifier in Deepstream as primary gie to analyze full frames?

• Hardware Platform (Jetson / GPU) : GPU
• DeepStream Version : 5.1
• TensorRT Version : 7.2.1-1+cuda11.1
• NVIDIA GPU Driver Version (valid for GPU only) : 455.23.05
• Issue Type(questions, new requirements, bugs) : Question

Hi,

We are using a classifier model trained with TLT in Deepstream as primary gie. We are classifiying full frames and we can access Deepstream metadata (NvDsLabelInfo) to get which is the result/prediction for each frame (and its probability). In NvDsLabelInfo, Deepstream stores 1 prediction when classifiying full frame.
However, we would like to know if deepstream is capable of returning a list of predictions for a frame, not only 1.

For example, we would like to be able to obtain something like this (for each frame):

  • Apple | 85%
  • Banana | 5%
  • Orange | 10 %

Right now for previous example we would get only:

  • Apple | 85%

Thanks in advance.

Check _NvDsClassifierMeta field NvDsLabelInfoList * label_info_list
it Holds a pointer to a list of pointers of type NvDsLabelInfo

Thank you for your response. I will check it out and come back to tell you.

Hi again,

I checked NvDsLabelInfoList * label_info_list and printed out the size and content of the list, but it has only one value.

As I am classifying full frames, there is only one bounding box (that covers each full frame), see bellow image:

In the borders of the image you can see the red bounding box generated by Deepstream because I am executing it with a classifier as primary gie and setting “process-mode” parameter of nvinfer plugin configuration to 1 (process-mode=1).

NvDsLabelInfoList seems to have only the prediction for that bounding box. I think Deepstream only returns 1 prediction for each bounding box, so that it´s not capable of returning multiple predictions for each frame in the way I said in the example of my first comment:

Correct me if I´m wrong.

Thanks in advance.

Oh, sorry.
_NvDsClassifierMeta field NvDsLabelInfoList * label_info_list just hold the lables for each object.
for different objects probability, you need to iterate each objects to get the data. and the scenario you mentioned reproed, we are investigating the issue. will update once progress.

lable: white lable id: 0 probity: 0.851683
lable: white lable id: 0 probity: 0.863412
lable: yellow lable id: 0 probity: 0.639699

Ok, thank you very much for your response.