How to get multiple predictions for each frame when using a classifier in Deepstream as primary gie to analyze full frames?

• Hardware Platform (Jetson / GPU) : GPU
• DeepStream Version : 5.1
• TensorRT Version : 7.2.1-1+cuda11.1
• NVIDIA GPU Driver Version (valid for GPU only) : 455.23.05
• Issue Type(questions, new requirements, bugs) : Question

Hi,

We are using a classifier model trained with TLT in Deepstream as primary gie. We are classifiying full frames and we can access Deepstream metadata (NvDsLabelInfo) to get which is the result/prediction for each frame (and its probability). In NvDsLabelInfo, Deepstream stores 1 prediction when classifiying full frame.
However, we would like to know if deepstream is capable of returning a list of predictions for a frame, not only 1.

For example, we would like to be able to obtain something like this (for each frame):

  • Apple | 85%
  • Banana | 5%
  • Orange | 10 %

Right now for previous example we would get only:

  • Apple | 85%

Thanks in advance.

Check _NvDsClassifierMeta field NvDsLabelInfoList * label_info_list
it Holds a pointer to a list of pointers of type NvDsLabelInfo

Thank you for your response. I will check it out and come back to tell you.

Hi again,

I checked NvDsLabelInfoList * label_info_list and printed out the size and content of the list, but it has only one value.

As I am classifying full frames, there is only one bounding box (that covers each full frame), see bellow image:

In the borders of the image you can see the red bounding box generated by Deepstream because I am executing it with a classifier as primary gie and setting “process-mode” parameter of nvinfer plugin configuration to 1 (process-mode=1).

NvDsLabelInfoList seems to have only the prediction for that bounding box. I think Deepstream only returns 1 prediction for each bounding box, so that it´s not capable of returning multiple predictions for each frame in the way I said in the example of my first comment:

Correct me if I´m wrong.

Thanks in advance.

Oh, sorry.
_NvDsClassifierMeta field NvDsLabelInfoList * label_info_list just hold the lables for each object.
for different objects probability, you need to iterate each objects to get the data. and the scenario you mentioned reproed, we are investigating the issue. will update once progress.

lable: white lable id: 0 probity: 0.851683
lable: white lable id: 0 probity: 0.863412
lable: yellow lable id: 0 probity: 0.639699

Ok, thank you very much for your response.

Hi,
We do support primary inference for classifier, but make sure the whole picture just have one class object, like car color classification, you need in the one picture, there only cars, no other objects.

Hi,

I know.

A classifier as primary inference is what I´m using. I´m sure that each whole picture has only one class object. However, my question here is if deepstream is capable of returning multiple predictions and probabilities for each whole picture/frame, like other image classifier libraries. I explained an example in my first comment:

It depends on your model, your model can classify multi objects or only on object in one frame.

We trained an image classifier model using EfficientNet B1 using NVIDIA TAO Toolkit. But as far as I know, image classifiers classify full image and return multiple predictions, and each prediction has one probability.
However, we integrated that model into Deepstream and we could only get one prediction.

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Did your model classify multiple predictions in one image?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.