• Hardware Platform (Jetson / GPU) : GPU • DeepStream Version : 5.1 • TensorRT Version : 7.2.1-1+cuda11.1 • NVIDIA GPU Driver Version (valid for GPU only) : 455.23.05 • Issue Type(questions, new requirements, bugs) : Question
Hi,
We are using a classifier model trained with TLT in Deepstream as primary gie. We are classifiying full frames and we can access Deepstream metadata (NvDsLabelInfo) to get which is the result/prediction for each frame (and its probability). In NvDsLabelInfo, Deepstream stores 1 prediction when classifiying full frame.
However, we would like to know if deepstream is capable of returning a list of predictions for a frame, not only 1.
For example, we would like to be able to obtain something like this (for each frame):
In the borders of the image you can see the red bounding box generated by Deepstream because I am executing it with a classifier as primary gie and setting “process-mode” parameter of nvinfer plugin configuration to 1 (process-mode=1).
NvDsLabelInfoList seems to have only the prediction for that bounding box. I think Deepstream only returns 1 prediction for each bounding box, so that it´s not capable of returning multiple predictions for each frame in the way I said in the example of my first comment:
Oh, sorry.
_NvDsClassifierMeta field NvDsLabelInfoList * label_info_list just hold the lables for each object.
for different objects probability, you need to iterate each objects to get the data. and the scenario you mentioned reproed, we are investigating the issue. will update once progress.
Hi,
We do support primary inference for classifier, but make sure the whole picture just have one class object, like car color classification, you need in the one picture, there only cars, no other objects.
A classifier as primary inference is what I´m using. I´m sure that each whole picture has only one class object. However, my question here is if deepstream is capable of returning multiple predictions and probabilities for each whole picture/frame, like other image classifier libraries. I explained an example in my first comment:
We trained an image classifier model using EfficientNet B1 using NVIDIA TAO Toolkit. But as far as I know, image classifiers classify full image and return multiple predictions, and each prediction has one probability.
However, we integrated that model into Deepstream and we could only get one prediction.
There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks
Did your model classify multiple predictions in one image?