Deepstream branch giving same output from a TAO trained model

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, , ,
1

Please provide complete information as applicable to your setup.

• Hardware Platform: GPU (Tesla T4)
• NVIDIA GPU Driver Version: 525.85.12
• DeepStream Version: 6.2
• TensorRT Version: 8.5.2.2
• Issue Type: Question

I worked with the TAO toolkit to fine-tune a pretrained classifier. There is a person detector in the primary config and then a classifier as secondary; I integrated the model with one of the deepstream branches as shown in deepstream parallel inference app.

I was able to successfully integrate the model and able to get inferences on input videos.

However, the inference on the video is always the same label (I have two classes in the labels.txt file). I believe that could be the model’s performance issue, but it should not be the same label every time; I would like to know what can be done here.

2

Did you evaluate the model performance in TAO?

4

Yes, I did evaluate on TAO, it trained with accuracy of 90, but using on a real data gives only a single label as output.

5

How did you verify that with the real data? With TensorRT?

6

I integrated the etlt model file with deesptream and tried to run on a video where the output is just a single label.

7

That means you use your classifier model as PGIE in a DeepStream pipeline, right?

Can you provide your complete pipeline and configurations?

8

There is sample of using classifier model as PGIE in NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream (github.com)

9

The classifier model is used as SGIE, at PGIE there is a person detector (peoplenet) on which classifier is to run.
please find the configurations:

[property]
gpu-id=0
# preprocessing parameters
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
batch-size=30
maintain-aspect-ratio=0

# Model specific paths. These need to be updated for every classification model.
# int8-calib-file=<Path to optional INT8 calibration cache>
labelfile-path=/root/export_ink/labels.txt
tlt-encoded-model=/root/export_ink/ink_resnet.etlt
tlt-model-key=nvidia_tlt
## input-dims=3;224;224;0
infer-dims=3;224;224  
uff-input-blob-name=input_1
uff-input-order=0
output-blob-names=predictions/Softmax
num-detected-classes=2

## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
# process-mode: 2 - inferences on crops from primary detector, 1 - inferences on whole frame
process-mode=2
network-type=1
gie-unique-id=5   
operate-on-gie-id=1
classifier-threshold=0.2
operate-on-class-ids=0
10

Please post the “/root/export_ink/labels.txt” for reviewing

11

Please find the requested file
labels.txt (18 Bytes)

12

It is the model you mentioned in ONNX model integration with deepstream parallel inference app - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums, right?

If so, you need to write the class parsing function by yourself since your output layer is not CHW dimension.

And the label file should be in such format:
ink_thrown;no__ink

13

No, this is the other model, I am working on both the ONNX model and a TAO trained model. I was asked to post them in different forums.

14

Hi, as I mentioned, this model is TAO trained (would request to not link the other post on the forum with this). The output layer is in CHW dimension.

I did try with this format of labels.txt too, but still the output was same.

15

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

So you need to check whether the model is configured correctly. Please follow the DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums to check the parameters.

18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.