Classification FAN :: Could not find output layer 'predictions/probs' in engine

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I have trained a classification model using TAO 5.2. When deploying with deepstream, I have applied the following config file:

[property]
gpu-id=0
net-scale-factor=0.017507
offsets=123.675;116.280;103.53
model-color-format=0
infer-dims=3;224;224
network-type=1
num-detected-classes=14
uff-input-blob-name=input_1
maintain-aspect-ratio=0
output-tensor-meta=0
onnx-file=classification_model_FHD_export_10.onnx
labelfile-path=labels_FHD.txt
model-engine-file=classification_model_b4_export_FHD_10.engine
batch-size=2
output-blob-names=predictions/probs
is-classifier=1
process-mode=2 ##1 Primary 2 Secondary
network-mode=2 ## 0=FP32, 1=INT8, 2=FP16 mode
interval=0
gie-unique-id=6
operate-on-class-ids=0;1
classifier-threshold=0.51
#classifier-async-mode=1

It runs as a SGIE attached to a PGIE. When running, I see the warning

ERROR: [TRT]: 3: Cannot find binding of given name: predictions/probs
0:00:04.078261419 309595      0x3587180 WARN                 nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<secondary-nvinference-engine> NvDsInferContext[UID 6]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1955> [UID = 6]: Could not find output layer 'predictions/probs' in engine

I found the config file of vehicletypenet classifier and applied it to the FAN model. But I doubt if it actually have predictions/probs output. Also, I’m not sure about the net-scale-factor, offsets, and model-color-format for the FAN model as well. TAO also does not export the label file as I found the label sequence during input doesn’t match with outputs.

Why TAO doesn’t export the config and label files? How do I find the correct config and labels?

3

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

4

• Hardware Platform: dGPU
• DeepStream Version: 6.2
• TensorRT Version: 8.6
• NVIDIA GPU Driver Version 535
• Issue Type( questions)
•TAO Toolkit Version: 5.2

5

you can use Netron tool open the onnx model, then you can see the output layers name.
you can get the explanation of net-scale-factor, offsets, and model-color-format in nvinfer doc. these parameters are needed when training model. please find them in TAO configuration files.

6

Yes I have already done that. On previous tao toolkit versions, the config parameters were exported along with the onnx or engine file. Don’t know why the latest tao toolkit doesnt support this.

However, I see significant difference between the label sequence that was provided during the training and got after the trained model. I have to manually alter the label sequence to get correct results. Can you explain why this happens? On previous tao toolkit, the final label files were also being exported, but the classification pyt misses this. Manually relabeling the labels are very frustrating process, and also questions the reliability of tao trained models if not automatically exported.

10

The classification_pyt does not generate deepstream config files and class labels.
Please set output-blob-names=probs based on tao_pytorch_backend/nvidia_tao_pytorch/cv/classification/tools/onnx_utils.py at main · NVIDIA/tao_pytorch_backend · GitHub.

For deepstream config file, please set as below according to Deploying to DeepStream for Classification TF1/TF2/PyTorch - NVIDIA Docs

  • preprocessing_mode: "torch":
net-scale-factor=0.017507
offsets=123.675;116.280;103.53
model-color-format=0

For class label, you can set to alphabetical order.
For example,

cats
dogs