Pytorch model deployed on deepstream no predictions

FROM nvcr.io/nvidia/deepstream:5.1-21.02-devel

• Hardware Platform (GPU)
• DeepStream Version 5.1
• JetPack Version -
• TensorRT Version 7.2
• NVIDIA GPU Driver Version (valid for GPU only) - 460.91.03
• Issue Type - question

Pytorch 1.9.1

I have used this script to convert model from Pytorch model zoo to onnx:

import torchvision.models as models
import torch.onnx


mobilenet_v3_large = models.mobilenet_v3_large().eval().cuda()

x = torch.randn(8, 3, 224, 224, requires_grad=True).cuda()
torch_out = mobilenet_v3_large(x)

torch.onnx.export(mobilenet_v3_large,               # model being run
                  x,                         # model input (or a tuple for multiple inputs)
                  "mobilenet_pytorch.onnx",   # where to save the model (can be a file or file-like object)
                  export_params=True,        # store the trained parameter weights inside the model file
                  opset_version=8,          # the ONNX version to export the model to
                  do_constant_folding=True,  # whether to execute constant folding for optimization
                  input_names = ['input'],   # the model's input names
                  output_names = ['preds'], # the model's output names
                  dynamic_axes={'input' : {0 : 'batch_size'},    # variable length axes
                                'preds' : {0 : 'batch_size'}})

When I add this model to deepstream as sgie0 I see that model builds correctly:

INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [Implicit Engine Info]: layers num: 2
0   INPUT  kFLOAT input           3x224x224       
1   OUTPUT kFLOAT preds           1000      

However when running on video I can see that the model does not predict anything and obj->classifier_meta_list is always NULL. So I am assuming the output is not accepted properly or something like that.

My configs:

config.txt

[primary-gie]
enable=1
gpu-id=0
batch-size=1
gie-unique-id=1
process-mode=1
nvbuf-memory-type=0
config-file=config_infer_yolo.txt

[secondary-gie0]
enable=1
process-mode=2
gpu-id=0
batch-size=1
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_mobilenet.txt

config_infer_mobilenet.txt

[property]
gpu-id=0
net-scale-factor=1
labelfile-path=../mobilenet/labels.txt
onnx-file=../mobilenet/mobilenet_pytorch.onnx
input-dims=3;224;224;0
model-color-format=0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
network-type=1
is-classifier=1
num-detected-classes=1000
interval=0
output-blob-names=preds
classifier-threshold=0.2

Something is not correct with my classifier even thought it builds okay, as I mentioned I assume it is the output of the network, but I am not sure how to fix this. Also when I attach different classifier the pipeline works as expected.

is your model classification model or detection model?
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html

The sgie or as seen in config_infer_mobilenet.txt is a classifier and set to network-type=1, pgie is a detector and is working fine, the only problems that I have is with the onnx converted model which is a classifier

please check if Secondary-gie use onnx model - #5 by mchi helps

Does not seem to help.

operate-on-gie-id=1 // was set on the main config file
infer-dims=3;224;224 // did not help

And all of these didn’t help:
net-scale-factor=1
offsets=…
model-color-format=0
maintain-aspect-ratio=…

I have tried various combinations. Also I don’t think the color parameters are relevant, because I can’t get my model to predict, the accuracy of the model is not in question right now.

Update:

I was actually wrong, when I set the threshold to 0 seems like it is classifying, I will continue to experiment with color parameters.

Hi @pyml ,
Thanks for the update!