Issue for DS 7.0 with Azure Custom Vision ONNX model

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Dear NV team,

I refer to this post:
https://forums.developer.nvidia.com/t/azure-customvision-onnx-model-stopped-working-in-ds-6-2/269393

We encountered the same issue when using DS 7.0 with the custom model. Will later DS version resolve this issue? Or any work around can be used, or just go back to DS 6.1.1?

Our system setup:
HW: Orin NX 16GB
Deepstream: 7.0
Custom model: trained and exported as ONNX file by Custom Vision from Microsoft Azure, used for human detection similar to PeopleNet
Issue description: Output dynamic num values

Please see the relative documents as attached.

andrew

ONNX model layer
ONNX model layer1314×830 33.7 KB

ds log file
ds log file1070×1337 330 KB

outputs
outputs499×231 8.92 KB

5

The “-1” dimension is not supported by DeepStream 7.0 and later versions. Only dynamic batch size is supported.

6

Dear Fiona,

Noted and thanks. May I say “dynamic batch size” will be supported in the future version? And in which version it will be supported? Thanks.

andrew

7

Dynamic batch size has already been supported by current version.

Your model’s batch size is fixed as 1. The “-1” of your model’s output is not considerred to be batch size.

8

Dear Fiona,

Understood, thanks for your instant reply. Our current issue looks DS could not accept dynamic dimensions value, is it? Is it caused by the custom model which was trained by Custom Vision? If using TAO it will not have this issue? If it is not caused by the model, any way to resolve this dynamic ‘num’ issue? Thanks.

andrew

9

No matter with which tool you train and generate the ONNX model, you need to change the model’s output. The first dimension will be considered as batch size, it can be dynamic. The other dimensions can’t be dynamic for DeepStream does not support it.

10

Dear Fiona,
Happy new year!

  1. We have tried to convert the model to a static shape using a Python script,
  2. Even we fixed the shapes in the ONNX file,
    but Deepstream still recognizes the model outputs as dynamic.

Do you have a standard script to do the output conversion?

Please refer to the attached config file and model files.

andrew

property.txt (1.5 KB)
Azure Custom vision compact S1.zip (9.9 MB)

11

Where and how did you generate the “/opt/nvidia/deepstream/deepstream-7.0/samples/models/custom_model/model.engine” you used in the nvinfer configuration?

12

Dear Fiona,

We use trtexec to generate model.engine from model.onnx. Now we use modified config and script files, and the model works now. Thank you very much for your great help!

andrew

14

Current DeepStream does not support the output layer dynamic shape. Please modify the model to output explicit dimension layers.

15

Sure, thanks, Fiona.

16

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.