Onnx to trt engine

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) xavierNX
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 7
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Opening in BLOCKING MODE
0:00:00.259453763 13135 0x24a8bf50 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1715> [UID = 1]: Trying to create engine from model files

Input filename: /home/pi/lpr/onnx-model.onnx
ONNX IR version: 0.0.5
Opset version: 10
Producer name: keras2onnx
Producer version: 1.7.0
Domain: onnxmltools
Model version: 0
Doc string:

WARNING: [TRT]: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
ERROR: [TRT]: Parameter check failed at: optimizationProfile.cpp::setDimensions::119, condition: std::all_of(dims.d, dims.d + dims.nbDims, (int x) { return x >= 0; })
ERROR: [TRT]: Parameter check failed at: optimizationProfile.cpp::setDimensions::119, condition: std::all_of(dims.d, dims.d + dims.nbDims, (int x) { return x >= 0; })
ERROR: [TRT]: Parameter check failed at: optimizationProfile.cpp::setDimensions::119, condition: std::all_of(dims.d, dims.d + dims.nbDims, (int x) { return x >= 0; })
ERROR: Explicit config dims is invalid
ERROR: Failed to configure builder options
ERROR: failed to build trt engine.
0:00:01.422767719 13135 0x24a8bf50 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1735> [UID = 1]: build engine file failed
0:00:01.423645133 13135 0x24a8bf50 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1821> [UID = 1]: build backend context failed
0:00:01.423696205 13135 0x24a8bf50 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1148> [UID = 1]: generate backend failed, check config file settings
0:00:01.423758894 13135 0x24a8bf50 WARN nvinfer gstnvinfer.cpp:809:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance
0:00:01.423784462 13135 0x24a8bf50 WARN nvinfer gstnvinfer.cpp:809:gst_nvinfer_start:<primary_gie> error: Config file path: /home/pi/lpr/onnx-config.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
** ERROR: main:655: Failed to set pipeline to PAUSED
Quitting
ERROR from primary_gie: Failed to create NvDsInferContext instance
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(809): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:
Config file path: /home/pi/lpr/onnx-config.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
App run failed

any idea? or suggestion?
this is a keras model with tf backend converted to onnx and got many errors as you see?
Any help would be appreciated.

3

Hi,

Deepstream adapts TensorRT as an inference engine.
Based on your log, the error occurs from the TensorRT itself.

Would you mind to run your model with TensorRT directly with VERBOSE enabled and share the log with us?

$ /usr/src/tensorrt/bin/trtexec --onnx=[your/onnx-model.onnx] --verbose

Thanks.

1 Like
4

[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu5 for ONNX node: Relu5
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: activation_18/Relu:0 for ONNX tensor: activation_18/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Relu5 [Relu] outputs: [activation_18/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_19 [Conv]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_18/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_19_W_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_19_B_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: conv2d_19 [Conv] inputs: [activation_18/Relu:0 → (-1, 128, -1, -1)], [conv2d_19_W_new → (128, 128, 3, 3)], [conv2d_19_B_new → (128)],
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_19 for ONNX node: conv2d_19
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 128
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output8 for ONNX tensor: convolution_output8
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: conv2d_19 [Conv] outputs: [convolution_output8 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu6 [Relu]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output8
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Relu6 [Relu] inputs: [convolution_output8 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu6 for ONNX node: Relu6
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: activation_19/Relu:0 for ONNX tensor: activation_19/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Relu6 [Relu] outputs: [activation_19/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_20 [Conv]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_19/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_20_W_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_20_B_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: conv2d_20 [Conv] inputs: [activation_19/Relu:0 → (-1, 128, -1, -1)], [conv2d_20_W_new → (128, 128, 3, 3)], [conv2d_20_B_new → (128)],
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_20 for ONNX node: conv2d_20
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 128
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output5 for ONNX tensor: convolution_output5
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: conv2d_20 [Conv] outputs: [convolution_output5 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Add2 [Add]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output5
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_18/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Add2 [Add] inputs: [convolution_output5 → (-1, 128, -1, -1)], [activation_18/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Add2 for ONNX node: Add2
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: add_7/add:0 for ONNX tensor: add_7/add:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Add2 [Add] outputs: [add_7/add:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu3 [Relu]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: add_7/add:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Relu3 [Relu] inputs: [add_7/add:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu3 for ONNX node: Relu3
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: activation_20/Relu:0 for ONNX tensor: activation_20/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Relu3 [Relu] outputs: [activation_20/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_21 [Conv]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_20/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_21_W_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_21_B_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: conv2d_21 [Conv] inputs: [activation_20/Relu:0 → (-1, 128, -1, -1)], [conv2d_21_W_new → (128, 128, 3, 3)], [conv2d_21_B_new → (128)],
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_21 for ONNX node: conv2d_21
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 128
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output6 for ONNX tensor: convolution_output6
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: conv2d_21 [Conv] outputs: [convolution_output6 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu4 [Relu]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output6
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Relu4 [Relu] inputs: [convolution_output6 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu4 for ONNX node: Relu4
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: activation_21/Relu:0 for ONNX tensor: activation_21/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Relu4 [Relu] outputs: [activation_21/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_22 [Conv]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_21/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_22_W_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_22_B_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: conv2d_22 [Conv] inputs: [activation_21/Relu:0 → (-1, 128, -1, -1)], [conv2d_22_W_new → (128, 128, 3, 3)], [conv2d_22_B_new → (128)],
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_22 for ONNX node: conv2d_22
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 128
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output3 for ONNX tensor: convolution_output3
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: conv2d_22 [Conv] outputs: [convolution_output3 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Add1 [Add]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output3
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_20/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Add1 [Add] inputs: [convolution_output3 → (-1, 128, -1, -1)], [activation_20/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Add1 for ONNX node: Add1
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: add_8/add:0 for ONNX tensor: add_8/add:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Add1 [Add] outputs: [add_8/add:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu1 [Relu]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: add_8/add:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Relu1 [Relu] inputs: [add_8/add:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu1 for ONNX node: Relu1
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: activation_22/Relu:0 for ONNX tensor: activation_22/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Relu1 [Relu] outputs: [activation_22/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_23 [Conv]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_22/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_23_W_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_23_B_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: conv2d_23 [Conv] inputs: [activation_22/Relu:0 → (-1, 128, -1, -1)], [conv2d_23_W_new → (128, 128, 3, 3)], [conv2d_23_B_new → (128)],
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_23 for ONNX node: conv2d_23
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 128
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output4 for ONNX tensor: convolution_output4
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: conv2d_23 [Conv] outputs: [convolution_output4 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu2 [Relu]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output4
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Relu2 [Relu] inputs: [convolution_output4 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu2 for ONNX node: Relu2
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: activation_23/Relu:0 for ONNX tensor: activation_23/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Relu2 [Relu] outputs: [activation_23/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_24 [Conv]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_23/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_24_W_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_24_B_new
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: conv2d_24 [Conv] inputs: [activation_23/Relu:0 → (-1, 128, -1, -1)], [conv2d_24_W_new → (128, 128, 3, 3)], [conv2d_24_B_new → (128)],
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_24 for ONNX node: conv2d_24
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 128
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output2 for ONNX tensor: convolution_output2
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: conv2d_24 [Conv] outputs: [convolution_output2 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Add [Add]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output2
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_22/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Add [Add] inputs: [convolution_output2 → (-1, 128, -1, -1)], [activation_22/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Add for ONNX node: Add
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: add_9/add:0 for ONNX tensor: add_9/add:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Add [Add] outputs: [add_9/add:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu [Relu]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: add_9/add:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Relu [Relu] inputs: [add_9/add:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu for ONNX node: Relu
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: activation_24/Relu:0 for ONNX tensor: activation_24/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Relu [Relu] outputs: [activation_24/Relu:0 → (-1, 128, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_25 [Conv]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_24/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_25/kernel:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_25/bias:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: conv2d_25 [Conv] inputs: [activation_24/Relu:0 → (-1, 128, -1, -1)], [conv2d_25/kernel:0 → (2, 128, 3, 3)], [conv2d_25/bias:0 → (2)],
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_25 for ONNX node: conv2d_25
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 2
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 2, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output for ONNX tensor: convolution_output
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: conv2d_25 [Conv] outputs: [convolution_output → (-1, 2, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_26 [Conv]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: activation_24/Relu:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_26/kernel:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_26/bias:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: conv2d_26 [Conv] inputs: [activation_24/Relu:0 → (-1, 128, -1, -1)], [conv2d_26/kernel:0 → (6, 128, 3, 3)], [conv2d_26/bias:0 → (6)],
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (-1, 128, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_26 for ONNX node: conv2d_26
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 6
[12/14/2020-07:48:45] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 6, -1, -1)
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output1 for ONNX tensor: convolution_output1
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: conv2d_26 [Conv] outputs: [convolution_output1 → (-1, 6, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Transpose1 [Transpose]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Transpose1 [Transpose] inputs: [convolution_output → (-1, 2, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Transpose1 for ONNX node: Transpose1
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: transpose_output for ONNX tensor: transpose_output
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Transpose1 [Transpose] outputs: [transpose_output → (-1, -1, -1, 2)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Transpose3 [Transpose]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output1
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Transpose3 [Transpose] inputs: [convolution_output1 → (-1, 6, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Transpose3 for ONNX node: Transpose3
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: transpose_output1 for ONNX tensor: transpose_output1
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Transpose3 [Transpose] outputs: [transpose_output1 → (-1, -1, -1, 6)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: Softmax [Softmax]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: transpose_output
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: Softmax [Softmax] inputs: [transpose_output → (-1, -1, -1, 2)],
[12/14/2020-07:48:45] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: Softmax for ONNX node: Softmax
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: conv2d_25/truediv:0 for ONNX tensor: conv2d_25/truediv:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: Softmax [Softmax] outputs: [conv2d_25/truediv:0 → (-1, -1, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:103: Parsing node: concatenate_1 [Concat]
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_25/truediv:0
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:119: Searching for input: transpose_output1
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:125: concatenate_1 [Concat] inputs: [conv2d_25/truediv:0 → (-1, -1, -1, -1)], [transpose_output1 → (-1, -1, -1, 6)],
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:141: Registering layer: concatenate_1 for ONNX node: concatenate_1
[12/14/2020-07:48:45] [V] [TRT] ImporterContext.hpp:116: Registering tensor: concatenate_1_1 for ONNX tensor: concatenate_1
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:179: concatenate_1 [Concat] outputs: [concatenate_1 → (-1, -1, -1, -1)],
[12/14/2020-07:48:45] [V] [TRT] ModelImporter.cpp:507: Marking concatenate_1_1 as output: concatenate_1
----- Parsing of ONNX model wpod0401.onnx is Done ----
[12/14/2020-07:48:45] [E] [TRT] Network has dynamic or shape inputs, but no optimization profile has been defined.
[12/14/2020-07:48:45] [E] [TRT] Network validation failed.
[12/14/2020-07:48:45] [E] Engine creation failed
[12/14/2020-07:48:45] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=wpod0401.onnx --verbose

5

Hi,

The error is caused by the Dynamic Shapes usage.
Have you set the below command in the Deepstream pipeline?

force-implicit-batch-dim=1

Or is it possible to set your onnx with a static input(pre-defined input size)?

Thanks.

6

Thank you very much. Actually, I had to use a code to redefine the batch sizes in my model and then turned it to the engine file.