Can't build a dynamic shapes network

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I’m trying to build a network with dynamic shapes, but it keeps failing, giving me the error:
../builder/symbolicDims.cpp (649) - Assertion Error in fromSymbolic: 0 (x must be build-time constant)

Environment

using tensorrt 20.11 docker image ( nvcr.io/nvidia/tensorrt:20.11-py3 )

Relevant Files

root@e5e2a4db1bc3:/workspace# /usr/src/tensorrt/bin/trtexec --explicitBatch --onnx=model.onnx --minShapes=input:1x1x1539 --optShapes=input:1x200x1539 --maxShapes=input:1x500x1539 --shapes=input:1x250x1539 --verbose
&&&& RUNNING TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --explicitBatch --onnx=model.onnx --minShapes=input:1x1x1539 --optShapes=input:1x200x1539 --maxShapes=input:1x500x1539 --shapes=input:1x250x1539 --verbose
[10/21/2021-11:10:13] [I] === Model Options ===
[10/21/2021-11:10:13] [I] Format: ONNX
[10/21/2021-11:10:13] [I] Model: model.onnx
[10/21/2021-11:10:13] [I] Output:
[10/21/2021-11:10:13] [I] === Build Options ===
[10/21/2021-11:10:13] [I] Max batch: explicit
[10/21/2021-11:10:13] [I] Workspace: 16 MiB
[10/21/2021-11:10:13] [I] minTiming: 1
[10/21/2021-11:10:13] [I] avgTiming: 8
[10/21/2021-11:10:13] [I] Precision: FP32
[10/21/2021-11:10:13] [I] Calibration: 
[10/21/2021-11:10:13] [I] Refit: Disabled
[10/21/2021-11:10:13] [I] Safe mode: Disabled
[10/21/2021-11:10:13] [I] Save engine: 
[10/21/2021-11:10:13] [I] Load engine: 
[10/21/2021-11:10:13] [I] Builder Cache: Enabled
[10/21/2021-11:10:13] [I] NVTX verbosity: 0
[10/21/2021-11:10:13] [I] Tactic sources: Using default tactic sources
[10/21/2021-11:10:13] [I] Input(s)s format: fp32:CHW
[10/21/2021-11:10:13] [I] Output(s)s format: fp32:CHW
[10/21/2021-11:10:13] [I] Input build shape: input=1x1x1539+1x200x1539+1x500x1539
[10/21/2021-11:10:13] [I] Input calibration shapes: model
[10/21/2021-11:10:13] [I] === System Options ===
[10/21/2021-11:10:13] [I] Device: 0
[10/21/2021-11:10:13] [I] DLACore: 
[10/21/2021-11:10:13] [I] Plugins:
[10/21/2021-11:10:13] [I] === Inference Options ===
[10/21/2021-11:10:13] [I] Batch: Explicit
[10/21/2021-11:10:13] [I] Input inference shape: input=1x250x1539
[10/21/2021-11:10:13] [I] Iterations: 10
[10/21/2021-11:10:13] [I] Duration: 3s (+ 200ms warm up)
[10/21/2021-11:10:13] [I] Sleep time: 0ms
[10/21/2021-11:10:13] [I] Streams: 1
[10/21/2021-11:10:13] [I] ExposeDMA: Disabled
[10/21/2021-11:10:13] [I] Data transfers: Enabled
[10/21/2021-11:10:13] [I] Spin-wait: Disabled
[10/21/2021-11:10:13] [I] Multithreading: Disabled
[10/21/2021-11:10:13] [I] CUDA Graph: Disabled
[10/21/2021-11:10:13] [I] Separate profiling: Disabled
[10/21/2021-11:10:13] [I] Skip inference: Disabled
[10/21/2021-11:10:13] [I] Inputs:
[10/21/2021-11:10:13] [I] === Reporting Options ===
[10/21/2021-11:10:13] [I] Verbose: Enabled
[10/21/2021-11:10:13] [I] Averages: 10 inferences
[10/21/2021-11:10:13] [I] Percentile: 99
[10/21/2021-11:10:13] [I] Dump refittable layers:Disabled
[10/21/2021-11:10:13] [I] Dump output: Disabled
[10/21/2021-11:10:13] [I] Profile: Disabled
[10/21/2021-11:10:13] [I] Export timing to JSON file: 
[10/21/2021-11:10:13] [I] Export output to JSON file: 
[10/21/2021-11:10:13] [I] Export profile to JSON file: 
[10/21/2021-11:10:13] [I] 
[10/21/2021-11:10:13] [I] === Device Information ===
[10/21/2021-11:10:13] [I] Selected Device: GeForce GTX 1070 Ti
[10/21/2021-11:10:13] [I] Compute Capability: 6.1
[10/21/2021-11:10:13] [I] SMs: 19
[10/21/2021-11:10:13] [I] Compute Clock Rate: 1.683 GHz
[10/21/2021-11:10:13] [I] Device Global Memory: 8116 MiB
[10/21/2021-11:10:13] [I] Shared Memory per SM: 96 KiB
[10/21/2021-11:10:13] [I] Memory Bus Width: 256 bits (ECC disabled)
[10/21/2021-11:10:13] [I] Memory Clock Rate: 4.004 GHz
[10/21/2021-11:10:13] [I] 
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::Proposal version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[10/21/2021-11:10:13] [V] [TRT] Registered plugin creator - ::Split version 1
----------------------------------------------------------------
Input filename:   model.onnx
ONNX IR version:  0.0.6
Opset version:    9
Producer name:    pytorch
Producer version: 1.7
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[10/21/2021-11:10:25] [V] [TRT] Plugin creator already registered - ::Split version 1
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:206: Adding network input: input with dtype: float32, dimensions: (1, -1, 1539)
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: input for ONNX tensor: input
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: 34
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: 35
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: 36
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: 37
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: 38
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: _classifier.0.bias
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: _classifier.1.bias
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:90: Importing initializer: _mil._inf
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Constant_0 [Constant]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Constant_0 [Constant] inputs: 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Constant_0 [Constant] outputs: [9 -> ()], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Equal_1 [Equal]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: input
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 9
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Equal_1 [Equal] inputs: [input -> (1, -1, 1539)], [9 -> ()], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: 9 for ONNX initializer: 9
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Equal_1 for ONNX node: Equal_1
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 10 for ONNX tensor: 10
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Equal_1 [Equal] outputs: [10 -> (1, -1, 1539)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Not_2 [Not]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 10
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Not_2 [Not] inputs: [10 -> (1, -1, 1539)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Not_2 for ONNX node: Not_2
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 11 for ONNX tensor: 11
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Not_2 [Not] outputs: [11 -> (1, -1, 1539)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Cast_3 [Cast]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 11
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Cast_3 [Cast] inputs: [11 -> (1, -1, 1539)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/builtin_op_importers.cpp:320: Casting to type: int32
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Cast_3 for ONNX node: Cast_3
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 12 for ONNX tensor: 12
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Cast_3 [Cast] outputs: [12 -> (1, -1, 1539)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: ReduceSum_4 [ReduceSum]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 12
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: ReduceSum_4 [ReduceSum] inputs: [12 -> (1, -1, 1539)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: ReduceSum_4 for ONNX node: ReduceSum_4
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 13 for ONNX tensor: 13
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: ReduceSum_4 [ReduceSum] outputs: [13 -> (1, -1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Constant_5 [Constant]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Constant_5 [Constant] inputs: 
[10/21/2021-11:10:25] [W] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Constant_5 [Constant] outputs: [14 -> ()], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Greater_6 [Greater]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 13
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 14
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Greater_6 [Greater] inputs: [13 -> (1, -1)], [14 -> ()], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: 14 for ONNX initializer: 14
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Greater_6 for ONNX node: Greater_6
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 15 for ONNX tensor: 15
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Greater_6 [Greater] outputs: [15 -> (1, -1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Unsqueeze_7 [Unsqueeze]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 15
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Unsqueeze_7 [Unsqueeze] inputs: [15 -> (1, -1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:1815: Original shape: (1, _), unsqueezing to: (_, _, _)
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Unsqueeze_7 for ONNX node: Unsqueeze_7
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 16 for ONNX tensor: 16
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Unsqueeze_7 [Unsqueeze] outputs: [16 -> (-1, -1, -1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: MatMul_8 [MatMul]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: input
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 34
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: MatMul_8 [MatMul] inputs: [input -> (1, -1, 1539)], [34 -> (1539, 128)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: 34 for ONNX initializer: 34
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: MatMul_8 for ONNX node: MatMul_8
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 18 for ONNX tensor: 18
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: MatMul_8 [MatMul] outputs: [18 -> (1, -1, 128)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: MatMul_9 [MatMul]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: input
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 35
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: MatMul_9 [MatMul] inputs: [input -> (1, -1, 1539)], [35 -> (1539, 128)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: 35 for ONNX initializer: 35
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: MatMul_9 for ONNX node: MatMul_9
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 20 for ONNX tensor: 20
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: MatMul_9 [MatMul] outputs: [20 -> (1, -1, 128)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Mul_10 [Mul]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 18
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 20
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Mul_10 [Mul] inputs: [18 -> (1, -1, 128)], [20 -> (1, -1, 128)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Mul_10 for ONNX node: Mul_10
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 21 for ONNX tensor: 21
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Mul_10 [Mul] outputs: [21 -> (1, -1, 128)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: MatMul_11 [MatMul]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 21
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 36
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: MatMul_11 [MatMul] inputs: [21 -> (1, -1, 128)], [36 -> (128, 1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: 36 for ONNX initializer: 36
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: MatMul_11 for ONNX node: MatMul_11
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 23 for ONNX tensor: 23
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: MatMul_11 [MatMul] outputs: [23 -> (1, -1, 1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Transpose_12 [Transpose]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 23
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Transpose_12 [Transpose] inputs: [23 -> (1, -1, 1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Transpose_12 for ONNX node: Transpose_12
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 24 for ONNX tensor: 24
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Transpose_12 [Transpose] outputs: [24 -> (1, 1, -1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Where_13 [Where]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 16
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 24
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: _mil._inf
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Where_13 [Where] inputs: [16 -> (-1, -1, -1)], [24 -> (1, 1, -1)], [_mil._inf -> ()], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: _mil._inf for ONNX initializer: _mil._inf
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Where_13 for ONNX node: Where_13
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 25 for ONNX tensor: 25
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Where_13 [Where] outputs: [25 -> (-1, -1, -1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Softmax_14 [Softmax]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 25
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Softmax_14 [Softmax] inputs: [25 -> (-1, -1, -1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Softmax_14 for ONNX node: Softmax_14
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 26 for ONNX tensor: 26
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Softmax_14 [Softmax] outputs: [26 -> (-1, -1, -1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: MatMul_15 [MatMul]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 26
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: input
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: MatMul_15 [MatMul] inputs: [26 -> (-1, -1, -1)], [input -> (1, -1, 1539)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: MatMul_15 for ONNX node: MatMul_15
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 27 for ONNX tensor: 27
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: MatMul_15 [MatMul] outputs: [27 -> (-1, -1, 1539)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: MatMul_16 [MatMul]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 27
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 37
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: MatMul_16 [MatMul] inputs: [27 -> (-1, -1, 1539)], [37 -> (1539, 512)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: 37 for ONNX initializer: 37
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: MatMul_16 for ONNX node: MatMul_16
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 29 for ONNX tensor: 29
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: MatMul_16 [MatMul] outputs: [29 -> (-1, -1, 512)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Add_17 [Add]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 29
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: _classifier.0.bias
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Add_17 [Add] inputs: [29 -> (-1, -1, 512)], [_classifier.0.bias -> (512)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: _classifier.0.bias for ONNX initializer: _classifier.0.bias
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Add_17 for ONNX node: Add_17
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 30 for ONNX tensor: 30
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Add_17 [Add] outputs: [30 -> (-1, -1, 512)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: MatMul_18 [MatMul]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 30
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 38
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: MatMul_18 [MatMul] inputs: [30 -> (-1, -1, 512)], [38 -> (512, 1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: 38 for ONNX initializer: 38
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: MatMul_18 for ONNX node: MatMul_18
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: 32 for ONNX tensor: 32
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: MatMul_18 [MatMul] outputs: [32 -> (-1, -1, 1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:107: Parsing node: Add_19 [Add]
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: 32
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:123: Searching for input: _classifier.1.bias
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:129: Add_19 [Add] inputs: [32 -> (-1, -1, 1)], [_classifier.1.bias -> (1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:150: Registering constant layer: _classifier.1.bias for ONNX initializer: _classifier.1.bias
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:154: Registering layer: Add_19 for ONNX node: Add_19
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ImporterContext.hpp:120: Registering tensor: class_0 for ONNX tensor: class
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:183: Add_19 [Add] outputs: [class -> (-1, -1, 1)], 
[10/21/2021-11:10:25] [V] [TRT] /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/ModelImporter.cpp:538: Marking class_0 as output: class
 ----- Parsing of ONNX model model.onnx is Done ---- 
[10/21/2021-11:10:25] [V] [TRT] Applying generic optimizations to the graph for inference.
[10/21/2021-11:10:25] [V] [TRT] Original: 40 layers
[10/21/2021-11:10:25] [V] [TRT] After dead-layer removal: 40 layers
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing 9 with (Unnamed Layer* 1) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing 14 with (Unnamed Layer* 7) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing 34 with (Unnamed Layer* 16) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing 35 with (Unnamed Layer* 19) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing 36 with (Unnamed Layer* 23) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing _mil._inf with (Unnamed Layer* 27) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing 37 with (Unnamed Layer* 44) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing _classifier.0.bias with (Unnamed Layer* 47) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing 38 with (Unnamed Layer* 50) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] BinaryFusion: Fusing _classifier.1.bias with (Unnamed Layer* 53) [Shuffle]
[10/21/2021-11:10:25] [V] [TRT] After Myelin optimization: 15 layers
[10/21/2021-11:10:25] [V] [TRT] After scale fusion: 15 layers
[10/21/2021-11:10:25] [F] [TRT] Assertion failed: x must be build-time constant
../builder/symbolicDims.cpp:649
Aborting...
[10/21/2021-11:10:25] [E] [TRT] ../builder/symbolicDims.cpp (649) - Assertion Error in fromSymbolic: 0 (x must be build-time constant)
[10/21/2021-11:10:25] [E] Engine creation failed
[10/21/2021-11:10:25] [E] Engine set up failed
2

Hi,
Please refer to below links related custom plugin implementation and sample:

While IPluginV2 and IPluginV2Ext interfaces are still supported for backward compatibility with TensorRT 5.1 and 6.0.x respectively, however, we recommend that you write new plugins or refactor existing ones to target the IPluginV2DynamicExt or IPluginV2IOExt interfaces instead.

Thanks!

4

Sorry, i thinks i’ve missed something! AFAIK, i don’t use any custom layers or plugins in this network, it’s really straightforward! here is a netron output of my network.

model.onnx
model.onnx754×1692 54.4 KB

5

Hi,

Could you please try on latest TensorRT version if you still face this issue, we recommend you to please share issue repro onnx model and trtexec --verbose logs to try from our end for better debugging.

Thank you.

1 Like
6

thanks, latest tensorrt fixed the issue.

1 Like