TensorRT Version: 7.0
GPU Type: titan xp
Nvidia Driver Version: 384.130
CUDA Version: 9.0
CUDNN Version: 7.6
Operating System + Version: ubuntu 16.04
Python Version (if applicable): 3.7
PyTorch Version (if applicable): 1.0.0
Baremetal or Container (if container which image + tag):
[b]I encounter one problem whenI use conv3d dynamic shape.
I set input shape as [-1, 1, -1, -1, -1]. when I build engineering, it will appear an error.[/b]
ONNX IR version: 0.0.3
Opset version: 9
Producer name: pytorch
Producer version: 0.4
Domain:
Model version: 0
Doc string:
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::GridAnchor_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::NMS_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Reorg_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Region_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Clip_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::LReLU_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::PriorBox_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Normalize_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::RPROI_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::BatchedNMS_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::FlattenConcat_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::CropAndResize
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::DetectionLayer_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Proposal
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::ProposalLayer_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::PyramidROIAlign_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::ResizeNearest_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Split
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::SpecialSlice_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::InstanceNormalization_TRT
ModelImporter.cpp:203: Adding network input: 0 with dtype: float32, dimensions: (-1, 1, -1, -1, -1)
ImporterContext.hpp:97: Registering tensor: 0 for ONNX tensor: 0
ModelImporter.cpp:90: Importing initializer: 1
ModelImporter.cpp:90: Importing initializer: 2
ModelImporter.cpp:90: Importing initializer: 3
ModelImporter.cpp:90: Importing initializer: 4
ModelImporter.cpp:90: Importing initializer: 5
ModelImporter.cpp:90: Importing initializer: 6
ModelImporter.cpp:90: Importing initializer: 7
onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
ModelImporter.cpp:107: Parsing node: [Conv]
ModelImporter.cpp:123: Searching for input: 0
ModelImporter.cpp:123: Searching for input: 1
ModelImporter.cpp:123: Searching for input: 2
ModelImporter.cpp:129: [Conv] inputs: [0 → (-1, 1, -1, -1, -1)], [1 → (16, 1, 3, 3, 3)], [2 → (16)],
builtin_op_importers.cpp:442: Convolution input dimensions: (-1, 1, -1, -1, -1)
builtin_op_importers.cpp:524: Using kernel: (3, 3, 3), strides: (1, 1, 1), padding: (1, 1, 1), dilations: (1, 1, 1), numOutputs: 16
builtin_op_importers.cpp:525: Convolution output dimensions: (-1, 16, -1, -1, -1)
ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 0) [Convolution] for ONNX node:
ImporterContext.hpp:97: Registering tensor: 8 for ONNX tensor: 8
ModelImporter.cpp:180: [Conv] outputs: [8 → (-1, 16, -1, -1, -1)],
ModelImporter.cpp:107: Parsing node: [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 8
ModelImporter.cpp:123: Searching for input: 3
ModelImporter.cpp:123: Searching for input: 4
ModelImporter.cpp:123: Searching for input: 5
ModelImporter.cpp:123: Searching for input: 6
ModelImporter.cpp:129: [BatchNormalization] inputs: [8 → (-1, 16, -1, -1, -1)], [3 → (16)], [4 → (16)], [5 → (16)], [6 → (16)],
ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 1) [Scale] for ONNX node:
ImporterContext.hpp:97: Registering tensor: 9 for ONNX tensor: 9
ModelImporter.cpp:180: [BatchNormalization] outputs: [9 → (-1, 16, -1, -1, -1)],
ModelImporter.cpp:107: Parsing node: [Relu]
ModelImporter.cpp:123: Searching for input: 9
ModelImporter.cpp:129: [Relu] inputs: [9 → (-1, 16, -1, -1, -1)],
ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 2) [Activation] for ONNX node:
ImporterContext.hpp:97: Registering tensor: 10_1 for ONNX tensor: 10
ModelImporter.cpp:180: [Relu] outputs: [10 → (-1, 16, -1, -1, -1)],
ModelImporter.cpp:494: Marking 10_1 as output: 10
Build engineering
Setting layouts of network and plugin input/output tensors to linear, as 3D operators are found and 3D non-linear IO formats are not supported, yet.
Applying generic optimizations to the graph for inference.
Original: 3 layers
After dead-layer removal: 3 layers
After Myelin optimization: 3 layers
Fusing convolution weights from (Unnamed Layer* 0) [Convolution] with scale (Unnamed Layer* 1) [Scale]
After scale fusion: 2 layers
Fusing (Unnamed Layer* 0) [Convolution] with (Unnamed Layer* 2) [Activation]
After vertical fusions: 1 layers
After final dead-layer removal: 1 layers
After tensor merging: 1 layers
After concat removal: 1 layers
Graph construction and optimization completed in 0.000405013 seconds.
Constructing optimization profile number 0 out of 1
*************** Autotuning format combination: Float(1,(# 4 (SHAPE 0)),(* (# 4 (SHAPE 0)) (# 3 (SHAPE 0))),(* (# 2 (SHAPE 0)) (* (# 4 (SHAPE 0)) (# 3 (SHAPE 0)))),(* (# 2 (SHAPE 0)) (* (# 4 (SHAPE 0)) (# 3 (SHAPE 0))))) → Float(1,(# 4 (SHAPE 0)),(* (# 4 (SHAPE 0)) (# 3 (SHAPE 0))),(* (# 2 (SHAPE 0)) (* (# 4 (SHAPE 0)) (# 3 (SHAPE 0)))),(* 16 (* (# 2 (SHAPE 0)) (* (# 4 (SHAPE 0)) (# 3 (SHAPE 0)))))) ***************
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (LegacySASSConvolution)
LegacySASSConvolution has no valid tactics for this config, skipping
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution)
FusedConvActConvolution has no valid tactics for this config, skipping
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution)
CaskConvolution has no valid tactics for this config, skipping
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution)
terminate called after throwing an instance of ‘std::bad_alloc’
what(): std::bad_alloc
The program has unexpectedly finished.
But when I set input shape as [-1, 1, 96, -1, -1], it works!!!
ONNX IR version: 0.0.3
Opset version: 9
Producer name: pytorch
Producer version: 0.4
Domain:
Model version: 0
Doc string:
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::GridAnchor_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::NMS_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Reorg_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Region_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Clip_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::LReLU_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::PriorBox_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Normalize_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::RPROI_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::BatchedNMS_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::FlattenConcat_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::CropAndResize
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::DetectionLayer_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Proposal
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::ProposalLayer_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::PyramidROIAlign_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::ResizeNearest_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Split
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::SpecialSlice_TRT
Plugin creator registration succeeded - ONNXTRT_NAMESPACE::InstanceNormalization_TRT
ModelImporter.cpp:203: Adding network input: 0 with dtype: float32, dimensions: (-1, 1, 96, -1, -1)
ImporterContext.hpp:97: Registering tensor: 0 for ONNX tensor: 0
ModelImporter.cpp:90: Importing initializer: 1
ModelImporter.cpp:90: Importing initializer: 2
ModelImporter.cpp:90: Importing initializer: 3
ModelImporter.cpp:90: Importing initializer: 4
ModelImporter.cpp:90: Importing initializer: 5
ModelImporter.cpp:90: Importing initializer: 6
ModelImporter.cpp:90: Importing initializer: 7
onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
ModelImporter.cpp:107: Parsing node: [Conv]
ModelImporter.cpp:123: Searching for input: 0
ModelImporter.cpp:123: Searching for input: 1
ModelImporter.cpp:123: Searching for input: 2
ModelImporter.cpp:129: [Conv] inputs: [0 → (-1, 1, 96, -1, -1)], [1 → (16, 1, 3, 3, 3)], [2 → (16)],
builtin_op_importers.cpp:442: Convolution input dimensions: (-1, 1, 96, -1, -1)
builtin_op_importers.cpp:524: Using kernel: (3, 3, 3), strides: (1, 1, 1), padding: (1, 1, 1), dilations: (1, 1, 1), numOutputs: 16
builtin_op_importers.cpp:525: Convolution output dimensions: (-1, 16, 96, -1, -1)
ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 0) [Convolution] for ONNX node:
ImporterContext.hpp:97: Registering tensor: 8 for ONNX tensor: 8
ModelImporter.cpp:180: [Conv] outputs: [8 → (-1, 16, 96, -1, -1)],
ModelImporter.cpp:107: Parsing node: [BatchNormalization]
ModelImporter.cpp:123: Searching for input: 8
ModelImporter.cpp:123: Searching for input: 3
ModelImporter.cpp:123: Searching for input: 4
ModelImporter.cpp:123: Searching for input: 5
ModelImporter.cpp:123: Searching for input: 6
ModelImporter.cpp:129: [BatchNormalization] inputs: [8 → (-1, 16, 96, -1, -1)], [3 → (16)], [4 → (16)], [5 → (16)], [6 → (16)],
ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 1) [Scale] for ONNX node:
ImporterContext.hpp:97: Registering tensor: 9 for ONNX tensor: 9
ModelImporter.cpp:180: [BatchNormalization] outputs: [9 → (-1, 16, 96, -1, -1)],
ModelImporter.cpp:107: Parsing node: [Relu]
ModelImporter.cpp:123: Searching for input: 9
ModelImporter.cpp:129: [Relu] inputs: [9 → (-1, 16, 96, -1, -1)],
ImporterContext.hpp:122: Registering layer: (Unnamed Layer* 2) [Activation] for ONNX node:
ImporterContext.hpp:97: Registering tensor: 10_1 for ONNX tensor: 10
ModelImporter.cpp:180: [Relu] outputs: [10 → (-1, 16, 96, -1, -1)],
ModelImporter.cpp:494: Marking 10_1 as output: 10
Build engineering
Setting layouts of network and plugin input/output tensors to linear, as 3D operators are found and 3D non-linear IO formats are not supported, yet.
Applying generic optimizations to the graph for inference.
Original: 3 layers
After dead-layer removal: 3 layers
After Myelin optimization: 3 layers
Fusing convolution weights from (Unnamed Layer* 0) [Convolution] with scale (Unnamed Layer* 1) [Scale]
After scale fusion: 2 layers
Fusing (Unnamed Layer* 0) [Convolution] with (Unnamed Layer* 2) [Activation]
After vertical fusions: 1 layers
After final dead-layer removal: 1 layers
After tensor merging: 1 layers
After concat removal: 1 layers
Graph construction and optimization completed in 0.000511656 seconds.
Constructing optimization profile number 0 out of 1
*************** Autotuning format combination: Float(1,(# 4 (SHAPE 0)),(* (# 4 (SHAPE 0)) (# 3 (SHAPE 0))),(* 96 (* (# 4 (SHAPE 0)) (# 3 (SHAPE 0)))),(* 96 (* (# 4 (SHAPE 0)) (# 3 (SHAPE 0))))) → Float(1,(# 4 (SHAPE 0)),(* (# 4 (SHAPE 0)) (# 3 (SHAPE 0))),(* 96 (* (# 4 (SHAPE 0)) (# 3 (SHAPE 0)))),(* 1536 (* (# 4 (SHAPE 0)) (# 3 (SHAPE 0))))) ***************
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (LegacySASSConvolution)
LegacySASSConvolution has no valid tactics for this config, skipping
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution)
FusedConvActConvolution has no valid tactics for this config, skipping
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution)
CaskConvolution has no valid tactics for this config, skipping
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution)
Tactic: 0 time 1.71642
Tactic: 1 time 2.34291
Tactic: 5 time 73.0613
Fastest Tactic: 0 Time: 1.71642
--------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution)
CudaDepthwiseConvolution has no valid tactics for this config, skipping
Chose Runner Type: CudaConvolution Tactic: 0
Formats and tactics selection completed in 0.261358 seconds.
After reformat layers: 1 layers
Block size 1073741824
Total Activation Memory: 1073741824
Engine generation completed in 1.58876 seconds.
Engine Layer Information:
Layer(Convolution): (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation], Tactic: 0, 0[Float(1,96,-2147483643,-2147483645)] → 10[Float(16,96,-2147483643,-2147483645)]
Finish build engineering
Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
Forward success !
[b]So I find something in TensorRT-Developer-Guid.pfd.
‣ IConvolutionLayer and IDeconvolutionLayer require that the channel dimension be a build-time constant
it shows that I need set channel as constant, but it need set depth as constant too!!!
Is it bug???[/b]
models.7z (5.29 KB)