Continuing the discussion from MaxPool: at least 5 dimensions are required for input:
Description
Is the MaxPool3D from Pytorch not supported by the TensorRT 8.2 (JetPack 32.7.4 on Jetson Nano)?
I use onnx-tensorrt / trtexec to convert the model to an engine, but it prints:
MaxPool: at least 5 dimensions are required for input
As I can see in support matrix, 3D pooling should be supported. Could you check the model from the original topic, please?
Environment
TensorRT Version: 8.2.
onnx-tensorrt version: 8.2-GA
Operating System + Version: JetPack 32.7.4
Baremetal: Jetson Nano DevKit 4GB
Steps to reproduce
$ ./onnx2trt OnnxMultiDynamicInputsTestDynamicModel.onnx -o OnnxMultiDynamicInputsTestDynamicModel.trt -b 1 -d 16
----------------------------------------------------------------
Input filename: OnnxMultiDynamicInputsTestDynamicModel.onnx
ONNX IR version: 0.0.6
Opset version: 9
Producer name: pytorch
Producer version: 1.7
Domain:
Model version: 0
Doc string:
----------------------------------------------------------------
Parsing model
[2024-05-02 16:34:42 ERROR] MaxPool_1: at least 5 dimensions are required for input.
While parsing node number 1 [MaxPool -> "12"]:
ERROR: ModelImporter.cpp:166 In function parseGraph:
[6] Invalid Node - MaxPool_1
MaxPool_1: at least 5 dimensions are required for input.
or
$ /usr/src/tensorrt/bin/trtexec --onnx=OnnxMultiDynamicInputsTestDynamicModel.onnx --verbose --iterations=100 --fp16
&&&& RUNNING TensorRT.trtexec [TensorRT v8201] # /usr/src/tensorrt/bin/trtexec --onnx=OnnxMultiDynamicInputsTestDynamicModel.onnx --verbose --iterations=100 --fp16
[05/02/2024-19:37:57] [I] === Model Options ===
[05/02/2024-19:37:57] [I] Format: ONNX
[05/02/2024-19:37:57] [I] Model: OnnxMultiDynamicInputsTestDynamicModel.onnx
[05/02/2024-19:37:57] [I] Output:
[05/02/2024-19:37:57] [I] === Build Options ===
[05/02/2024-19:37:57] [I] Max batch: explicit batch
[05/02/2024-19:37:57] [I] Workspace: 16 MiB
[05/02/2024-19:37:57] [I] minTiming: 1
[05/02/2024-19:37:57] [I] avgTiming: 8
[05/02/2024-19:37:57] [I] Precision: FP32+FP16
[05/02/2024-19:37:57] [I] Calibration:
[05/02/2024-19:37:57] [I] Refit: Disabled
[05/02/2024-19:37:57] [I] Sparsity: Disabled
[05/02/2024-19:37:57] [I] Safe mode: Disabled
[05/02/2024-19:37:57] [I] DirectIO mode: Disabled
[05/02/2024-19:37:57] [I] Restricted mode: Disabled
[05/02/2024-19:37:57] [I] Save engine:
[05/02/2024-19:37:57] [I] Load engine:
[05/02/2024-19:37:57] [I] Profiling verbosity: 0
[05/02/2024-19:37:57] [I] Tactic sources: Using default tactic sources
[05/02/2024-19:37:57] [I] timingCacheMode: local
[05/02/2024-19:37:57] [I] timingCacheFile:
[05/02/2024-19:37:57] [I] Input(s)s format: fp32:CHW
[05/02/2024-19:37:57] [I] Output(s)s format: fp32:CHW
[05/02/2024-19:37:57] [I] Input build shapes: model
[05/02/2024-19:37:57] [I] Input calibration shapes: model
[05/02/2024-19:37:57] [I] === System Options ===
[05/02/2024-19:37:57] [I] Device: 0
[05/02/2024-19:37:57] [I] DLACore:
[05/02/2024-19:37:57] [I] Plugins:
[05/02/2024-19:37:57] [I] === Inference Options ===
[05/02/2024-19:37:57] [I] Batch: Explicit
[05/02/2024-19:37:57] [I] Input inference shapes: model
[05/02/2024-19:37:57] [I] Iterations: 100
[05/02/2024-19:37:57] [I] Duration: 3s (+ 200ms warm up)
[05/02/2024-19:37:57] [I] Sleep time: 0ms
[05/02/2024-19:37:57] [I] Idle time: 0ms
[05/02/2024-19:37:57] [I] Streams: 1
[05/02/2024-19:37:57] [I] ExposeDMA: Disabled
[05/02/2024-19:37:57] [I] Data transfers: Enabled
[05/02/2024-19:37:57] [I] Spin-wait: Disabled
[05/02/2024-19:37:57] [I] Multithreading: Disabled
[05/02/2024-19:37:57] [I] CUDA Graph: Disabled
[05/02/2024-19:37:57] [I] Separate profiling: Disabled
[05/02/2024-19:37:57] [I] Time Deserialize: Disabled
[05/02/2024-19:37:57] [I] Time Refit: Disabled
[05/02/2024-19:37:57] [I] Skip inference: Disabled
[05/02/2024-19:37:57] [I] Inputs:
[05/02/2024-19:37:57] [I] === Reporting Options ===
[05/02/2024-19:37:57] [I] Verbose: Enabled
[05/02/2024-19:37:57] [I] Averages: 10 inferences
[05/02/2024-19:37:57] [I] Percentile: 99
[05/02/2024-19:37:57] [I] Dump refittable layers:Disabled
[05/02/2024-19:37:57] [I] Dump output: Disabled
[05/02/2024-19:37:57] [I] Profile: Disabled
[05/02/2024-19:37:57] [I] Export timing to JSON file:
[05/02/2024-19:37:57] [I] Export output to JSON file:
[05/02/2024-19:37:57] [I] Export profile to JSON file:
[05/02/2024-19:37:57] [I]
[05/02/2024-19:37:57] [I] === Device Information ===
[05/02/2024-19:37:57] [I] Selected Device: NVIDIA Tegra X1
[05/02/2024-19:37:57] [I] Compute Capability: 5.3
[05/02/2024-19:37:57] [I] SMs: 1
[05/02/2024-19:37:57] [I] Compute Clock Rate: 0.9216 GHz
[05/02/2024-19:37:57] [I] Device Global Memory: 3963 MiB
[05/02/2024-19:37:57] [I] Shared Memory per SM: 64 KiB
[05/02/2024-19:37:57] [I] Memory Bus Width: 64 bits (ECC disabled)
[05/02/2024-19:37:57] [I] Memory Clock Rate: 0.01275 GHz
[05/02/2024-19:37:57] [I]
[05/02/2024-19:37:57] [I] TensorRT version: 8.2.1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::EfficientNMS_TFTRT_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::Proposal version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::Split version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[05/02/2024-19:37:57] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[05/02/2024-19:37:59] [I] [TRT] [MemUsageChange] Init CUDA: CPU +229, GPU +0, now: CPU 248, GPU 2901 (MiB)
[05/02/2024-19:38:00] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 248 MiB, GPU 2901 MiB
[05/02/2024-19:38:00] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 278 MiB, GPU 2930 MiB
[05/02/2024-19:38:00] [I] Start parsing network model
[05/02/2024-19:38:00] [I] [TRT] ----------------------------------------------------------------
[05/02/2024-19:38:00] [I] [TRT] Input filename: OnnxMultiDynamicInputsTestDynamicModel.onnx
[05/02/2024-19:38:00] [I] [TRT] ONNX IR version: 0.0.6
[05/02/2024-19:38:00] [I] [TRT] Opset version: 9
[05/02/2024-19:38:00] [I] [TRT] Producer name: pytorch
[05/02/2024-19:38:00] [I] [TRT] Producer version: 1.7
[05/02/2024-19:38:00] [I] [TRT] Domain:
[05/02/2024-19:38:00] [I] [TRT] Model version: 0
[05/02/2024-19:38:00] [I] [TRT] Doc string:
[05/02/2024-19:38:00] [I] [TRT] ----------------------------------------------------------------
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::ScatterND version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TFTRT_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::Split version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[05/02/2024-19:38:00] [V] [TRT] Adding network input: my_input1 with dtype: float32, dimensions: (-1, -1, -1, -1)
[05/02/2024-19:38:00] [V] [TRT] Registering tensor: my_input1 for ONNX tensor: my_input1
[05/02/2024-19:38:00] [V] [TRT] Adding network input: my_input2 with dtype: float32, dimensions: (-1, -1, -1, -1)
[05/02/2024-19:38:00] [V] [TRT] Registering tensor: my_input2 for ONNX tensor: my_input2
[05/02/2024-19:38:00] [V] [TRT] Importing initializer: conv1.bias
[05/02/2024-19:38:00] [V] [TRT] Importing initializer: conv1.weight
[05/02/2024-19:38:00] [V] [TRT] Parsing node: Conv_0 [Conv]
[05/02/2024-19:38:00] [V] [TRT] Searching for input: my_input1
[05/02/2024-19:38:00] [V] [TRT] Searching for input: conv1.weight
[05/02/2024-19:38:00] [V] [TRT] Searching for input: conv1.bias
[05/02/2024-19:38:00] [V] [TRT] Conv_0 [Conv] inputs: [my_input1 -> (-1, -1, -1, -1)[FLOAT]], [conv1.weight -> (32, 32, 3, 3)[FLOAT]], [conv1.bias -> (32)[FLOAT]],
[05/02/2024-19:38:00] [V] [TRT] Convolution input dimensions: (-1, -1, -1, -1)
[05/02/2024-19:38:00] [V] [TRT] Registering layer: Conv_0 for ONNX node: Conv_0
[05/02/2024-19:38:00] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 32
[05/02/2024-19:38:00] [V] [TRT] Convolution output dimensions: (-1, 32, -1, -1)
[05/02/2024-19:38:00] [V] [TRT] Registering tensor: 11 for ONNX tensor: 11
[05/02/2024-19:38:00] [V] [TRT] Conv_0 [Conv] outputs: [11 -> (-1, 32, -1, -1)[FLOAT]],
[05/02/2024-19:38:00] [V] [TRT] Parsing node: MaxPool_1 [MaxPool]
[05/02/2024-19:38:00] [V] [TRT] Searching for input: my_input2
[05/02/2024-19:38:00] [V] [TRT] MaxPool_1 [MaxPool] inputs: [my_input2 -> (-1, -1, -1, -1)[FLOAT]],
[05/02/2024-19:38:00] [V] [TRT] Registering layer: MaxPool_1 for ONNX node: MaxPool_1
[05/02/2024-19:38:00] [E] Error[3]: MaxPool_1: at least 5 dimensions are required for input.
[05/02/2024-19:38:00] [E] [TRT] ModelImporter.cpp:773: While parsing node number 1 [MaxPool -> "12"]:
[05/02/2024-19:38:00] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[05/02/2024-19:38:00] [E] [TRT] ModelImporter.cpp:775: input: "my_input2"
output: "12"
name: "MaxPool_1"
op_type: "MaxPool"
attribute {
name: "kernel_shape"
ints: 2
ints: 2
ints: 2
type: INTS
}
attribute {
name: "pads"
ints: 0
ints: 0
ints: 0
ints: 0
ints: 0
ints: 0
type: INTS
}
attribute {
name: "strides"
ints: 2
ints: 2
ints: 2
type: INTS
}
[05/02/2024-19:38:00] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[05/02/2024-19:38:00] [E] [TRT] ModelImporter.cpp:779: ERROR: ModelImporter.cpp:166 In function parseGraph:
[6] Invalid Node - MaxPool_1
MaxPool_1: at least 5 dimensions are required for input.
[05/02/2024-19:38:00] [E] Failed to parse onnx file
[05/02/2024-19:38:00] [I] Finish parsing network model
[05/02/2024-19:38:00] [E] Parsing model failed
[05/02/2024-19:38:00] [E] Failed to create engine from model.
[05/02/2024-19:38:00] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8201] # /usr/src/tensorrt/bin/trtexec --onnx=OnnxMultiDynamicInputsTestDynamicModel.onnx --verbose --iterations=100 --fp16