Trtexec Error: only activation types allowed as input to this layer

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Environment

TensorRT Version: 8.5.1.7
GPU Type: NVIDIA T1200 Laptop GPU
Nvidia Driver Version: 522.06
CUDA Version: 11.8
CUDNN Version: 8.2
Operating System + Version: Windows10
PyTorch Version (if applicable): 2.0.0

Description

Hello, I am new to tensorrt . When I try to export an onnx model to tensorrt model, it crashed. Here is the error message.

[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear_1 [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_1 [DequantizeLinear] inputs: [/0_34/Cast_3_output_0 -> (32, 3, 3, 3)[INT8]], [/0_34/Cast_4_output_0 -> (32)[FLOAT]], [/0_34/Cast_5_output_0 -> (32)[INT8]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_3_output_0 for ONNX node: /0_34/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_4_output_0 for ONNX node: /0_34/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_5_output_0 for ONNX node: /0_34/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering tensor: /0_34/DequantizeLinear_1_output_0 for ONNX tensor: /0_34/DequantizeLinear_1_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_1 [DequantizeLinear] outputs: [/0_34/DequantizeLinear_1_output_0 -> (32, 3, 3, 3)[FLOAT]],
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear_2 [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_2 [DequantizeLinear] inputs: [/0_34/Cast_7_output_0 -> (32)[INT32]], [/0_34/Cast_8_output_0 -> (32)[FLOAT]], [/0_34/ConstantOfShape_output_0 -> (32)[INT32]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_7_output_0 for ONNX node: /0_34/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_8_output_0 for ONNX node: /0_34/Cast_8_output_0
[05/24/2023-16:08:28] [E] Error[3]: /0_34/DequantizeLinear_2: only activation types allowed as input to this layer.
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:726: While parsing node number 3 [DequantizeLinear -> "/0_34/DequantizeLinear_2_output_0"]:
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:728: input: "/0_34/Cast_7_output_0"
input: "/0_34/Cast_8_output_0"
input: "/0_34/ConstantOfShape_output_0"
output: "/0_34/DequantizeLinear_2_output_0"
name: "/0_34/DequantizeLinear_2"
op_type: "DequantizeLinear"
attribute {
  name: "axis"
  i: 0
  type: INT
}

process

I have carried out int8 quantization operation for this origin model by pytorch(torch.ao.quantization)，Then I convert the model to onnx format. When I use trtexec to convert onnx2engine, an error is reported. The onnx model node where the error occurs is as follows:

image
image1151×798 33.9 KB

The full command and message is as follows

trtexec --onnx=../weights/quantize_onnx_sim.onnx --int8 --saveEngine=../weights/quantize_onnx_sim.engine --verbose


[05/24/2023-16:08:25] [I] === Model Options ===
[05/24/2023-16:08:25] [I] Format: ONNX
[05/24/2023-16:08:25] [I] Model: ../weights/quantize_onnx_sim.onnx
[05/24/2023-16:08:25] [I] Output:
[05/24/2023-16:08:25] [I] === Build Options ===
[05/24/2023-16:08:25] [I] Max batch: explicit batch
[05/24/2023-16:08:25] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[05/24/2023-16:08:25] [I] minTiming: 1
[05/24/2023-16:08:25] [I] avgTiming: 8
[05/24/2023-16:08:25] [I] Precision: FP32+INT8
[05/24/2023-16:08:25] [I] LayerPrecisions:
[05/24/2023-16:08:25] [I] Calibration: Dynamic
[05/24/2023-16:08:25] [I] Refit: Disabled
[05/24/2023-16:08:25] [I] Sparsity: Disabled
[05/24/2023-16:08:25] [I] Safe mode: Disabled
[05/24/2023-16:08:25] [I] DirectIO mode: Disabled
[05/24/2023-16:08:25] [I] Restricted mode: Disabled
[05/24/2023-16:08:25] [I] Build only: Disabled
[05/24/2023-16:08:25] [I] Save engine: ../weights/quantize_onnx_sim.engine
[05/24/2023-16:08:25] [I] Load engine:
[05/24/2023-16:08:25] [I] Profiling verbosity: 0
[05/24/2023-16:08:25] [I] Tactic sources: Using default tactic sources
[05/24/2023-16:08:25] [I] timingCacheMode: local
[05/24/2023-16:08:25] [I] timingCacheFile:
[05/24/2023-16:08:25] [I] Heuristic: Disabled
[05/24/2023-16:08:25] [I] Preview Features: Use default preview flags.
[05/24/2023-16:08:25] [I] Input(s)s format: fp32:CHW
[05/24/2023-16:08:25] [I] Output(s)s format: fp32:CHW
[05/24/2023-16:08:25] [I] Input build shapes: model
[05/24/2023-16:08:25] [I] Input calibration shapes: model
[05/24/2023-16:08:25] [I] === System Options ===
[05/24/2023-16:08:25] [I] Device: 0
[05/24/2023-16:08:25] [I] DLACore: 
[05/24/2023-16:08:25] [I] Plugins:
[05/24/2023-16:08:25] [I] === Inference Options ===
[05/24/2023-16:08:25] [I] Batch: Explicit
[05/24/2023-16:08:25] [I] Input inference shapes: model
[05/24/2023-16:08:25] [I] Iterations: 10
[05/24/2023-16:08:25] [I] Duration: 3s (+ 200ms warm up)
[05/24/2023-16:08:25] [I] Sleep time: 0ms
[05/24/2023-16:08:25] [I] Idle time: 0ms
[05/24/2023-16:08:25] [I] Streams: 1
[05/24/2023-16:08:25] [I] ExposeDMA: Disabled
[05/24/2023-16:08:25] [I] Data transfers: Enabled
[05/24/2023-16:08:25] [I] Spin-wait: Disabled
[05/24/2023-16:08:25] [I] Multithreading: Disabled
[05/24/2023-16:08:25] [I] CUDA Graph: Disabled
[05/24/2023-16:08:25] [I] Separate profiling: Disabled
[05/24/2023-16:08:25] [I] Time Deserialize: Disabled
[05/24/2023-16:08:25] [I] Time Refit: Disabled
[05/24/2023-16:08:25] [I] NVTX verbosity: 0
[05/24/2023-16:08:25] [I] Persistent Cache Ratio: 0
[05/24/2023-16:08:25] [I] Inputs:
[05/24/2023-16:08:25] [I] === Reporting Options ===
[05/24/2023-16:08:25] [I] Verbose: Enabled
[05/24/2023-16:08:25] [I] Averages: 10 inferences
[05/24/2023-16:08:25] [I] Percentiles: 90,95,99
[05/24/2023-16:08:25] [I] Dump refittable layers:Disabled
[05/24/2023-16:08:25] [I] Dump output: Disabled
[05/24/2023-16:08:25] [I] Profile: Disabled
[05/24/2023-16:08:25] [I] Export timing to JSON file:
[05/24/2023-16:08:25] [I] Export output to JSON file:
[05/24/2023-16:08:25] [I] Export profile to JSON file:
[05/24/2023-16:08:25] [I]
[05/24/2023-16:08:25] [I] === Device Information ===
[05/24/2023-16:08:25] [I] Selected Device: NVIDIA T1200 Laptop GPU
[05/24/2023-16:08:25] [I] Compute Capability: 7.5
[05/24/2023-16:08:25] [I] SMs: 16
[05/24/2023-16:08:25] [I] Compute Clock Rate: 1.56 GHz
[05/24/2023-16:08:25] [I] Device Global Memory: 4095 MiB
[05/24/2023-16:08:25] [I] Shared Memory per SM: 64 KiB
[05/24/2023-16:08:25] [I] Memory Bus Width: 128 bits (ECC disabled)
[05/24/2023-16:08:25] [I] Memory Clock Rate: 5.001 GHz
[05/24/2023-16:08:25] [I]
[05/24/2023-16:08:25] [I] TensorRT version: 8.5.1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Proposal version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ROIAlign_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Split version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[05/24/2023-16:08:26] [I] [TRT] [MemUsageChange] Init CUDA: CPU +411, GPU +0, now: CPU 15497, GPU 901 (MiB)
[05/24/2023-16:08:26] [V] [TRT] Trying to load shared library nvinfer_builder_resource.dll
[05/24/2023-16:08:26] [V] [TRT] Loaded shared library nvinfer_builder_resource.dll
[05/24/2023-16:08:27] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +266, GPU +74, now: CPU 16240, GPU 975 (MiB)
[05/24/2023-16:08:27] [W] [TRT] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
[05/24/2023-16:08:27] [I] Start parsing network model
[05/24/2023-16:08:27] [I] [TRT] ----------------------------------------------------------------
[05/24/2023-16:08:27] [I] [TRT] Input filename:   ../weights/quantize_onnx_sim.onnx
[05/24/2023-16:08:27] [I] [TRT] ONNX IR version:  0.0.7
[05/24/2023-16:08:27] [I] [TRT] Opset version:    13
[05/24/2023-16:08:27] [I] [TRT] Producer name:    pytorch
[05/24/2023-16:08:27] [I] [TRT] Producer version: 2.0.0
[05/24/2023-16:08:27] [I] [TRT] Domain:
[05/24/2023-16:08:27] [I] [TRT] Model version:    0
[05/24/2023-16:08:27] [I] [TRT] Doc string:
[05/24/2023-16:08:27] [I] [TRT] ----------------------------------------------------------------
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 2
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ROIAlign_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ScatterND version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Split version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1
[05/24/2023-16:08:27] [V] [TRT] Adding network input: input0 with dtype: float32, dimensions: (1, 3, 224, 224)
[05/24/2023-16:08:27] [V] [TRT] Registering tensor: input0 for ONNX tensor: input0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /Cast_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /Cast_1_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_16/Constant_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_16/Constant_1_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_26/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_26/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_16_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_17_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_24_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_32_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_40_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_48_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_56_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_64_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_72_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_80_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_34/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1_1/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_32/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_15/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_30/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /QuantizeLinear [QuantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: input0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /Cast_1_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /Cast_output_0
[05/24/2023-16:08:28] [V] [TRT] /QuantizeLinear [QuantizeLinear] inputs: [input0 -> (1, 3, 224, 224)[FLOAT]], [/Cast_1_output_0 -> ()[FLOAT]], [/Cast_output_0 -> ()[INT32]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /Cast_1_output_0 for ONNX node: /Cast_1_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /Cast_output_0 for ONNX node: /Cast_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering tensor: /QuantizeLinear_output_0 for ONNX tensor: /QuantizeLinear_output_0
[05/24/2023-16:08:28] [V] [TRT] /QuantizeLinear [QuantizeLinear] outputs: [/QuantizeLinear_output_0 -> (1, 3, 224, 224)[FLOAT]],
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /QuantizeLinear_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /Cast_1_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /Cast_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear [DequantizeLinear] inputs: [/QuantizeLinear_output_0 -> (1, 3, 224, 224)[FLOAT]], [/Cast_1_output_0 -> ()[FLOAT]], [/Cast_output_0 -> ()[INT32]],   
[05/24/2023-16:08:28] [V] [TRT] Registering tensor: /0_34/DequantizeLinear_output_0 for ONNX tensor: /0_34/DequantizeLinear_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear [DequantizeLinear] outputs: [/0_34/DequantizeLinear_output_0 -> (1, 3, 224, 224)[FLOAT]],
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear_1 [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_1 [DequantizeLinear] inputs: [/0_34/Cast_3_output_0 -> (32, 3, 3, 3)[INT8]], [/0_34/Cast_4_output_0 -> (32)[FLOAT]], [/0_34/Cast_5_output_0 -> (32)[INT8]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_3_output_0 for ONNX node: /0_34/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_4_output_0 for ONNX node: /0_34/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_5_output_0 for ONNX node: /0_34/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering tensor: /0_34/DequantizeLinear_1_output_0 for ONNX tensor: /0_34/DequantizeLinear_1_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_1 [DequantizeLinear] outputs: [/0_34/DequantizeLinear_1_output_0 -> (32, 3, 3, 3)[FLOAT]],
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear_2 [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_2 [DequantizeLinear] inputs: [/0_34/Cast_7_output_0 -> (32)[INT32]], [/0_34/Cast_8_output_0 -> (32)[FLOAT]], [/0_34/ConstantOfShape_output_0 -> (32)[INT32]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_7_output_0 for ONNX node: /0_34/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_8_output_0 for ONNX node: /0_34/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/ConstantOfShape_output_0 for ONNX node: /0_34/ConstantOfShape_output_0
[05/24/2023-16:08:28] [E] Error[3]: /0_34/DequantizeLinear_2: only activation types allowed as input to this layer.
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:726: While parsing node number 3 [DequantizeLinear -> "/0_34/DequantizeLinear_2_output_0"]:
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:728: input: "/0_34/Cast_7_output_0"
input: "/0_34/Cast_8_output_0"
input: "/0_34/ConstantOfShape_output_0"
output: "/0_34/DequantizeLinear_2_output_0"
name: "/0_34/DequantizeLinear_2"
op_type: "DequantizeLinear"
attribute {
  name: "axis"
  i: 0
  type: INT
}

[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:732: ERROR: ModelImporter.cpp:185 In function parseGraph:
[6] Invalid Node - /0_34/DequantizeLinear_2
/0_34/DequantizeLinear_2: only activation types allowed as input to this layer.
[05/24/2023-16:08:28] [E] Failed to parse onnx file
[05/24/2023-16:08:28] [I] Finish parsing network model
[05/24/2023-16:08:28] [E] Parsing model failed
[05/24/2023-16:08:28] [E] Failed to create engine from model or file.
[05/24/2023-16:08:28] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # trtexec --onnx=../weights/quantize_onnx_sim.onnx --int8 --saveEngine=../weights/quantize_onnx_sim.engine --verbose

Steps To Reproduce

Here is the model I use, it is the mobileNetv2 model quantized by pytorch.
quantize_onnx_sim.onnx (2.4 MB)

2

Download this tensorrt project then use the polygraphy executable to sanitise the onnx:

Use this command:

python polygraphy surgeon sanitize --fold-constants ./model.onnx  -o ./model_out.onnx

Then run trtexec on the resulting onnx again.

3

Hi,

Could you please check if the input tensor is an INT32 tensor? Int32 are index types, not activation types.

Thank you.

4

Thanks for your help. I tried your suggestion, but it seems not work for me. After using this operation, and then using trtexec to convert, the same error will still be reported.

(tensorrt) E:\TensorRT\TensorRT-main\tools\Polygraphy\bin>python polygraphy surgeon sanitize --fold-constants E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx -o ./model_out2.onnx     
[I] RUNNING | Command: polygraphy surgeon sanitize --fold-constants E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx -o ./model_out2.onnx
[I] Inferring shapes in the model with `onnxruntime.tools.symbolic_shape_infer`.
    Note: To force Polygraphy to use `onnx.shape_inference` instead, set `allow_onnxruntime=False` or use the `--no-onnxruntime-shape-inference` command-line option.
[I] Loading model: E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx
[W] ONNX shape inference exited with an error:
    integer division or modulo by zero
[I] Loading model: E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx
[I] Original Model:
    Name: torch_jit | ONNX Opset: 13

    ---- 1 Graph Input(s) ----
    {input0 [dtype=float32, shape=(1, 3, 224, 224)]}

    ---- 1 Graph Output(s) ----
    {output0 [dtype=float32, shape=(1, 10)]}

    ---- 2 Initializer(s) ----

    ---- 1836 Node(s) ----

[I] Folding Constants | Pass 1
[I]     Total Nodes | Original:  1836, After Folding:   529 |  1307 Nodes Folded
[I] Folding Constants | Pass 2
[I]     Total Nodes | Original:   529, After Folding:   529 |     0 Nodes Folded
[I] Saving ONNX model to: ./model_out2.onnx
[I] New Model:
    Name: torch_jit | ONNX Opset: 13

    ---- 1 Graph Input(s) ----
    {input0 [dtype=float32, shape=(1, 3, 224, 224)]}

    ---- 1 Graph Output(s) ----
    {output0 [dtype=float32, shape=(1, 10)]}

    ---- 738 Initializer(s) ----

    ---- 529 Node(s) ----

[I] PASSED | Runtime: 2.447s | Command: polygraphy surgeon sanitize --fold-constants E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx -o ./model_out2.onnx

then use trtexec, same error as before

[05/29/2023-10:35:14] [V] [TRT] /0_34/DequantizeLinear [DequantizeLinear] inputs: [/0_34/Cast_output_0 -> (1, 3, 224, 224)[UINT8]], [/0_34/Cast_1_output_0 -> ()[FLOAT]], [/0_34/Cast_2_output_0 -> ()[INT32]],
[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:726: While parsing node number 2 [DequantizeLinear -> "/0_34/DequantizeLinear_output_0"]:
[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:728: input: "/0_34/Cast_output_0"
input: "/0_34/Cast_1_output_0"
input: "/0_34/Cast_2_output_0"
output: "/0_34/DequantizeLinear_output_0"
name: "/0_34/DequantizeLinear"
op_type: "DequantizeLinear"

[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:732: ERROR: onnx2trt_utils.cpp:23 In function notInvalidType:
[8] Found invalid input type of UINT8
[05/29/2023-10:35:14] [E] Failed to parse onnx file
[05/29/2023-10:35:14] [I] Finish parsing network model
[05/29/2023-10:35:14] [E] Parsing model failed
[05/29/2023-10:35:14] [E] Failed to create engine from model or file.
[05/29/2023-10:35:14] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # trtexec --onnx=./model_out2.onnx --int8 --saveEngine=./model_out2.engine --verbose
5

Hi, thanks for your help.
As you said, the input is int32 type, not activation type.

image

I don’t know how to deal with this kind of error. Can I change the type of input？ Is there any type conversion operation for onnx or tensorrt？
Thank you.

8

Hi,

Please try ONNX GraphSurgeon. I do not have a sample to share at this moment.
https://docs.nvidia.com/deeplearning/tensorrt/onnx-graphsurgeon/docs/index.html

Thank you.