Trtexec Error: only activation types allowed as input to this layer

Environment

TensorRT Version: 8.5.1.7
GPU Type: NVIDIA T1200 Laptop GPU
Nvidia Driver Version: 522.06
CUDA Version: 11.8
CUDNN Version: 8.2
Operating System + Version: Windows10
PyTorch Version (if applicable): 2.0.0

Description

Hello, I am new to tensorrt . When I try to export an onnx model to tensorrt model, it crashed. Here is the error message.

[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear_1 [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_1 [DequantizeLinear] inputs: [/0_34/Cast_3_output_0 -> (32, 3, 3, 3)[INT8]], [/0_34/Cast_4_output_0 -> (32)[FLOAT]], [/0_34/Cast_5_output_0 -> (32)[INT8]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_3_output_0 for ONNX node: /0_34/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_4_output_0 for ONNX node: /0_34/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_5_output_0 for ONNX node: /0_34/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering tensor: /0_34/DequantizeLinear_1_output_0 for ONNX tensor: /0_34/DequantizeLinear_1_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_1 [DequantizeLinear] outputs: [/0_34/DequantizeLinear_1_output_0 -> (32, 3, 3, 3)[FLOAT]],
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear_2 [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_2 [DequantizeLinear] inputs: [/0_34/Cast_7_output_0 -> (32)[INT32]], [/0_34/Cast_8_output_0 -> (32)[FLOAT]], [/0_34/ConstantOfShape_output_0 -> (32)[INT32]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_7_output_0 for ONNX node: /0_34/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_8_output_0 for ONNX node: /0_34/Cast_8_output_0
[05/24/2023-16:08:28] [E] Error[3]: /0_34/DequantizeLinear_2: only activation types allowed as input to this layer.
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:726: While parsing node number 3 [DequantizeLinear -> "/0_34/DequantizeLinear_2_output_0"]:
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:728: input: "/0_34/Cast_7_output_0"
input: "/0_34/Cast_8_output_0"
input: "/0_34/ConstantOfShape_output_0"
output: "/0_34/DequantizeLinear_2_output_0"
name: "/0_34/DequantizeLinear_2"
op_type: "DequantizeLinear"
attribute {
  name: "axis"
  i: 0
  type: INT
}

process

I have carried out int8 quantization operation for this origin model by pytorch(torch.ao.quantization),Then I convert the model to onnx format. When I use trtexec to convert onnx2engine, an error is reported. The onnx model node where the error occurs is as follows:

The full command and message is as follows

trtexec --onnx=../weights/quantize_onnx_sim.onnx --int8 --saveEngine=../weights/quantize_onnx_sim.engine --verbose

[05/24/2023-16:08:25] [I] === Model Options ===
[05/24/2023-16:08:25] [I] Format: ONNX
[05/24/2023-16:08:25] [I] Model: ../weights/quantize_onnx_sim.onnx
[05/24/2023-16:08:25] [I] Output:
[05/24/2023-16:08:25] [I] === Build Options ===
[05/24/2023-16:08:25] [I] Max batch: explicit batch
[05/24/2023-16:08:25] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[05/24/2023-16:08:25] [I] minTiming: 1
[05/24/2023-16:08:25] [I] avgTiming: 8
[05/24/2023-16:08:25] [I] Precision: FP32+INT8
[05/24/2023-16:08:25] [I] LayerPrecisions:
[05/24/2023-16:08:25] [I] Calibration: Dynamic
[05/24/2023-16:08:25] [I] Refit: Disabled
[05/24/2023-16:08:25] [I] Sparsity: Disabled
[05/24/2023-16:08:25] [I] Safe mode: Disabled
[05/24/2023-16:08:25] [I] DirectIO mode: Disabled
[05/24/2023-16:08:25] [I] Restricted mode: Disabled
[05/24/2023-16:08:25] [I] Build only: Disabled
[05/24/2023-16:08:25] [I] Save engine: ../weights/quantize_onnx_sim.engine
[05/24/2023-16:08:25] [I] Load engine:
[05/24/2023-16:08:25] [I] Profiling verbosity: 0
[05/24/2023-16:08:25] [I] Tactic sources: Using default tactic sources
[05/24/2023-16:08:25] [I] timingCacheMode: local
[05/24/2023-16:08:25] [I] timingCacheFile:
[05/24/2023-16:08:25] [I] Heuristic: Disabled
[05/24/2023-16:08:25] [I] Preview Features: Use default preview flags.
[05/24/2023-16:08:25] [I] Input(s)s format: fp32:CHW
[05/24/2023-16:08:25] [I] Output(s)s format: fp32:CHW
[05/24/2023-16:08:25] [I] Input build shapes: model
[05/24/2023-16:08:25] [I] Input calibration shapes: model
[05/24/2023-16:08:25] [I] === System Options ===
[05/24/2023-16:08:25] [I] Device: 0
[05/24/2023-16:08:25] [I] DLACore: 
[05/24/2023-16:08:25] [I] Plugins:
[05/24/2023-16:08:25] [I] === Inference Options ===
[05/24/2023-16:08:25] [I] Batch: Explicit
[05/24/2023-16:08:25] [I] Input inference shapes: model
[05/24/2023-16:08:25] [I] Iterations: 10
[05/24/2023-16:08:25] [I] Duration: 3s (+ 200ms warm up)
[05/24/2023-16:08:25] [I] Sleep time: 0ms
[05/24/2023-16:08:25] [I] Idle time: 0ms
[05/24/2023-16:08:25] [I] Streams: 1
[05/24/2023-16:08:25] [I] ExposeDMA: Disabled
[05/24/2023-16:08:25] [I] Data transfers: Enabled
[05/24/2023-16:08:25] [I] Spin-wait: Disabled
[05/24/2023-16:08:25] [I] Multithreading: Disabled
[05/24/2023-16:08:25] [I] CUDA Graph: Disabled
[05/24/2023-16:08:25] [I] Separate profiling: Disabled
[05/24/2023-16:08:25] [I] Time Deserialize: Disabled
[05/24/2023-16:08:25] [I] Time Refit: Disabled
[05/24/2023-16:08:25] [I] NVTX verbosity: 0
[05/24/2023-16:08:25] [I] Persistent Cache Ratio: 0
[05/24/2023-16:08:25] [I] Inputs:
[05/24/2023-16:08:25] [I] === Reporting Options ===
[05/24/2023-16:08:25] [I] Verbose: Enabled
[05/24/2023-16:08:25] [I] Averages: 10 inferences
[05/24/2023-16:08:25] [I] Percentiles: 90,95,99
[05/24/2023-16:08:25] [I] Dump refittable layers:Disabled
[05/24/2023-16:08:25] [I] Dump output: Disabled
[05/24/2023-16:08:25] [I] Profile: Disabled
[05/24/2023-16:08:25] [I] Export timing to JSON file:
[05/24/2023-16:08:25] [I] Export output to JSON file:
[05/24/2023-16:08:25] [I] Export profile to JSON file:
[05/24/2023-16:08:25] [I]
[05/24/2023-16:08:25] [I] === Device Information ===
[05/24/2023-16:08:25] [I] Selected Device: NVIDIA T1200 Laptop GPU
[05/24/2023-16:08:25] [I] Compute Capability: 7.5
[05/24/2023-16:08:25] [I] SMs: 16
[05/24/2023-16:08:25] [I] Compute Clock Rate: 1.56 GHz
[05/24/2023-16:08:25] [I] Device Global Memory: 4095 MiB
[05/24/2023-16:08:25] [I] Shared Memory per SM: 64 KiB
[05/24/2023-16:08:25] [I] Memory Bus Width: 128 bits (ECC disabled)
[05/24/2023-16:08:25] [I] Memory Clock Rate: 5.001 GHz
[05/24/2023-16:08:25] [I]
[05/24/2023-16:08:25] [I] TensorRT version: 8.5.1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Proposal version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ROIAlign_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::Split version 1
[05/24/2023-16:08:25] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[05/24/2023-16:08:26] [I] [TRT] [MemUsageChange] Init CUDA: CPU +411, GPU +0, now: CPU 15497, GPU 901 (MiB)
[05/24/2023-16:08:26] [V] [TRT] Trying to load shared library nvinfer_builder_resource.dll
[05/24/2023-16:08:26] [V] [TRT] Loaded shared library nvinfer_builder_resource.dll
[05/24/2023-16:08:27] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +266, GPU +74, now: CPU 16240, GPU 975 (MiB)
[05/24/2023-16:08:27] [W] [TRT] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
[05/24/2023-16:08:27] [I] Start parsing network model
[05/24/2023-16:08:27] [I] [TRT] ----------------------------------------------------------------
[05/24/2023-16:08:27] [I] [TRT] Input filename:   ../weights/quantize_onnx_sim.onnx
[05/24/2023-16:08:27] [I] [TRT] ONNX IR version:  0.0.7
[05/24/2023-16:08:27] [I] [TRT] Opset version:    13
[05/24/2023-16:08:27] [I] [TRT] Producer name:    pytorch
[05/24/2023-16:08:27] [I] [TRT] Producer version: 2.0.0
[05/24/2023-16:08:27] [I] [TRT] Domain:
[05/24/2023-16:08:27] [I] [TRT] Model version:    0
[05/24/2023-16:08:27] [I] [TRT] Doc string:
[05/24/2023-16:08:27] [I] [TRT] ----------------------------------------------------------------
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 2
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ROIAlign_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::ScatterND version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::Split version 1
[05/24/2023-16:08:27] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1
[05/24/2023-16:08:27] [V] [TRT] Adding network input: input0 with dtype: float32, dimensions: (1, 3, 224, 224)
[05/24/2023-16:08:27] [V] [TRT] Registering tensor: input0 for ONNX tensor: input0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /Cast_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /Cast_1_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_34/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_16/Constant_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_16/Constant_1_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_33/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /1_1/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_32/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_31/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_15/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_5_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_30/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_29/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_14/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_28/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_27/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_4_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_7_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_8_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_10_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /2_13/Cast_11_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_26/Cast_3_output_0
[05/24/2023-16:08:27] [V] [TRT] Importing initializer: /0_26/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_25/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_12/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_16_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_17_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_24/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_23/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_11/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_24_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_22/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_21/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_19/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_9/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_32_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_18/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_17/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_8/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_40_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_16/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_15/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_7/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_48_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_14/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_13/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_6/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_11/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_5/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_56_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_10/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_9/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_4/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_64_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_8/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_7/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_5/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_2/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_72_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_4/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_3/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_1/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /Cast_80_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_2/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_1/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_10_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/Cast_11_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_34/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1_1/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_32/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_15/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_30/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_26/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_10/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_20/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_12/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2_3/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0_6/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /2/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /0/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Importing initializer: /1/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /QuantizeLinear [QuantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: input0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /Cast_1_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /Cast_output_0
[05/24/2023-16:08:28] [V] [TRT] /QuantizeLinear [QuantizeLinear] inputs: [input0 -> (1, 3, 224, 224)[FLOAT]], [/Cast_1_output_0 -> ()[FLOAT]], [/Cast_output_0 -> ()[INT32]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /Cast_1_output_0 for ONNX node: /Cast_1_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /Cast_output_0 for ONNX node: /Cast_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering tensor: /QuantizeLinear_output_0 for ONNX tensor: /QuantizeLinear_output_0
[05/24/2023-16:08:28] [V] [TRT] /QuantizeLinear [QuantizeLinear] outputs: [/QuantizeLinear_output_0 -> (1, 3, 224, 224)[FLOAT]],
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /QuantizeLinear_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /Cast_1_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /Cast_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear [DequantizeLinear] inputs: [/QuantizeLinear_output_0 -> (1, 3, 224, 224)[FLOAT]], [/Cast_1_output_0 -> ()[FLOAT]], [/Cast_output_0 -> ()[INT32]],   
[05/24/2023-16:08:28] [V] [TRT] Registering tensor: /0_34/DequantizeLinear_output_0 for ONNX tensor: /0_34/DequantizeLinear_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear [DequantizeLinear] outputs: [/0_34/DequantizeLinear_output_0 -> (1, 3, 224, 224)[FLOAT]],
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear_1 [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_1 [DequantizeLinear] inputs: [/0_34/Cast_3_output_0 -> (32, 3, 3, 3)[INT8]], [/0_34/Cast_4_output_0 -> (32)[FLOAT]], [/0_34/Cast_5_output_0 -> (32)[INT8]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_3_output_0 for ONNX node: /0_34/Cast_3_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_4_output_0 for ONNX node: /0_34/Cast_4_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_5_output_0 for ONNX node: /0_34/Cast_5_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering tensor: /0_34/DequantizeLinear_1_output_0 for ONNX tensor: /0_34/DequantizeLinear_1_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_1 [DequantizeLinear] outputs: [/0_34/DequantizeLinear_1_output_0 -> (32, 3, 3, 3)[FLOAT]],
[05/24/2023-16:08:28] [V] [TRT] Parsing node: /0_34/DequantizeLinear_2 [DequantizeLinear]
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Searching for input: /0_34/ConstantOfShape_output_0
[05/24/2023-16:08:28] [V] [TRT] /0_34/DequantizeLinear_2 [DequantizeLinear] inputs: [/0_34/Cast_7_output_0 -> (32)[INT32]], [/0_34/Cast_8_output_0 -> (32)[FLOAT]], [/0_34/ConstantOfShape_output_0 -> (32)[INT32]],
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_7_output_0 for ONNX node: /0_34/Cast_7_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/Cast_8_output_0 for ONNX node: /0_34/Cast_8_output_0
[05/24/2023-16:08:28] [V] [TRT] Registering layer: /0_34/ConstantOfShape_output_0 for ONNX node: /0_34/ConstantOfShape_output_0
[05/24/2023-16:08:28] [E] Error[3]: /0_34/DequantizeLinear_2: only activation types allowed as input to this layer.
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:726: While parsing node number 3 [DequantizeLinear -> "/0_34/DequantizeLinear_2_output_0"]:
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:728: input: "/0_34/Cast_7_output_0"
input: "/0_34/Cast_8_output_0"
input: "/0_34/ConstantOfShape_output_0"
output: "/0_34/DequantizeLinear_2_output_0"
name: "/0_34/DequantizeLinear_2"
op_type: "DequantizeLinear"
attribute {
  name: "axis"
  i: 0
  type: INT
}

[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[05/24/2023-16:08:28] [E] [TRT] ModelImporter.cpp:732: ERROR: ModelImporter.cpp:185 In function parseGraph:
[6] Invalid Node - /0_34/DequantizeLinear_2
/0_34/DequantizeLinear_2: only activation types allowed as input to this layer.
[05/24/2023-16:08:28] [E] Failed to parse onnx file
[05/24/2023-16:08:28] [I] Finish parsing network model
[05/24/2023-16:08:28] [E] Parsing model failed
[05/24/2023-16:08:28] [E] Failed to create engine from model or file.
[05/24/2023-16:08:28] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # trtexec --onnx=../weights/quantize_onnx_sim.onnx --int8 --saveEngine=../weights/quantize_onnx_sim.engine --verbose

Steps To Reproduce

Here is the model I use, it is the mobileNetv2 model quantized by pytorch.
quantize_onnx_sim.onnx (2.4 MB)

Download this tensorrt project then use the polygraphy executable to sanitise the onnx:

Use this command:

python polygraphy surgeon sanitize --fold-constants ./model.onnx  -o ./model_out.onnx

Then run trtexec on the resulting onnx again.

Hi,

Could you please check if the input tensor is an INT32 tensor? Int32 are index types, not activation types.

Thank you.

Thanks for your help. I tried your suggestion, but it seems not work for me. After using this operation, and then using trtexec to convert, the same error will still be reported.

(tensorrt) E:\TensorRT\TensorRT-main\tools\Polygraphy\bin>python polygraphy surgeon sanitize --fold-constants E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx -o ./model_out2.onnx     
[I] RUNNING | Command: polygraphy surgeon sanitize --fold-constants E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx -o ./model_out2.onnx
[I] Inferring shapes in the model with `onnxruntime.tools.symbolic_shape_infer`.
    Note: To force Polygraphy to use `onnx.shape_inference` instead, set `allow_onnxruntime=False` or use the `--no-onnxruntime-shape-inference` command-line option.
[I] Loading model: E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx
[W] ONNX shape inference exited with an error:
    integer division or modulo by zero
[I] Loading model: E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx
[I] Original Model:
    Name: torch_jit | ONNX Opset: 13

    ---- 1 Graph Input(s) ----
    {input0 [dtype=float32, shape=(1, 3, 224, 224)]}

    ---- 1 Graph Output(s) ----
    {output0 [dtype=float32, shape=(1, 10)]}

    ---- 2 Initializer(s) ----

    ---- 1836 Node(s) ----

[I] Folding Constants | Pass 1
[I]     Total Nodes | Original:  1836, After Folding:   529 |  1307 Nodes Folded
[I] Folding Constants | Pass 2
[I]     Total Nodes | Original:   529, After Folding:   529 |     0 Nodes Folded
[I] Saving ONNX model to: ./model_out2.onnx
[I] New Model:
    Name: torch_jit | ONNX Opset: 13

    ---- 1 Graph Input(s) ----
    {input0 [dtype=float32, shape=(1, 3, 224, 224)]}

    ---- 1 Graph Output(s) ----
    {output0 [dtype=float32, shape=(1, 10)]}

    ---- 738 Initializer(s) ----

    ---- 529 Node(s) ----

[I] PASSED | Runtime: 2.447s | Command: polygraphy surgeon sanitize --fold-constants E:\TensorRT\TensorRT-main\mycode\ptq_static\weights\quantize_onnx.onnx -o ./model_out2.onnx

then use trtexec, same error as before

[05/29/2023-10:35:14] [V] [TRT] /0_34/DequantizeLinear [DequantizeLinear] inputs: [/0_34/Cast_output_0 -> (1, 3, 224, 224)[UINT8]], [/0_34/Cast_1_output_0 -> ()[FLOAT]], [/0_34/Cast_2_output_0 -> ()[INT32]],
[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:726: While parsing node number 2 [DequantizeLinear -> "/0_34/DequantizeLinear_output_0"]:
[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:728: input: "/0_34/Cast_output_0"
input: "/0_34/Cast_1_output_0"
input: "/0_34/Cast_2_output_0"
output: "/0_34/DequantizeLinear_output_0"
name: "/0_34/DequantizeLinear"
op_type: "DequantizeLinear"

[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[05/29/2023-10:35:14] [E] [TRT] ModelImporter.cpp:732: ERROR: onnx2trt_utils.cpp:23 In function notInvalidType:
[8] Found invalid input type of UINT8
[05/29/2023-10:35:14] [E] Failed to parse onnx file
[05/29/2023-10:35:14] [I] Finish parsing network model
[05/29/2023-10:35:14] [E] Parsing model failed
[05/29/2023-10:35:14] [E] Failed to create engine from model or file.
[05/29/2023-10:35:14] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # trtexec --onnx=./model_out2.onnx --int8 --saveEngine=./model_out2.engine --verbose

Hi, thanks for your help.
As you said, the input is int32 type, not activation type.

image

I don’t know how to deal with this kind of error. Can I change the type of input? Is there any type conversion operation for onnx or tensorrt?
Thank you.

Hi,

Please try ONNX GraphSurgeon. I do not have a sample to share at this moment.
https://docs.nvidia.com/deeplearning/tensorrt/onnx-graphsurgeon/docs/index.html

Thank you.

Hi, I have a same problem with you. And you have found an answer?