Trtexec cannot convert QAT onnx model to trt model

Wilbur · December 22, 2022, 7:09am

Description

follow here to convert QAT save_model format to onnx,
then run /usr/src/tensorrt/bin/trtexec --onnx=saved_model_qat_no_auto.onnx --saveEngine=saved_model_qat.trt --minShapes=input_1:1x224x224x1 --optShapes=input_1:2x224x224x1 --maxShapes=input_1:2x224x224x1 --int8 --verbose to convert onnx to trt, but failed;

Below is the error message：

ModelImporter.cpp:778: ERROR: builtin_op_importers.cpp:1173 In function QuantDequantLinearHelper:
[6] Assertion failed: shiftIsAllZeros(zeroPoint) && "TRT only supports symmetric quantization - zeroPt must be all zeros

Environment

TensorRT Version: jetpack 5.0.2(tensorrt 8.4)
GPU Type: agx Xavier
Nvidia Driver Version:
CUDA Version:
CUDNN Version:
Operating System + Version:
Python Version (if applicable): python3.8
TensorFlow Version (if applicable): tensorflow 2.8
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
my onnx
saved_model_qat_auto.onnx (452.6 KB)

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

Exact steps/commands to build your repro
Exact steps/commands to run your repro
Full traceback of errors encountered

NVES · December 22, 2022, 7:37am

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

Wilbur · December 22, 2022, 7:43am

I had run check_model.py nothing output, and the onnx had shared;
and I used the below method to quantize the tf model,
tensorflow_model_optimization.python.core.quantization.keras.quantize_model(model)

Wilbur · December 22, 2022, 7:44am

root@caper-desktop:/media/sbot1/wilbur/models# /usr/src/tensorrt/bin/trtexec --onnx=saved_model_qat_no_auto.onnx --saveEngine=saved_model_qat.trt --minShapes=input_1:1x224x224x1 --optShapes=input_1:2x224x224x1 --maxShapes=input_1:2x224x224x1   --int8 --verbose
&&&& RUNNING TensorRT.trtexec [TensorRT v8401] # /usr/src/tensorrt/bin/trtexec --onnx=saved_model_qat_no_auto.onnx --saveEngine=saved_model_qat.trt --minShapes=input_1:1x224x224x1 --optShapes=input_1:2x224x224x1 --maxShapes=input_1:2x224x224x1 --int8 --verbose
[12/22/2022-04:34:52] [I] === Model Options ===
[12/22/2022-04:34:52] [I] Format: ONNX
[12/22/2022-04:34:52] [I] Model: saved_model_qat_no_auto.onnx
[12/22/2022-04:34:52] [I] Output:
[12/22/2022-04:34:52] [I] === Build Options ===
[12/22/2022-04:34:52] [I] Max batch: explicit batch
[12/22/2022-04:34:52] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[12/22/2022-04:34:52] [I] minTiming: 1
[12/22/2022-04:34:52] [I] avgTiming: 8
[12/22/2022-04:34:52] [I] Precision: FP32+INT8
[12/22/2022-04:34:52] [I] LayerPrecisions:
[12/22/2022-04:34:52] [I] Calibration: Dynamic
[12/22/2022-04:34:52] [I] Refit: Disabled
[12/22/2022-04:34:52] [I] Sparsity: Disabled
[12/22/2022-04:34:52] [I] Safe mode: Disabled
[12/22/2022-04:34:52] [I] DirectIO mode: Disabled
[12/22/2022-04:34:52] [I] Restricted mode: Disabled
[12/22/2022-04:34:52] [I] Build only: Disabled
[12/22/2022-04:34:52] [I] Save engine: saved_model_qat.trt
[12/22/2022-04:34:52] [I] Load engine:
[12/22/2022-04:34:52] [I] Profiling verbosity: 0
[12/22/2022-04:34:52] [I] Tactic sources: Using default tactic sources
[12/22/2022-04:34:52] [I] timingCacheMode: local
[12/22/2022-04:34:52] [I] timingCacheFile:
[12/22/2022-04:34:52] [I] Input(s)s format: fp32:CHW
[12/22/2022-04:34:52] [I] Output(s)s format: fp32:CHW
[12/22/2022-04:34:52] [I] Input build shape: input_1=1x224x224x1+2x224x224x1+2x224x224x1
[12/22/2022-04:34:52] [I] Input calibration shapes: model
[12/22/2022-04:34:52] [I] === System Options ===
[12/22/2022-04:34:52] [I] Device: 0
[12/22/2022-04:34:52] [I] DLACore:
[12/22/2022-04:34:52] [I] Plugins:
[12/22/2022-04:34:52] [I] === Inference Options ===
[12/22/2022-04:34:52] [I] Batch: Explicit
[12/22/2022-04:34:52] [I] Input inference shape: input_1=2x224x224x1
[12/22/2022-04:34:52] [I] Iterations: 10
[12/22/2022-04:34:52] [I] Duration: 3s (+ 200ms warm up)
[12/22/2022-04:34:52] [I] Sleep time: 0ms
[12/22/2022-04:34:52] [I] Idle time: 0ms
[12/22/2022-04:34:52] [I] Streams: 1
[12/22/2022-04:34:52] [I] ExposeDMA: Disabled
[12/22/2022-04:34:52] [I] Data transfers: Enabled
[12/22/2022-04:34:52] [I] Spin-wait: Disabled
[12/22/2022-04:34:52] [I] Multithreading: Disabled
[12/22/2022-04:34:52] [I] CUDA Graph: Disabled
[12/22/2022-04:34:52] [I] Separate profiling: Disabled
[12/22/2022-04:34:52] [I] Time Deserialize: Disabled
[12/22/2022-04:34:52] [I] Time Refit: Disabled
[12/22/2022-04:34:52] [I] Inputs:
[12/22/2022-04:34:52] [I] === Reporting Options ===
[12/22/2022-04:34:52] [I] Verbose: Enabled
[12/22/2022-04:34:52] [I] Averages: 10 inferences
[12/22/2022-04:34:52] [I] Percentile: 99
[12/22/2022-04:34:52] [I] Dump refittable layers:Disabled
[12/22/2022-04:34:52] [I] Dump output: Disabled
[12/22/2022-04:34:52] [I] Profile: Disabled
[12/22/2022-04:34:52] [I] Export timing to JSON file:
[12/22/2022-04:34:52] [I] Export output to JSON file:
[12/22/2022-04:34:52] [I] Export profile to JSON file:
[12/22/2022-04:34:52] [I]
[12/22/2022-04:34:52] [I] === Device Information ===
[12/22/2022-04:34:52] [I] Selected Device: Xavier
[12/22/2022-04:34:52] [I] Compute Capability: 7.2
[12/22/2022-04:34:52] [I] SMs: 8
[12/22/2022-04:34:52] [I] Compute Clock Rate: 1.377 GHz
[12/22/2022-04:34:52] [I] Device Global Memory: 31002 MiB
[12/22/2022-04:34:52] [I] Shared Memory per SM: 96 KiB
[12/22/2022-04:34:52] [I] Memory Bus Width: 256 bits (ECC disabled)
[12/22/2022-04:34:52] [I] Memory Clock Rate: 1.377 GHz
[12/22/2022-04:34:52] [I]
[12/22/2022-04:34:52] [I] TensorRT version: 8.4.1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::Proposal version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::Split version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[12/22/2022-04:34:52] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[12/22/2022-04:34:53] [I] [TRT] [MemUsageChange] Init CUDA: CPU +185, GPU +0, now: CPU 209, GPU 5902 (MiB)
[12/22/2022-04:34:55] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +131, GPU +123, now: CPU 359, GPU 6042 (MiB)
[12/22/2022-04:34:55] [I] Start parsing network model
[12/22/2022-04:34:55] [I] [TRT] ----------------------------------------------------------------
[12/22/2022-04:34:55] [I] [TRT] Input filename:   saved_model_qat_no_auto.onnx
[12/22/2022-04:34:55] [I] [TRT] ONNX IR version:  0.0.7
[12/22/2022-04:34:55] [I] [TRT] Opset version:    13
[12/22/2022-04:34:55] [I] [TRT] Producer name:    tf2onnx
[12/22/2022-04:34:55] [I] [TRT] Producer version: 1.10.1 a37f29
[12/22/2022-04:34:55] [I] [TRT] Domain:
[12/22/2022-04:34:55] [I] [TRT] Model version:    0
[12/22/2022-04:34:55] [I] [TRT] Doc string:
[12/22/2022-04:34:55] [I] [TRT] ----------------------------------------------------------------
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::ScatterND version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::Split version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 2
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1
[12/22/2022-04:34:55] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1
[12/22/2022-04:34:55] [V] [TRT] Adding network input: input_1 with dtype: float32, dimensions: (-1, 224, 224, 1)
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: input_1 for ONNX tensor: input_1
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: const_fold_opt__540
[12/22/2022-04:34:55] [W] [TRT] onnx2trt_utils.cpp:367: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: const_fold_opt__538
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: const_fold_opt__529
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: const_fold_opt__527
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: const_axes__81
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: const_axes__543
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_zpy__94
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_scaley__93
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_9_relu/MovingAvgQuantize/FakeQuantWithMinMaxVars_scaley__386
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_9/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_9/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_8/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_8/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_7/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_7/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_6/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_6/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_5_relu/MovingAvgQuantize/FakeQuantWithMinMaxVars_zpy__263
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_5/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_5/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_4/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_4/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_3_relu/Relu6_max__199
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_3/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_3/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_2/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_2/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_13/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_13/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_12/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_12/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_11/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_11/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_10/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_10/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_1/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_pw_1/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_9/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_9/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_8/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_8/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_7/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_7/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_6/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_6/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_5_relu/Relu6_min__246
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_5/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_5/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_4/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_4/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_3/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_3/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_2_relu/MovingAvgQuantize/FakeQuantWithMinMaxVars_scaley__155
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_2/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_2/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_1_relu/MovingAvgQuantize/FakeQuantWithMinMaxVars_scaley__124
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_13/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_13/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_12/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_12/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_11/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_11/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_10/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_10/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_1/depthwise_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv_dw_1/depthwise_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv1_relu/MovingAvgQuantize/FakeQuantWithMinMaxVars_zpy__108
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv1_relu/MovingAvgQuantize/FakeQuantWithMinMaxVars_scaley__107
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv1/Conv2D_weights_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quant_conv1/Conv2D_bias_fused_bn
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/dense/MatMul/ReadVariableOp:0
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1/y:0
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_5__37
[12/22/2022-04:34:55] [V] [TRT] Importing initializer: StatefulPartitionedCall/angle_model/coordinate_channel/Cast_3__35
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/Shape [Shape]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: input_1
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/Shape [Shape] inputs: [input_1 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/Shape for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/Shape
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/Shape:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/Shape:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/Shape [Shape] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/Shape:0 -> (4)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45 [Cast]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/Shape:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45 [Cast] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/Shape:0 -> (4)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Casting to type: int32
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45 [Cast] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45:0 -> (4)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice [Slice]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_axes__81
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__540
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_axes__81
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice [Slice] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/Shape__45:0 -> (4)[INT32]], [const_axes__81 -> (1)[INT32]], [const_fold_opt__540 -> (1)[INT32]], [const_axes__81 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice [Slice] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice:0 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85 [Concat]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__529
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85 [Concat] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice:0 -> (1)[INT32]], [const_fold_opt__529 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: const_fold_opt__529 for ONNX node: const_fold_opt__529
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85 [Concat] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85:0 -> (2)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/ones__86 [Cast]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ones__86 [Cast] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/stack_Concat__85:0 -> (2)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Casting to type: int32
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ones__86 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ones__86
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ones__86:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ones__86:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ones__86 [Cast] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ones__86:0 -> (2)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/ones [Expand]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ones__86:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ones [Expand] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0 -> ()[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/ones__86:0 -> (2)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ones for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ones
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ones:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ones:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ones [Expand] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ones:0 -> (-1, 224)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55 [Concat]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__527
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55 [Concat] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/strided_slice:0 -> (1)[INT32]], [const_fold_opt__527 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: const_fold_opt__527 for ONNX node: const_fold_opt__527
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55 [Concat] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55:0 -> (2)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77 [Cast]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77 [Cast] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/stack_3_Concat__55:0 -> (2)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Casting to type: int32
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77 [Cast] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77:0 -> (2)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/Tile [Tile]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_5__37
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/Tile [Tile] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_5__37 -> (1, 224)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/Tile__77:0 -> (2)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_5__37 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_5__37
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/Tile for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/Tile
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/Tile:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/Tile:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/Tile [Tile] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/Tile:0 -> (-1, 224)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6 [Unsqueeze]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/Tile:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__538
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6 [Unsqueeze] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/Tile:0 -> (-1, 224)[FLOAT]], [const_fold_opt__538 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Original shape: (_, 224), unsqueezing to: (_, _, _)
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6 [Unsqueeze] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6:0 -> (-1, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4 [Unsqueeze]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ones:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__540
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4 [Unsqueeze] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ones:0 -> (-1, 224)[FLOAT]], [const_fold_opt__540 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Original shape: (_, 224), unsqueezing to: (_, _, _)
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4 [Unsqueeze] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4:0 -> (-1, 1, 224)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1 [MatMul]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1 [MatMul] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_6:0 -> (-1, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_4:0 -> (-1, 1, 224)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1 [MatMul] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1:0 -> (-1, 224, 224)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7 [Unsqueeze]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__538
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7 [Unsqueeze] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/MatMul_1:0 -> (-1, 224, 224)[FLOAT]], [const_fold_opt__538 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Original shape: (_, 224, 224), unsqueezing to: (_, _, _, _)
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7 [Unsqueeze] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1 [Transpose]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1 [Transpose] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_7:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1 [Transpose] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2 [Unsqueeze]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/Tile:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__540
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2 [Unsqueeze] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/Tile:0 -> (-1, 224)[FLOAT]], [const_fold_opt__540 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Original shape: (_, 224), unsqueezing to: (_, _, _)
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2 [Unsqueeze] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2:0 -> (-1, 1, 224)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims [Unsqueeze]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ones:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__538
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims [Unsqueeze] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ones:0 -> (-1, 224)[FLOAT]], [const_fold_opt__538 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Original shape: (_, 224), unsqueezing to: (_, _, _)
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims [Unsqueeze] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims:0 -> (-1, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul [MatMul]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/MatMul [MatMul] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims:0 -> (-1, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_2:0 -> (-1, 1, 224)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/MatMul [MatMul] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/MatMul:0 -> (-1, 224, 224)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3 [Unsqueeze]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/MatMul:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: const_fold_opt__538
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3 [Unsqueeze] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/MatMul:0 -> (-1, 224, 224)[FLOAT]], [const_fold_opt__538 -> (1)[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Original shape: (_, 224, 224), unsqueezing to: (_, _, _, _)
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3 [Unsqueeze] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/transpose [Transpose]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/transpose [Transpose] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/ExpandDims_3:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/transpose for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/transpose
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/transpose:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/transpose:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/transpose [Transpose] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/transpose:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1 [Div]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/Cast_3__35
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1 [Div] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/transpose_1:0 -> (-1, 224, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/Cast_3__35 -> ()[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/Cast_3__35 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/Cast_3__35
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1 [Div] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1 [Mul]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1/y:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/mul_1 [Mul] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/truediv_1:0 -> (-1, 224, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/mul_1/y:0 -> ()[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1/y:0 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1/y:0
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/mul_1 [Mul] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/mul_1:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/sub_3 [Sub]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/sub_3 [Sub] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/mul_1:0 -> (-1, 224, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0 -> ()[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/sub_3 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/sub_3
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/sub_3:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/sub_3:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/sub_3 [Sub] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/sub_3:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/truediv [Div]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/transpose:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/Cast_3__35
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/truediv [Div] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/transpose:0 -> (-1, 224, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/Cast_3__35 -> ()[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/truediv for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/truediv
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/truediv:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/truediv:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/truediv [Div] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/truediv:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/mul [Mul]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/truediv:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/mul_1/y:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/mul [Mul] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/truediv:0 -> (-1, 224, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/mul_1/y:0 -> ()[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/mul for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/mul
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/mul:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/mul:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/mul [Mul] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/mul:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/sub_1 [Sub]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/mul:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/sub_1 [Sub] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/mul:0 -> (-1, 224, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/ones_1/Const:0 -> ()[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/sub_1 for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/sub_1
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/sub_1:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/sub_1:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/sub_1 [Sub] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/sub_1:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/coordinate_channel/concat [Concat]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: input_1
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/sub_1:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/sub_3:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/concat [Concat] inputs: [input_1 -> (-1, 224, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/sub_1:0 -> (-1, 224, 224, 1)[FLOAT]], [StatefulPartitionedCall/angle_model/coordinate_channel/sub_3:0 -> (-1, 224, 224, 1)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/coordinate_channel/concat for ONNX node: StatefulPartitionedCall/angle_model/coordinate_channel/concat
[12/22/2022-04:34:55] [V] [TRT] Registering tensor: StatefulPartitionedCall/angle_model/coordinate_channel/concat:0 for ONNX tensor: StatefulPartitionedCall/angle_model/coordinate_channel/concat:0
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/coordinate_channel/concat [Concat] outputs: [StatefulPartitionedCall/angle_model/coordinate_channel/concat:0 -> (-1, 224, 224, 3)[FLOAT]],
[12/22/2022-04:34:55] [V] [TRT] Parsing node: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_QuantizeLinear__95 [QuantizeLinear]
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/coordinate_channel/concat:0
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_scaley__93
[12/22/2022-04:34:55] [V] [TRT] Searching for input: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_zpy__94
[12/22/2022-04:34:55] [V] [TRT] StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_QuantizeLinear__95 [QuantizeLinear] inputs: [StatefulPartitionedCall/angle_model/coordinate_channel/concat:0 -> (-1, 224, 224, 3)[FLOAT]], [StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_scaley__93 -> ()[FLOAT]], [StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_zpy__94 -> ()[INT32]],
[12/22/2022-04:34:55] [V] [TRT] Registering layer: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_scaley__93 for ONNX node: StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_scaley__93
[12/22/2022-04:34:55] [E] [TRT] ModelImporter.cpp:773: While parsing node number 26 [QuantizeLinear -> "StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_QuantizeLinear__95:0"]:
[12/22/2022-04:34:55] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[12/22/2022-04:34:55] [E] [TRT] ModelImporter.cpp:775: input: "StatefulPartitionedCall/angle_model/coordinate_channel/concat:0"
input: "StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_scaley__93"
input: "StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_zpy__94"
output: "StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_QuantizeLinear__95:0"
name: "StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_QuantizeLinear__95"
op_type: "QuantizeLinear"
attribute {
  name: "axis"
  i: 1
  type: INT
}
domain: ""

[12/22/2022-04:34:55] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[12/22/2022-04:34:55] [E] [TRT] ModelImporter.cpp:778: ERROR: builtin_op_importers.cpp:1173 In function QuantDequantLinearHelper:
[6] Assertion failed: shiftIsAllZeros(zeroPoint) && "TRT only supports symmetric quantization - zeroPt must be all zeros"
[12/22/2022-04:34:55] [E] Failed to parse onnx file
[12/22/2022-04:34:55] [I] Finish parsing network model
[12/22/2022-04:34:55] [E] Parsing model failed
[12/22/2022-04:34:55] [E] Failed to create engine from model or file.
[12/22/2022-04:34:55] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8401] # /usr/src/tensorrt/bin/trtexec --onnx=saved_model_qat_no_auto.onnx --saveEngine=saved_model_qat.trt --minShapes=input_1:1x224x224x1 --optShapes=input_1:2x224x224x1 --maxShapes=input_1:2x224x224x1 --int8 --verbose

spolisetty · January 3, 2023, 4:41pm

Hi,

We could reproduce the following error.

output: "StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_QuantizeLinear__95:0"
name: "StatefulPartitionedCall/angle_model/mobilenet_0.15_224/quantize_layer/AllValuesQuantize/FakeQuantWithMinMaxVars_QuantizeLinear__95"
op_type: "QuantizeLinear"
attribute {
  name: "axis"
  i: 1
  type: INT
}
domain: ""

[01/03/2023-16:33:19] [E] [TRT] parsers/onnx/ModelImporter.cpp:729: --- End node ---
[01/03/2023-16:33:19] [E] [TRT] parsers/onnx/ModelImporter.cpp:731: ERROR: parsers/onnx/builtin_op_importers.cpp:1227 In function QuantDequantLinearHelper:
[6] Assertion failed: shiftIsAllZeros(zeroPoint) && "TRT only supports symmetric quantization - zeroPt must be all zeros"
[01/03/2023-16:33:19] [E] Failed to parse onnx file

TRT only supports symmetric quantization. So it is recommended that you train your networks with input being symmetric ranges (like [-127, 127])

Thank you.

Wilbur · January 4, 2023, 3:10am

Hi, thanks for replying, where can I get an example or documentation about this?

gcunhasergio · January 9, 2023, 1:58pm

Hi @Wilbur,

More details regarding TensorRT’s support for symmetric quantization can be found here: Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

I also see that you quantized your model with TensorFlow’s native quantization function. However, that does not guarantee symmetric quantization (see error above), which is needed for TensorRT. Can you please try quantizing your model using our official tensorflow-quantization toolkit and try deploying that with TensorRT?

Thanks.

gcunhasergio · January 26, 2023, 3:43pm

@Wilbur please let us know if you have further questions.

Topic		Replies	Views
Trtexec conversion to int8 Jetson AGX Orin tensorrt	2	922	October 11, 2023
Trtexec "Failed to sort the model topologically." TensorRT tensorrt , cudnn , onnx	2	526	March 5, 2024
Trtexec Error: only activation types allowed as input to this layer TensorRT tensorrt	8	1450	June 28, 2024
Tensorrt8.2 convert .onnx model to .trt ,throw "two inputs (data and weights) are allowed only in explicit-quantization mode"error TensorRT	9	1932	April 6, 2022
[onnx->TensorRT]Parser problem about padding TensorRT	5	2065	October 19, 2021
Running trtexec on Google's EfficientDet-D4 in ONNX Format TensorRT	9	1318	June 11, 2021
How to export the Pytorch model Keypoint R-CNN to onnx and benchmark with trtexec TensorRT	7	1322	July 14, 2022
Parse onnx file failed: Parameter check failed. condition:condition: allDimsGtEq(windowSize, 1) && volume(windowSize) < MAX_KERNEL_DIMS_PRODUCT(nbSpat TensorRT tensorrt	3	940	May 17, 2022
ONNX to TensorRT conversion Jetson Nano tensorrt , pytorch , onnx	4	301	July 3, 2024
Error converting gru model to TRT engine TensorRT	3	988	October 25, 2021