Trtexec conversion to int8

Description

i am using this line of code “trtexec --onnx=models/onnx_models/vgg19.onnx --saveEngine=models/trt_engines/TRT_INT8.trt --int8” to convert my model to int8, the onnx model is checked. When trying to run inference on the trt engine, the model performs too well so i am wondering if the conversion engine build was correct… After that i try running this to make sure input layer and output layer are coded correctly using this “trtexec --onnx=models/onnx_models/vgg19_f16.onnx --precisionConstraints=obey --layerPrecisions=:fp16,layer_1:fp32 --layerOutputTypes=:fp16,layer_1:fp32+fp16 --verbose --profilingVerbosity=detailed --useCudaGraph --noDataTransfers --duration=60 --saveEngine=models/trt_engines/TRT_INT8.trt --int8” . I get the following log :

(.venv) thimonnier@thimonnier-desktop:~/defect-recognition$ trtexec --onnx=models/onnx_models/vgg19_f16.onnx --precisionConstraints=obey --layerPrecisions=:int8,layer_1:fp32 --layerOutputTypes=:fp16,layer_1:fp32+fp16 --verbose --profilingVerbosity=detailed --useCudaGraph --noDataTransfers --duration=60 --saveEngine=models/trt_engines/TRT_INT8.trt --int8
&&&& RUNNING TensorRT.trtexec [TensorRT v8502] # trtexec --onnx=models/onnx_models/vgg19_f16.onnx --precisionConstraints=obey --layerPrecisions=:int8,layer_1:fp32 --layerOutputTypes=:fp16,layer_1:fp32+fp16 --verbose --profilingVerbosity=detailed --useCudaGraph --noDataTransfers --duration=60 --saveEngine=models/trt_engines/TRT_INT8.trt --int8
[10/07/2023-11:12:28] [I] === Model Options ===
[10/07/2023-11:12:28] [I] Format: ONNX
[10/07/2023-11:12:28] [I] Model: models/onnx_models/vgg19_f16.onnx
[10/07/2023-11:12:28] [I] Output:
[10/07/2023-11:12:28] [I] === Build Options ===
[10/07/2023-11:12:28] [I] Max batch: explicit batch
[10/07/2023-11:12:28] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[10/07/2023-11:12:28] [I] minTiming: 1
[10/07/2023-11:12:28] [I] avgTiming: 8
[10/07/2023-11:12:28] [I] Precision: FP32+INT8 (obey precision constraints)
[10/07/2023-11:12:28] [I] LayerPrecisions: layer_1:fp32,:int8
[10/07/2023-11:12:28] [I] Calibration: Dynamic
[10/07/2023-11:12:28] [I] Refit: Disabled
[10/07/2023-11:12:28] [I] Sparsity: Disabled
[10/07/2023-11:12:28] [I] Safe mode: Disabled
[10/07/2023-11:12:28] [I] DirectIO mode: Disabled
[10/07/2023-11:12:28] [I] Restricted mode: Disabled
[10/07/2023-11:12:28] [I] Build only: Disabled
[10/07/2023-11:12:28] [I] Save engine: models/trt_engines/TRT_INT8.trt
[10/07/2023-11:12:28] [I] Load engine:
[10/07/2023-11:12:28] [I] Profiling verbosity: 2
[10/07/2023-11:12:28] [I] Tactic sources: Using default tactic sources
[10/07/2023-11:12:28] [I] timingCacheMode: local
[10/07/2023-11:12:28] [I] timingCacheFile:
[10/07/2023-11:12:28] [I] Heuristic: Disabled
[10/07/2023-11:12:28] [I] Preview Features: Use default preview flags.
[10/07/2023-11:12:28] [I] Input(s)s format: fp32:CHW
[10/07/2023-11:12:28] [I] Output(s)s format: fp32:CHW
[10/07/2023-11:12:28] [I] Input build shapes: model
[10/07/2023-11:12:28] [I] Input calibration shapes: model
[10/07/2023-11:12:28] [I] === System Options ===
[10/07/2023-11:12:28] [I] Device: 0
[10/07/2023-11:12:28] [I] DLACore:
[10/07/2023-11:12:28] [I] Plugins:
[10/07/2023-11:12:28] [I] === Inference Options ===
[10/07/2023-11:12:28] [I] Batch: Explicit
[10/07/2023-11:12:28] [I] Input inference shapes: model
[10/07/2023-11:12:28] [I] Iterations: 10
[10/07/2023-11:12:28] [I] Duration: 60s (+ 200ms warm up)
[10/07/2023-11:12:28] [I] Sleep time: 0ms
[10/07/2023-11:12:28] [I] Idle time: 0ms
[10/07/2023-11:12:28] [I] Streams: 1
[10/07/2023-11:12:28] [I] ExposeDMA: Disabled
[10/07/2023-11:12:28] [I] Data transfers: Disabled
[10/07/2023-11:12:28] [I] Spin-wait: Disabled
[10/07/2023-11:12:28] [I] Multithreading: Disabled
[10/07/2023-11:12:28] [I] CUDA Graph: Enabled
[10/07/2023-11:12:28] [I] Separate profiling: Disabled
[10/07/2023-11:12:28] [I] Time Deserialize: Disabled
[10/07/2023-11:12:28] [I] Time Refit: Disabled
[10/07/2023-11:12:28] [I] NVTX verbosity: 2
[10/07/2023-11:12:28] [I] Persistent Cache Ratio: 0
[10/07/2023-11:12:28] [I] Inputs:
[10/07/2023-11:12:28] [I] === Reporting Options ===
[10/07/2023-11:12:28] [I] Verbose: Enabled
[10/07/2023-11:12:28] [I] Averages: 10 inferences
[10/07/2023-11:12:28] [I] Percentiles: 90,95,99
[10/07/2023-11:12:28] [I] Dump refittable layers:Disabled
[10/07/2023-11:12:28] [I] Dump output: Disabled
[10/07/2023-11:12:28] [I] Profile: Disabled
[10/07/2023-11:12:28] [I] Export timing to JSON file:
[10/07/2023-11:12:28] [I] Export output to JSON file:
[10/07/2023-11:12:28] [I] Export profile to JSON file:
[10/07/2023-11:12:28] [I]
[10/07/2023-11:12:28] [I] === Device Information ===
[10/07/2023-11:12:28] [I] Selected Device: Orin
[10/07/2023-11:12:28] [I] Compute Capability: 8.7
[10/07/2023-11:12:28] [I] SMs: 16
[10/07/2023-11:12:28] [I] Compute Clock Rate: 1.3 GHz
[10/07/2023-11:12:28] [I] Device Global Memory: 62796 MiB
[10/07/2023-11:12:28] [I] Shared Memory per SM: 164 KiB
[10/07/2023-11:12:28] [I] Memory Bus Width: 128 bits (ECC disabled)
[10/07/2023-11:12:28] [I] Memory Clock Rate: 1.3 GHz
[10/07/2023-11:12:28] [I]
[10/07/2023-11:12:28] [I] TensorRT version: 8.5.2
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::GroupNorm version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::LayerNorm version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::Proposal version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::ROIAlign_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::SeqLen2Spatial version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::SplitGeLU version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::Split version 1
[10/07/2023-11:12:28] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[10/07/2023-11:12:28] [I] [TRT] [MemUsageChange] Init CUDA: CPU +220, GPU +0, now: CPU 246, GPU 17517 (MiB)
[10/07/2023-11:12:28] [V] [TRT] Trying to load shared library libnvinfer_builder_resource.so.8.5.2
[10/07/2023-11:12:28] [V] [TRT] Loaded shared library libnvinfer_builder_resource.so.8.5.2
[10/07/2023-11:12:30] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +302, GPU +436, now: CPU 571, GPU 17970 (MiB)
[10/07/2023-11:12:30] [I] Start parsing network model
[10/07/2023-11:12:30] [I] [TRT] ----------------------------------------------------------------
[10/07/2023-11:12:30] [I] [TRT] Input filename: models/onnx_models/vgg19_f16.onnx
[10/07/2023-11:12:30] [I] [TRT] ONNX IR version: 0.0.8
[10/07/2023-11:12:30] [I] [TRT] Opset version: 15
[10/07/2023-11:12:30] [I] [TRT] Producer name: tf2onnx
[10/07/2023-11:12:30] [I] [TRT] Producer version: 1.15.1 37820d
[10/07/2023-11:12:30] [I] [TRT] Domain:
[10/07/2023-11:12:30] [I] [TRT] Model version: 0
[10/07/2023-11:12:30] [I] [TRT] Doc string:
[10/07/2023-11:12:30] [I] [TRT] ----------------------------------------------------------------
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::GroupNorm version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 2
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::LayerNorm version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::ROIAlign_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::ScatterND version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::SeqLen2Spatial version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::SplitGeLU version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::Split version 1
[10/07/2023-11:12:30] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1
[10/07/2023-11:12:30] [V] [TRT] Adding network input: input_2 with dtype: float16, dimensions: (-1, 224, 224, 3)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: input_2 for ONNX tensor: input_2
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_9/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_9/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_8/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_8/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_7/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_7/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_6/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_6/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_10/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/dense_10/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block5_conv4/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block5_conv4/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block5_conv3/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block5_conv3/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block5_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block5_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block5_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block5_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block4_conv4/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block4_conv4/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block4_conv3/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block4_conv3/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block4_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block4_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block4_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block4_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block3_conv4/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block3_conv4/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block3_conv3/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block3_conv3/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block3_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block3_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block3_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block3_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block2_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block2_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block2_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block2_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block1_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block1_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block1_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: model_1/block1_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Importing initializer: const_fold_opt__91
[10/07/2023-11:12:30] [W] [TRT] onnx2trt_utils.cpp:375: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block1_conv1/BiasAdd__6 [Transpose]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: input_2
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv1/BiasAdd__6 [Transpose] inputs: [input_2 → (-1, 224, 224, 3)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block1_conv1/BiasAdd__6 for ONNX node: model_1/block1_conv1/BiasAdd__6
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block1_conv1/BiasAdd__6:0 for ONNX tensor: model_1/block1_conv1/BiasAdd__6:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv1/BiasAdd__6 [Transpose] outputs: [model_1/block1_conv1/BiasAdd__6:0 → (-1, 3, 224, 224)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block1_conv1/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv1/BiasAdd__6:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv1/BiasAdd [Conv] inputs: [model_1/block1_conv1/BiasAdd__6:0 → (-1, 3, 224, 224)[FLOAT]], [model_1/block1_conv1/Conv2D/ReadVariableOp:0 → (64, 3, 3, 3)[HALF]], [model_1/block1_conv1/BiasAdd/ReadVariableOp:0 → (64)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 3, 224, 224)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block1_conv1/BiasAdd for ONNX node: model_1/block1_conv1/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 64
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 64, 224, 224)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block1_conv1/BiasAdd:0 for ONNX tensor: model_1/block1_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv1/BiasAdd [Conv] outputs: [model_1/block1_conv1/BiasAdd:0 → (-1, 64, 224, 224)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block1_conv1/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv1/Relu [Relu] inputs: [model_1/block1_conv1/BiasAdd:0 → (-1, 64, 224, 224)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block1_conv1/Relu for ONNX node: model_1/block1_conv1/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block1_conv1/Relu:0 for ONNX tensor: model_1/block1_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv1/Relu [Relu] outputs: [model_1/block1_conv1/Relu:0 → (-1, 64, 224, 224)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block1_conv2/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv2/BiasAdd [Conv] inputs: [model_1/block1_conv1/Relu:0 → (-1, 64, 224, 224)[FLOAT]], [model_1/block1_conv2/Conv2D/ReadVariableOp:0 → (64, 64, 3, 3)[HALF]], [model_1/block1_conv2/BiasAdd/ReadVariableOp:0 → (64)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 64, 224, 224)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block1_conv2/BiasAdd for ONNX node: model_1/block1_conv2/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 64
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 64, 224, 224)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block1_conv2/BiasAdd:0 for ONNX tensor: model_1/block1_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv2/BiasAdd [Conv] outputs: [model_1/block1_conv2/BiasAdd:0 → (-1, 64, 224, 224)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block1_conv2/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv2/Relu [Relu] inputs: [model_1/block1_conv2/BiasAdd:0 → (-1, 64, 224, 224)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block1_conv2/Relu for ONNX node: model_1/block1_conv2/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block1_conv2/Relu:0 for ONNX tensor: model_1/block1_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_conv2/Relu [Relu] outputs: [model_1/block1_conv2/Relu:0 → (-1, 64, 224, 224)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block1_pool/MaxPool [MaxPool]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_pool/MaxPool [MaxPool] inputs: [model_1/block1_conv2/Relu:0 → (-1, 64, 224, 224)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block1_pool/MaxPool for ONNX node: model_1/block1_pool/MaxPool
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block1_pool/MaxPool:0 for ONNX tensor: model_1/block1_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block1_pool/MaxPool [MaxPool] outputs: [model_1/block1_pool/MaxPool:0 → (-1, 64, 112, 112)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block2_conv1/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block1_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_conv1/BiasAdd [Conv] inputs: [model_1/block1_pool/MaxPool:0 → (-1, 64, 112, 112)[FLOAT]], [model_1/block2_conv1/Conv2D/ReadVariableOp:0 → (128, 64, 3, 3)[HALF]], [model_1/block2_conv1/BiasAdd/ReadVariableOp:0 → (128)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 64, 112, 112)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block2_conv1/BiasAdd for ONNX node: model_1/block2_conv1/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 128
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 128, 112, 112)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block2_conv1/BiasAdd:0 for ONNX tensor: model_1/block2_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_conv1/BiasAdd [Conv] outputs: [model_1/block2_conv1/BiasAdd:0 → (-1, 128, 112, 112)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block2_conv1/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_conv1/Relu [Relu] inputs: [model_1/block2_conv1/BiasAdd:0 → (-1, 128, 112, 112)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block2_conv1/Relu for ONNX node: model_1/block2_conv1/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block2_conv1/Relu:0 for ONNX tensor: model_1/block2_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_conv1/Relu [Relu] outputs: [model_1/block2_conv1/Relu:0 → (-1, 128, 112, 112)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block2_conv2/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_conv2/BiasAdd [Conv] inputs: [model_1/block2_conv1/Relu:0 → (-1, 128, 112, 112)[FLOAT]], [model_1/block2_conv2/Conv2D/ReadVariableOp:0 → (128, 128, 3, 3)[HALF]], [model_1/block2_conv2/BiasAdd/ReadVariableOp:0 → (128)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 128, 112, 112)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block2_conv2/BiasAdd for ONNX node: model_1/block2_conv2/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 128
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 128, 112, 112)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block2_conv2/BiasAdd:0 for ONNX tensor: model_1/block2_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_conv2/BiasAdd [Conv] outputs: [model_1/block2_conv2/BiasAdd:0 → (-1, 128, 112, 112)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block2_conv2/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_conv2/Relu [Relu] inputs: [model_1/block2_conv2/BiasAdd:0 → (-1, 128, 112, 112)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block2_conv2/Relu for ONNX node: model_1/block2_conv2/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block2_conv2/Relu:0 for ONNX tensor: model_1/block2_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_conv2/Relu [Relu] outputs: [model_1/block2_conv2/Relu:0 → (-1, 128, 112, 112)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block2_pool/MaxPool [MaxPool]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_pool/MaxPool [MaxPool] inputs: [model_1/block2_conv2/Relu:0 → (-1, 128, 112, 112)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block2_pool/MaxPool for ONNX node: model_1/block2_pool/MaxPool
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block2_pool/MaxPool:0 for ONNX tensor: model_1/block2_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block2_pool/MaxPool [MaxPool] outputs: [model_1/block2_pool/MaxPool:0 → (-1, 128, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_conv1/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block2_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv1/BiasAdd [Conv] inputs: [model_1/block2_pool/MaxPool:0 → (-1, 128, 56, 56)[FLOAT]], [model_1/block3_conv1/Conv2D/ReadVariableOp:0 → (256, 128, 3, 3)[HALF]], [model_1/block3_conv1/BiasAdd/ReadVariableOp:0 → (256)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 128, 56, 56)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_conv1/BiasAdd for ONNX node: model_1/block3_conv1/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 256
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 256, 56, 56)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_conv1/BiasAdd:0 for ONNX tensor: model_1/block3_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv1/BiasAdd [Conv] outputs: [model_1/block3_conv1/BiasAdd:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_conv1/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv1/Relu [Relu] inputs: [model_1/block3_conv1/BiasAdd:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_conv1/Relu for ONNX node: model_1/block3_conv1/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_conv1/Relu:0 for ONNX tensor: model_1/block3_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv1/Relu [Relu] outputs: [model_1/block3_conv1/Relu:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_conv2/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv2/BiasAdd [Conv] inputs: [model_1/block3_conv1/Relu:0 → (-1, 256, 56, 56)[FLOAT]], [model_1/block3_conv2/Conv2D/ReadVariableOp:0 → (256, 256, 3, 3)[HALF]], [model_1/block3_conv2/BiasAdd/ReadVariableOp:0 → (256)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 256, 56, 56)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_conv2/BiasAdd for ONNX node: model_1/block3_conv2/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 256
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 256, 56, 56)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_conv2/BiasAdd:0 for ONNX tensor: model_1/block3_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv2/BiasAdd [Conv] outputs: [model_1/block3_conv2/BiasAdd:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_conv2/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv2/Relu [Relu] inputs: [model_1/block3_conv2/BiasAdd:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_conv2/Relu for ONNX node: model_1/block3_conv2/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_conv2/Relu:0 for ONNX tensor: model_1/block3_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv2/Relu [Relu] outputs: [model_1/block3_conv2/Relu:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_conv3/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv3/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv3/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv3/BiasAdd [Conv] inputs: [model_1/block3_conv2/Relu:0 → (-1, 256, 56, 56)[FLOAT]], [model_1/block3_conv3/Conv2D/ReadVariableOp:0 → (256, 256, 3, 3)[HALF]], [model_1/block3_conv3/BiasAdd/ReadVariableOp:0 → (256)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 256, 56, 56)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_conv3/BiasAdd for ONNX node: model_1/block3_conv3/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 256
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 256, 56, 56)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_conv3/BiasAdd:0 for ONNX tensor: model_1/block3_conv3/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv3/BiasAdd [Conv] outputs: [model_1/block3_conv3/BiasAdd:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_conv3/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv3/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv3/Relu [Relu] inputs: [model_1/block3_conv3/BiasAdd:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_conv3/Relu for ONNX node: model_1/block3_conv3/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_conv3/Relu:0 for ONNX tensor: model_1/block3_conv3/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv3/Relu [Relu] outputs: [model_1/block3_conv3/Relu:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_conv4/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv3/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv4/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv4/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv4/BiasAdd [Conv] inputs: [model_1/block3_conv3/Relu:0 → (-1, 256, 56, 56)[FLOAT]], [model_1/block3_conv4/Conv2D/ReadVariableOp:0 → (256, 256, 3, 3)[HALF]], [model_1/block3_conv4/BiasAdd/ReadVariableOp:0 → (256)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 256, 56, 56)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_conv4/BiasAdd for ONNX node: model_1/block3_conv4/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 256
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 256, 56, 56)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_conv4/BiasAdd:0 for ONNX tensor: model_1/block3_conv4/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv4/BiasAdd [Conv] outputs: [model_1/block3_conv4/BiasAdd:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_conv4/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv4/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv4/Relu [Relu] inputs: [model_1/block3_conv4/BiasAdd:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_conv4/Relu for ONNX node: model_1/block3_conv4/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_conv4/Relu:0 for ONNX tensor: model_1/block3_conv4/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_conv4/Relu [Relu] outputs: [model_1/block3_conv4/Relu:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block3_pool/MaxPool [MaxPool]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_conv4/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_pool/MaxPool [MaxPool] inputs: [model_1/block3_conv4/Relu:0 → (-1, 256, 56, 56)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block3_pool/MaxPool for ONNX node: model_1/block3_pool/MaxPool
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block3_pool/MaxPool:0 for ONNX tensor: model_1/block3_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block3_pool/MaxPool [MaxPool] outputs: [model_1/block3_pool/MaxPool:0 → (-1, 256, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_conv1/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block3_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv1/BiasAdd [Conv] inputs: [model_1/block3_pool/MaxPool:0 → (-1, 256, 28, 28)[FLOAT]], [model_1/block4_conv1/Conv2D/ReadVariableOp:0 → (512, 256, 3, 3)[HALF]], [model_1/block4_conv1/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 256, 28, 28)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_conv1/BiasAdd for ONNX node: model_1/block4_conv1/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 512, 28, 28)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_conv1/BiasAdd:0 for ONNX tensor: model_1/block4_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv1/BiasAdd [Conv] outputs: [model_1/block4_conv1/BiasAdd:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_conv1/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv1/Relu [Relu] inputs: [model_1/block4_conv1/BiasAdd:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_conv1/Relu for ONNX node: model_1/block4_conv1/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_conv1/Relu:0 for ONNX tensor: model_1/block4_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv1/Relu [Relu] outputs: [model_1/block4_conv1/Relu:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_conv2/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv2/BiasAdd [Conv] inputs: [model_1/block4_conv1/Relu:0 → (-1, 512, 28, 28)[FLOAT]], [model_1/block4_conv2/Conv2D/ReadVariableOp:0 → (512, 512, 3, 3)[HALF]], [model_1/block4_conv2/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 512, 28, 28)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_conv2/BiasAdd for ONNX node: model_1/block4_conv2/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 512, 28, 28)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_conv2/BiasAdd:0 for ONNX tensor: model_1/block4_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv2/BiasAdd [Conv] outputs: [model_1/block4_conv2/BiasAdd:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_conv2/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv2/Relu [Relu] inputs: [model_1/block4_conv2/BiasAdd:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_conv2/Relu for ONNX node: model_1/block4_conv2/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_conv2/Relu:0 for ONNX tensor: model_1/block4_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv2/Relu [Relu] outputs: [model_1/block4_conv2/Relu:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_conv3/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv3/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv3/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv3/BiasAdd [Conv] inputs: [model_1/block4_conv2/Relu:0 → (-1, 512, 28, 28)[FLOAT]], [model_1/block4_conv3/Conv2D/ReadVariableOp:0 → (512, 512, 3, 3)[HALF]], [model_1/block4_conv3/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 512, 28, 28)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_conv3/BiasAdd for ONNX node: model_1/block4_conv3/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 512, 28, 28)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_conv3/BiasAdd:0 for ONNX tensor: model_1/block4_conv3/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv3/BiasAdd [Conv] outputs: [model_1/block4_conv3/BiasAdd:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_conv3/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv3/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv3/Relu [Relu] inputs: [model_1/block4_conv3/BiasAdd:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_conv3/Relu for ONNX node: model_1/block4_conv3/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_conv3/Relu:0 for ONNX tensor: model_1/block4_conv3/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv3/Relu [Relu] outputs: [model_1/block4_conv3/Relu:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_conv4/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv3/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv4/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv4/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv4/BiasAdd [Conv] inputs: [model_1/block4_conv3/Relu:0 → (-1, 512, 28, 28)[FLOAT]], [model_1/block4_conv4/Conv2D/ReadVariableOp:0 → (512, 512, 3, 3)[HALF]], [model_1/block4_conv4/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 512, 28, 28)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_conv4/BiasAdd for ONNX node: model_1/block4_conv4/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 512, 28, 28)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_conv4/BiasAdd:0 for ONNX tensor: model_1/block4_conv4/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv4/BiasAdd [Conv] outputs: [model_1/block4_conv4/BiasAdd:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_conv4/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv4/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv4/Relu [Relu] inputs: [model_1/block4_conv4/BiasAdd:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_conv4/Relu for ONNX node: model_1/block4_conv4/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_conv4/Relu:0 for ONNX tensor: model_1/block4_conv4/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_conv4/Relu [Relu] outputs: [model_1/block4_conv4/Relu:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block4_pool/MaxPool [MaxPool]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_conv4/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_pool/MaxPool [MaxPool] inputs: [model_1/block4_conv4/Relu:0 → (-1, 512, 28, 28)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block4_pool/MaxPool for ONNX node: model_1/block4_pool/MaxPool
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block4_pool/MaxPool:0 for ONNX tensor: model_1/block4_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block4_pool/MaxPool [MaxPool] outputs: [model_1/block4_pool/MaxPool:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_conv1/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block4_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv1/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv1/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv1/BiasAdd [Conv] inputs: [model_1/block4_pool/MaxPool:0 → (-1, 512, 14, 14)[FLOAT]], [model_1/block5_conv1/Conv2D/ReadVariableOp:0 → (512, 512, 3, 3)[HALF]], [model_1/block5_conv1/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 512, 14, 14)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_conv1/BiasAdd for ONNX node: model_1/block5_conv1/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 512, 14, 14)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_conv1/BiasAdd:0 for ONNX tensor: model_1/block5_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv1/BiasAdd [Conv] outputs: [model_1/block5_conv1/BiasAdd:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_conv1/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv1/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv1/Relu [Relu] inputs: [model_1/block5_conv1/BiasAdd:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_conv1/Relu for ONNX node: model_1/block5_conv1/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_conv1/Relu:0 for ONNX tensor: model_1/block5_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv1/Relu [Relu] outputs: [model_1/block5_conv1/Relu:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_conv2/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv1/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv2/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv2/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv2/BiasAdd [Conv] inputs: [model_1/block5_conv1/Relu:0 → (-1, 512, 14, 14)[FLOAT]], [model_1/block5_conv2/Conv2D/ReadVariableOp:0 → (512, 512, 3, 3)[HALF]], [model_1/block5_conv2/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 512, 14, 14)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_conv2/BiasAdd for ONNX node: model_1/block5_conv2/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 512, 14, 14)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_conv2/BiasAdd:0 for ONNX tensor: model_1/block5_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv2/BiasAdd [Conv] outputs: [model_1/block5_conv2/BiasAdd:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_conv2/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv2/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv2/Relu [Relu] inputs: [model_1/block5_conv2/BiasAdd:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_conv2/Relu for ONNX node: model_1/block5_conv2/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_conv2/Relu:0 for ONNX tensor: model_1/block5_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv2/Relu [Relu] outputs: [model_1/block5_conv2/Relu:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_conv3/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv2/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv3/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv3/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv3/BiasAdd [Conv] inputs: [model_1/block5_conv2/Relu:0 → (-1, 512, 14, 14)[FLOAT]], [model_1/block5_conv3/Conv2D/ReadVariableOp:0 → (512, 512, 3, 3)[HALF]], [model_1/block5_conv3/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 512, 14, 14)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_conv3/BiasAdd for ONNX node: model_1/block5_conv3/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 512, 14, 14)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_conv3/BiasAdd:0 for ONNX tensor: model_1/block5_conv3/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv3/BiasAdd [Conv] outputs: [model_1/block5_conv3/BiasAdd:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_conv3/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv3/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv3/Relu [Relu] inputs: [model_1/block5_conv3/BiasAdd:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_conv3/Relu for ONNX node: model_1/block5_conv3/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_conv3/Relu:0 for ONNX tensor: model_1/block5_conv3/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv3/Relu [Relu] outputs: [model_1/block5_conv3/Relu:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_conv4/BiasAdd [Conv]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv3/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv4/Conv2D/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv4/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv4/BiasAdd [Conv] inputs: [model_1/block5_conv3/Relu:0 → (-1, 512, 14, 14)[FLOAT]], [model_1/block5_conv4/Conv2D/ReadVariableOp:0 → (512, 512, 3, 3)[HALF]], [model_1/block5_conv4/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Convolution input dimensions: (-1, 512, 14, 14)
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_conv4/BiasAdd for ONNX node: model_1/block5_conv4/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 512
[10/07/2023-11:12:30] [V] [TRT] Convolution output dimensions: (-1, 512, 14, 14)
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_conv4/BiasAdd:0 for ONNX tensor: model_1/block5_conv4/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv4/BiasAdd [Conv] outputs: [model_1/block5_conv4/BiasAdd:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_conv4/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv4/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv4/Relu [Relu] inputs: [model_1/block5_conv4/BiasAdd:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_conv4/Relu for ONNX node: model_1/block5_conv4/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_conv4/Relu:0 for ONNX tensor: model_1/block5_conv4/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_conv4/Relu [Relu] outputs: [model_1/block5_conv4/Relu:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_pool/MaxPool [MaxPool]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_conv4/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_pool/MaxPool [MaxPool] inputs: [model_1/block5_conv4/Relu:0 → (-1, 512, 14, 14)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_pool/MaxPool for ONNX node: model_1/block5_pool/MaxPool
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_pool/MaxPool:0 for ONNX tensor: model_1/block5_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_pool/MaxPool [MaxPool] outputs: [model_1/block5_pool/MaxPool:0 → (-1, 512, 7, 7)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/block5_pool/MaxPool__88 [Transpose]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_pool/MaxPool:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_pool/MaxPool__88 [Transpose] inputs: [model_1/block5_pool/MaxPool:0 → (-1, 512, 7, 7)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/block5_pool/MaxPool__88 for ONNX node: model_1/block5_pool/MaxPool__88
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/block5_pool/MaxPool__88:0 for ONNX tensor: model_1/block5_pool/MaxPool__88:0
[10/07/2023-11:12:30] [V] [TRT] model_1/block5_pool/MaxPool__88 [Transpose] outputs: [model_1/block5_pool/MaxPool__88:0 → (-1, 7, 7, 512)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/flatten_1/Reshape [Reshape]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/block5_pool/MaxPool__88:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: const_fold_opt__91
[10/07/2023-11:12:30] [V] [TRT] model_1/flatten_1/Reshape [Reshape] inputs: [model_1/block5_pool/MaxPool__88:0 → (-1, 7, 7, 512)[FLOAT]], [const_fold_opt__91 → (2)[INT32]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/flatten_1/Reshape for ONNX node: model_1/flatten_1/Reshape
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/flatten_1/Reshape:0 for ONNX tensor: model_1/flatten_1/Reshape:0
[10/07/2023-11:12:30] [V] [TRT] model_1/flatten_1/Reshape [Reshape] outputs: [model_1/flatten_1/Reshape:0 → (-1, 25088)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_6/MatMul [MatMul]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/flatten_1/Reshape:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_6/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_6/MatMul [MatMul] inputs: [model_1/flatten_1/Reshape:0 → (-1, 25088)[FLOAT]], [model_1/dense_6/MatMul/ReadVariableOp:0 → (25088, 1024)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_6/MatMul/ReadVariableOp:0 for ONNX node: model_1/dense_6/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_6/MatMul for ONNX node: model_1/dense_6/MatMul
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_6/MatMul:0 for ONNX tensor: model_1/dense_6/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_6/MatMul [MatMul] outputs: [model_1/dense_6/MatMul:0 → (-1, 1024)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_6/BiasAdd [Add]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_6/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_6/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_6/BiasAdd [Add] inputs: [model_1/dense_6/MatMul:0 → (-1, 1024)[FLOAT]], [model_1/dense_6/BiasAdd/ReadVariableOp:0 → (1024)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_6/BiasAdd/ReadVariableOp:0 for ONNX node: model_1/dense_6/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_6/BiasAdd for ONNX node: model_1/dense_6/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_6/BiasAdd:0 for ONNX tensor: model_1/dense_6/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_6/BiasAdd [Add] outputs: [model_1/dense_6/BiasAdd:0 → (-1, 1024)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_6/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_6/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_6/Relu [Relu] inputs: [model_1/dense_6/BiasAdd:0 → (-1, 1024)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_6/Relu for ONNX node: model_1/dense_6/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_6/Relu:0 for ONNX tensor: model_1/dense_6/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_6/Relu [Relu] outputs: [model_1/dense_6/Relu:0 → (-1, 1024)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_7/MatMul [MatMul]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_6/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_7/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_7/MatMul [MatMul] inputs: [model_1/dense_6/Relu:0 → (-1, 1024)[FLOAT]], [model_1/dense_7/MatMul/ReadVariableOp:0 → (1024, 512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_7/MatMul/ReadVariableOp:0 for ONNX node: model_1/dense_7/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_7/MatMul for ONNX node: model_1/dense_7/MatMul
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_7/MatMul:0 for ONNX tensor: model_1/dense_7/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_7/MatMul [MatMul] outputs: [model_1/dense_7/MatMul:0 → (-1, 512)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_7/BiasAdd [Add]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_7/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_7/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_7/BiasAdd [Add] inputs: [model_1/dense_7/MatMul:0 → (-1, 512)[FLOAT]], [model_1/dense_7/BiasAdd/ReadVariableOp:0 → (512)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_7/BiasAdd/ReadVariableOp:0 for ONNX node: model_1/dense_7/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_7/BiasAdd for ONNX node: model_1/dense_7/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_7/BiasAdd:0 for ONNX tensor: model_1/dense_7/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_7/BiasAdd [Add] outputs: [model_1/dense_7/BiasAdd:0 → (-1, 512)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_7/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_7/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_7/Relu [Relu] inputs: [model_1/dense_7/BiasAdd:0 → (-1, 512)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_7/Relu for ONNX node: model_1/dense_7/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_7/Relu:0 for ONNX tensor: model_1/dense_7/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_7/Relu [Relu] outputs: [model_1/dense_7/Relu:0 → (-1, 512)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_8/MatMul [MatMul]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_7/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_8/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_8/MatMul [MatMul] inputs: [model_1/dense_7/Relu:0 → (-1, 512)[FLOAT]], [model_1/dense_8/MatMul/ReadVariableOp:0 → (512, 256)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_8/MatMul/ReadVariableOp:0 for ONNX node: model_1/dense_8/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_8/MatMul for ONNX node: model_1/dense_8/MatMul
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_8/MatMul:0 for ONNX tensor: model_1/dense_8/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_8/MatMul [MatMul] outputs: [model_1/dense_8/MatMul:0 → (-1, 256)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_8/BiasAdd [Add]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_8/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_8/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_8/BiasAdd [Add] inputs: [model_1/dense_8/MatMul:0 → (-1, 256)[FLOAT]], [model_1/dense_8/BiasAdd/ReadVariableOp:0 → (256)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_8/BiasAdd/ReadVariableOp:0 for ONNX node: model_1/dense_8/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_8/BiasAdd for ONNX node: model_1/dense_8/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_8/BiasAdd:0 for ONNX tensor: model_1/dense_8/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_8/BiasAdd [Add] outputs: [model_1/dense_8/BiasAdd:0 → (-1, 256)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_8/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_8/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_8/Relu [Relu] inputs: [model_1/dense_8/BiasAdd:0 → (-1, 256)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_8/Relu for ONNX node: model_1/dense_8/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_8/Relu:0 for ONNX tensor: model_1/dense_8/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_8/Relu [Relu] outputs: [model_1/dense_8/Relu:0 → (-1, 256)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_9/MatMul [MatMul]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_8/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_9/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_9/MatMul [MatMul] inputs: [model_1/dense_8/Relu:0 → (-1, 256)[FLOAT]], [model_1/dense_9/MatMul/ReadVariableOp:0 → (256, 128)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_9/MatMul/ReadVariableOp:0 for ONNX node: model_1/dense_9/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_9/MatMul for ONNX node: model_1/dense_9/MatMul
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_9/MatMul:0 for ONNX tensor: model_1/dense_9/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_9/MatMul [MatMul] outputs: [model_1/dense_9/MatMul:0 → (-1, 128)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_9/BiasAdd [Add]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_9/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_9/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_9/BiasAdd [Add] inputs: [model_1/dense_9/MatMul:0 → (-1, 128)[FLOAT]], [model_1/dense_9/BiasAdd/ReadVariableOp:0 → (128)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_9/BiasAdd/ReadVariableOp:0 for ONNX node: model_1/dense_9/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_9/BiasAdd for ONNX node: model_1/dense_9/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_9/BiasAdd:0 for ONNX tensor: model_1/dense_9/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_9/BiasAdd [Add] outputs: [model_1/dense_9/BiasAdd:0 → (-1, 128)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_9/Relu [Relu]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_9/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_9/Relu [Relu] inputs: [model_1/dense_9/BiasAdd:0 → (-1, 128)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_9/Relu for ONNX node: model_1/dense_9/Relu
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_9/Relu:0 for ONNX tensor: model_1/dense_9/Relu:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_9/Relu [Relu] outputs: [model_1/dense_9/Relu:0 → (-1, 128)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_10/MatMul [MatMul]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_9/Relu:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_10/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_10/MatMul [MatMul] inputs: [model_1/dense_9/Relu:0 → (-1, 128)[FLOAT]], [model_1/dense_10/MatMul/ReadVariableOp:0 → (128, 2)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_10/MatMul/ReadVariableOp:0 for ONNX node: model_1/dense_10/MatMul/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_10/MatMul for ONNX node: model_1/dense_10/MatMul
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_10/MatMul:0 for ONNX tensor: model_1/dense_10/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_10/MatMul [MatMul] outputs: [model_1/dense_10/MatMul:0 → (-1, 2)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_10/BiasAdd [Add]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_10/MatMul:0
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_10/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_10/BiasAdd [Add] inputs: [model_1/dense_10/MatMul:0 → (-1, 2)[FLOAT]], [model_1/dense_10/BiasAdd/ReadVariableOp:0 → (2)[HALF]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_10/BiasAdd/ReadVariableOp:0 for ONNX node: model_1/dense_10/BiasAdd/ReadVariableOp:0
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_10/BiasAdd for ONNX node: model_1/dense_10/BiasAdd
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: model_1/dense_10/BiasAdd:0 for ONNX tensor: model_1/dense_10/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_10/BiasAdd [Add] outputs: [model_1/dense_10/BiasAdd:0 → (-1, 2)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Parsing node: model_1/dense_10/Softmax [Softmax]
[10/07/2023-11:12:30] [V] [TRT] Searching for input: model_1/dense_10/BiasAdd:0
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_10/Softmax [Softmax] inputs: [model_1/dense_10/BiasAdd:0 → (-1, 2)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Registering layer: model_1/dense_10/Softmax for ONNX node: model_1/dense_10/Softmax
[10/07/2023-11:12:30] [V] [TRT] Registering tensor: dense_10_0 for ONNX tensor: dense_10
[10/07/2023-11:12:30] [V] [TRT] model_1/dense_10/Softmax [Softmax] outputs: [dense_10 → (-1, 2)[FLOAT]],
[10/07/2023-11:12:30] [V] [TRT] Marking dense_10_0 as output: dense_10
[10/07/2023-11:12:30] [I] Finish parsing network model
[10/07/2023-11:12:30] [W] Dynamic dimensions required for input: input_2, but no shapes were provided. Automatically overriding shape to: 1x224x224x3
[10/07/2023-11:12:30] [I] FP32 and INT8 precisions have been specified - more performance might be enabled by additionally specifying --fp16 or --best
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for input_2 to [-2,2]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block1_conv1/BiasAdd__6:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block1_conv1/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block1_conv1/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block1_conv2/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block1_conv2/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block1_pool/MaxPool:0 to [-2,2]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block2_conv1/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block2_conv1/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block2_conv2/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block2_conv2/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block2_pool/MaxPool:0 to [-2,2]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_conv1/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_conv1/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_conv2/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_conv2/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_conv3/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_conv3/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_conv4/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_conv4/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block3_pool/MaxPool:0 to [-2,2]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_conv1/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_conv1/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_conv2/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_conv2/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_conv3/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_conv3/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_conv4/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_conv4/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block4_pool/MaxPool:0 to [-2,2]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_conv1/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_conv1/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_conv2/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_conv2/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_conv3/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_conv3/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_conv4/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_conv4/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_pool/MaxPool:0 to [-2,2]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/block5_pool/MaxPool__88:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/flatten_1/Reshape:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer
40) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_6/MatMul:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 42) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 43) [Shuffle]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_6/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_6/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 46) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_7/MatMul:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 48) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 49) [Shuffle]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_7/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_7/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 52) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_8/MatMul:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 54) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 55) [Shuffle]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_8/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_8/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 58) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_9/MatMul:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 60) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 61) [Shuffle]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_9/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_9/Relu:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 64) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_10/MatMul:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 66) [Constant]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 67) [Shuffle]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for model_1/dense_10/BiasAdd:0 to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 69) [Softmax]_output to [-4,4]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for (Unnamed Layer* 71) [Shape]_output to [-2,2]
[10/07/2023-11:12:30] [V] [TRT] Setting dynamic range for dense_10 to [-4,4]
[10/07/2023-11:12:30] [V] Skipped setting precision for layer (Unnamed Layer* 71) [Shape] because the default layer precision is INT32 or Bool.
[10/07/2023-11:12:30] [I] Skipped setting precisions for some layers. Check verbose logs for more details.
[10/07/2023-11:12:30] [V] Skipped setting output types for layer (Unnamed Layer* 71) [Shape] because the default layer precision is INT32 or Bool.
[10/07/2023-11:12:30] [I] Skipped setting output types for some layers. Check verbose logs for more details.
[10/07/2023-11:12:30] [E] Error[4]: [network.cpp::validate::2925] Error Code 4: Internal Error (fp16 precision has been set for a layer or layer output, but fp16 is not configured in the builder)
[10/07/2023-11:12:30] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[10/07/2023-11:12:30] [E] Engine could not be created from network
[10/07/2023-11:12:30] [E] Building engine failed
[10/07/2023-11:12:30] [E] Failed to create engine from model or file.
[10/07/2023-11:12:30] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8502] # trtexec --onnx=models/onnx_models/vgg19_f16.onnx --precisionConstraints=obey --layerPrecisions=:int8,layer_1:fp32 --layerOutputTypes=:fp16,layer_1:fp32+fp16 --verbose --profilingVerbosity=detailed --useCudaGraph --noDataTransfers --duration=60 --saveEngine=models/trt_engines/TRT_INT8.trt --int8

Environment

TensorRT v8502:
GPU jetson agx orin dev kit:
CUDA Version 11.4:

Relevant Files

I cannot share the models but i am trying to understand the log better.

Dear @thim.moumout,
Could you give a try adding --fp16 to command?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.