Whats the shapes in Autotuning format combination part mean using TensorRT with trtexec?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi all,

I runned the infrerence of a simple CNN i made (ONNX format) with trtexec to see what TensorRT will change on my graph with the command line

sudo /usr/src/tensorrt/bin/trtexec --onnx=CNN_shaped.onnx --fp16 --workspace=2048 --verbose

In the logs i’ve noticed that TensorRT select the fatest tactic to execute a layer but i didn’t understand what TensorRT is doing in this part of the logs, using Half means that TensorRT tests FP16 tactics but the shape of the Tensor is too different from the input shape which is (1,3,32,32)

`*************** Autotuning format combination: Half(1,32,1024:2,2048) → Half(1,30,900:2,900) ***************
*************** Autotuning format combination: Half(1,32,1:4,1024) → Half(1,30,1:8,900) ***************
*************** Autotuning format combination: Half(1,32,1:8,1024) → Half(1,30,1:8,900) ***************

CNN in netron

image
image221×948 24.9 KB

Environment

TensorRT Version: 7.7.3
GPU Type: Volta 512 CUDA Cores 64 TensorCores
Nvidia Driver Version:
CUDA Version: 10.2
CUDNN Version: 8.0
Operating System + Version: Ubuntu 18.04 L4T 32
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Can you please explain whats the shapes in Autotuning format combination part mean ?

Thanks

2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

4

Hi @NVES,

I checked my model with check_model.py, that are the logs using trtexec:


&&&& RUNNING TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=CNN_shaped.onnx --fp16 --workspace=2048 --saveEngine=CNN_shaped.trt --verbose --dumpProfile
[06/28/2021-16:30:35] [I] === Model Options ===
[06/28/2021-16:30:35] [I] Format: ONNX
[06/28/2021-16:30:35] [I] Model: CNN_shaped.onnx
[06/28/2021-16:30:35] [I] Output:
[06/28/2021-16:30:35] [I] === Build Options ===
[06/28/2021-16:30:35] [I] Max batch: 1
[06/28/2021-16:30:35] [I] Workspace: 2048 MB
[06/28/2021-16:30:35] [I] minTiming: 1
[06/28/2021-16:30:35] [I] avgTiming: 8
[06/28/2021-16:30:35] [I] Precision: FP32+FP16
[06/28/2021-16:30:35] [I] Calibration: 
[06/28/2021-16:30:35] [I] Safe mode: Disabled
[06/28/2021-16:30:35] [I] Save engine: CNN_shaped.trt
[06/28/2021-16:30:35] [I] Load engine: 
[06/28/2021-16:30:35] [I] Builder Cache: Enabled
[06/28/2021-16:30:35] [I] NVTX verbosity: 0
[06/28/2021-16:30:35] [I] Inputs format: fp32:CHW
[06/28/2021-16:30:35] [I] Outputs format: fp32:CHW
[06/28/2021-16:30:35] [I] Input build shapes: model
[06/28/2021-16:30:35] [I] Input calibration shapes: model
[06/28/2021-16:30:35] [I] === System Options ===
[06/28/2021-16:30:35] [I] Device: 0
[06/28/2021-16:30:35] [I] DLACore: 
[06/28/2021-16:30:35] [I] Plugins:
[06/28/2021-16:30:35] [I] === Inference Options ===
[06/28/2021-16:30:35] [I] Batch: 1
[06/28/2021-16:30:35] [I] Input inference shapes: model
[06/28/2021-16:30:35] [I] Iterations: 10
[06/28/2021-16:30:35] [I] Duration: 3s (+ 200ms warm up)
[06/28/2021-16:30:35] [I] Sleep time: 0ms
[06/28/2021-16:30:35] [I] Streams: 1
[06/28/2021-16:30:35] [I] ExposeDMA: Disabled
[06/28/2021-16:30:35] [I] Spin-wait: Disabled
[06/28/2021-16:30:35] [I] Multithreading: Disabled
[06/28/2021-16:30:35] [I] CUDA Graph: Disabled
[06/28/2021-16:30:35] [I] Skip inference: Disabled
[06/28/2021-16:30:35] [I] Inputs:
[06/28/2021-16:30:35] [I] === Reporting Options ===
[06/28/2021-16:30:35] [I] Verbose: Enabled
[06/28/2021-16:30:35] [I] Averages: 10 inferences
[06/28/2021-16:30:35] [I] Percentile: 99
[06/28/2021-16:30:35] [I] Dump output: Disabled
[06/28/2021-16:30:35] [I] Profile: Enabled
[06/28/2021-16:30:35] [I] Export timing to JSON file: 
[06/28/2021-16:30:35] [I] Export output to JSON file: 
[06/28/2021-16:30:35] [I] Export profile to JSON file: 
[06/28/2021-16:30:35] [I] 
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::Proposal version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::Split version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[06/28/2021-16:30:35] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
----------------------------------------------------------------
Input filename:   CNN_shaped.onnx
ONNX IR version:  0.0.7
Opset version:    13
Producer name:    keras2onnx
Producer version: 1.8.1
Domain:           onnxmltools
Model version:    0
Doc string:       
----------------------------------------------------------------
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::Split version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:202: Adding network input: my_input with dtype: float32, dimensions: (1, 32, 32, 3)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: my_input for ONNX tensor: my_input
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: dense_1/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: dense_1/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: dense/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: dense/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2d_2/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2d_2/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2d_1/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2d_1/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2d/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv2d/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: Transpose6 [Transpose]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: my_input
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: Transpose6 [Transpose] inputs: [my_input -> (1, 32, 32, 3)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: Transpose6 for ONNX node: Transpose6
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: adjusted_input2 for ONNX tensor: adjusted_input2
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: Transpose6 [Transpose] outputs: [adjusted_input2 -> (1, 3, 32, 32)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d [Conv]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: adjusted_input2
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: conv2d [Conv] inputs: [adjusted_input2 -> (1, 3, 32, 32)], [conv2d/kernel:0 -> (1, 3, 3, 3)], [conv2d/bias:0 -> (1)], 
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 3, 32, 32)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d for ONNX node: conv2d
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 1
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 1, 30, 30)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output2 for ONNX tensor: convolution_output2
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: conv2d [Conv] outputs: [convolution_output2 -> (1, 1, 30, 30)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu3 [Relu]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output2
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: Relu3 [Relu] inputs: [convolution_output2 -> (1, 1, 30, 30)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu3 for ONNX node: Relu3
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: conv2d/Relu:0 for ONNX tensor: conv2d/Relu:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: Relu3 [Relu] outputs: [conv2d/Relu:0 -> (1, 1, 30, 30)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d/Relu:0_pooling [MaxPool]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d/Relu:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: conv2d/Relu:0_pooling [MaxPool] inputs: [conv2d/Relu:0 -> (1, 1, 30, 30)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d/Relu:0_pooling for ONNX node: conv2d/Relu:0_pooling
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: conv2d/Relu:0_pooling0 for ONNX tensor: conv2d/Relu:0_pooling0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: conv2d/Relu:0_pooling [MaxPool] outputs: [conv2d/Relu:0_pooling0 -> (1, 1, 15, 15)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_1 [Conv]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d/Relu:0_pooling0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_1/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_1/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: conv2d_1 [Conv] inputs: [conv2d/Relu:0_pooling0 -> (1, 1, 15, 15)], [conv2d_1/kernel:0 -> (64, 1, 3, 3)], [conv2d_1/bias:0 -> (64)], 
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 1, 15, 15)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_1 for ONNX node: conv2d_1
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 64
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 64, 13, 13)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output1 for ONNX tensor: convolution_output1
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: conv2d_1 [Conv] outputs: [convolution_output1 -> (1, 64, 13, 13)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu2 [Relu]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output1
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: Relu2 [Relu] inputs: [convolution_output1 -> (1, 64, 13, 13)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu2 for ONNX node: Relu2
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: conv2d_1/Relu:0 for ONNX tensor: conv2d_1/Relu:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: Relu2 [Relu] outputs: [conv2d_1/Relu:0 -> (1, 64, 13, 13)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_1/Relu:0_pooling [MaxPool]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_1/Relu:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: conv2d_1/Relu:0_pooling [MaxPool] inputs: [conv2d_1/Relu:0 -> (1, 64, 13, 13)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_1/Relu:0_pooling for ONNX node: conv2d_1/Relu:0_pooling
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: conv2d_1/Relu:0_pooling0 for ONNX tensor: conv2d_1/Relu:0_pooling0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: conv2d_1/Relu:0_pooling [MaxPool] outputs: [conv2d_1/Relu:0_pooling0 -> (1, 64, 6, 6)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: conv2d_2 [Conv]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_1/Relu:0_pooling0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_2/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_2/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: conv2d_2 [Conv] inputs: [conv2d_1/Relu:0_pooling0 -> (1, 64, 6, 6)], [conv2d_2/kernel:0 -> (128, 64, 3, 3)], [conv2d_2/bias:0 -> (128)], 
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 64, 6, 6)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: conv2d_2 for ONNX node: conv2d_2
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 128
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 128, 4, 4)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: convolution_output for ONNX tensor: convolution_output
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: conv2d_2 [Conv] outputs: [convolution_output -> (1, 128, 4, 4)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu1 [Relu]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: convolution_output
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: Relu1 [Relu] inputs: [convolution_output -> (1, 128, 4, 4)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu1 for ONNX node: Relu1
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: conv2d_2/Relu:0 for ONNX tensor: conv2d_2/Relu:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: Relu1 [Relu] outputs: [conv2d_2/Relu:0 -> (1, 128, 4, 4)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: PushTranspose_0 [Transpose]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv2d_2/Relu:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: PushTranspose_0 [Transpose] inputs: [conv2d_2/Relu:0 -> (1, 128, 4, 4)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: PushTranspose_0 for ONNX node: PushTranspose_0
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: push_transpose_out_0 for ONNX tensor: push_transpose_out_0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: PushTranspose_0 [Transpose] outputs: [push_transpose_out_0 -> (1, 4, 4, 128)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: flatten [Flatten]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: push_transpose_out_0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: flatten [Flatten] inputs: [push_transpose_out_0 -> (1, 4, 4, 128)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: flatten for ONNX node: flatten
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: flatten/Reshape:0 for ONNX tensor: flatten/Reshape:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: flatten [Flatten] outputs: [flatten/Reshape:0 -> (1, 2048)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: dense [MatMul]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: flatten/Reshape:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: dense/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: dense [MatMul] inputs: [flatten/Reshape:0 -> (1, 2048)], [dense/kernel:0 -> (2048, 64)], 
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:2053: GEMM: using FC layer instead of MM because all criteria were met.
[06/28/2021-16:30:36] [V] [TRT] onnx2trt_utils.cpp:1793: Original shape: (1, 2048), unsqueezing to: (1, 1, 1, 2048)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: dense for ONNX node: dense
[06/28/2021-16:30:36] [V] [TRT] onnx2trt_utils.cpp:1641: Original shape: (1, 64, 1, 1), squeezing to: (1, 64)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: dense0 for ONNX tensor: dense0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: dense [MatMul] outputs: [dense0 -> (1, 64)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: Add1 [Add]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: dense0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: dense/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: Add1 [Add] inputs: [dense0 -> (1, 64)], [dense/bias:0 -> (64)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: Add1 for ONNX node: Add1
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: biased_tensor_name1 for ONNX tensor: biased_tensor_name1
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: Add1 [Add] outputs: [biased_tensor_name1 -> (1, 64)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu [Relu]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: biased_tensor_name1
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: Relu [Relu] inputs: [biased_tensor_name1 -> (1, 64)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu for ONNX node: Relu
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: dense/Relu:0 for ONNX tensor: dense/Relu:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: Relu [Relu] outputs: [dense/Relu:0 -> (1, 64)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: dense_1 [MatMul]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: dense/Relu:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: dense_1/kernel:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: dense_1 [MatMul] inputs: [dense/Relu:0 -> (1, 64)], [dense_1/kernel:0 -> (64, 10)], 
[06/28/2021-16:30:36] [V] [TRT] builtin_op_importers.cpp:2053: GEMM: using FC layer instead of MM because all criteria were met.
[06/28/2021-16:30:36] [V] [TRT] onnx2trt_utils.cpp:1793: Original shape: (1, 64), unsqueezing to: (1, 1, 1, 64)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: dense_1 for ONNX node: dense_1
[06/28/2021-16:30:36] [V] [TRT] onnx2trt_utils.cpp:1641: Original shape: (1, 10, 1, 1), squeezing to: (1, 10)
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: dense_10 for ONNX tensor: dense_10
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: dense_1 [MatMul] outputs: [dense_10 -> (1, 10)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: Add [Add]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: dense_10
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: dense_1/bias:0
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: Add [Add] inputs: [dense_10 -> (1, 10)], [dense_1/bias:0 -> (10)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: Add for ONNX node: Add
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: biased_tensor_name for ONNX tensor: biased_tensor_name
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: Add [Add] outputs: [biased_tensor_name -> (1, 10)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:103: Parsing node: softmax [Softmax]
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:119: Searching for input: biased_tensor_name
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:125: softmax [Softmax] inputs: [biased_tensor_name -> (1, 10)], 
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:141: Registering layer: softmax for ONNX node: softmax
[06/28/2021-16:30:36] [V] [TRT] ImporterContext.hpp:116: Registering tensor: softmax_1 for ONNX tensor: softmax
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:179: softmax [Softmax] outputs: [softmax -> (1, 10)], 
[06/28/2021-16:30:36] [V] [TRT] ModelImporter.cpp:507: Marking softmax_1 as output: softmax
 ----- Parsing of ONNX model CNN_shaped.onnx is Done ---- 
[06/28/2021-16:30:36] [V] [TRT] Applying generic optimizations to the graph for inference.
[06/28/2021-16:30:36] [V] [TRT] Original: 27 layers
[06/28/2021-16:30:36] [V] [TRT] After dead-layer removal: 27 layers
[06/28/2021-16:30:36] [V] [TRT] Fusing PushTranspose_0 with flatten
[06/28/2021-16:30:36] [V] [TRT] Fusing PushTranspose_0 + flatten with (Unnamed Layer* 12) [Shuffle]
[06/28/2021-16:30:36] [V] [TRT] Fusing (Unnamed Layer* 15) [Constant] with (Unnamed Layer* 16) [Shuffle]
[06/28/2021-16:30:36] [V] [TRT] Fusing (Unnamed Layer* 23) [Constant] with (Unnamed Layer* 24) [Shuffle]
[06/28/2021-16:30:36] [V] [TRT] Removing (Unnamed Layer* 26) [Shuffle]
[06/28/2021-16:30:36] [V] [TRT] Removing (Unnamed Layer* 28) [Shuffle]
[06/28/2021-16:30:36] [V] [TRT] After Myelin optimization: 21 layers
[06/28/2021-16:30:36] [V] [TRT] After scale fusion: 21 layers
[06/28/2021-16:30:36] [V] [TRT] -----------SqueezePushDown kSQUEEZE_JOIN case: dense --> (Unnamed Layer* 14) [Shuffle] --> Add1
[06/28/2021-16:30:36] [V] [TRT] -----------SqueezePushDown kSQUEEZE_JOIN case: dense_1 --> (Unnamed Layer* 22) [Shuffle] --> Add
[06/28/2021-16:30:36] [V] [TRT] Fusing (Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] with unsqueeze_node_after_(Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle]
[06/28/2021-16:30:36] [V] [TRT] Fusing (Unnamed Layer* 23) [Constant] + (Unnamed Layer* 24) [Shuffle] with unsqueeze_node_after_(Unnamed Layer* 23) [Constant] + (Unnamed Layer* 24) [Shuffle]
[06/28/2021-16:30:36] [V] [TRT] -----------SqueezePushDown kSQUEEZE_FORK case: Add1 --> copied_squeeze_after_Add1 --> Relu
[06/28/2021-16:30:36] [V] [TRT] Fusing squeeze_after_Relu with (Unnamed Layer* 20) [Shuffle]
[06/28/2021-16:30:36] [V] [TRT] Fusing conv2d with Relu3
[06/28/2021-16:30:36] [V] [TRT] Fusing conv2d_1 with Relu2
[06/28/2021-16:30:36] [V] [TRT] Fusing conv2d_2 with Relu1
[06/28/2021-16:30:36] [V] [TRT] Fusing (Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] + unsqueeze_node_after_(Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] with Add1
[06/28/2021-16:30:36] [V] [TRT] Fusing (Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] + unsqueeze_node_after_(Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] + Add1 with Relu
[06/28/2021-16:30:36] [V] [TRT] Fusing (Unnamed Layer* 23) [Constant] + (Unnamed Layer* 24) [Shuffle] + unsqueeze_node_after_(Unnamed Layer* 23) [Constant] + (Unnamed Layer* 24) [Shuffle] with Add
[06/28/2021-16:30:36] [V] [TRT] Fusing dense with (Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] + unsqueeze_node_after_(Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] + Add1 + Relu
[06/28/2021-16:30:36] [V] [TRT] Fusing dense_1 with (Unnamed Layer* 23) [Constant] + (Unnamed Layer* 24) [Shuffle] + unsqueeze_node_after_(Unnamed Layer* 23) [Constant] + (Unnamed Layer* 24) [Shuffle] + Add
[06/28/2021-16:30:36] [V] [TRT] After vertical fusions: 12 layers
[06/28/2021-16:30:36] [V] [TRT] After final dead-layer removal: 12 layers
[06/28/2021-16:30:36] [V] [TRT] After tensor merging: 12 layers
[06/28/2021-16:30:36] [V] [TRT] After concat removal: 12 layers
[06/28/2021-16:30:36] [V] [TRT] Graph construction and optimization completed in 0.00534046 seconds.
[06/28/2021-16:30:36] [I] [TRT] 
[06/28/2021-16:30:36] [I] [TRT] --------------- Layers running on DLA: 
[06/28/2021-16:30:36] [I] [TRT] 
[06/28/2021-16:30:36] [I] [TRT] --------------- Layers running on GPU: 
[06/28/2021-16:30:36] [I] [TRT] Transpose6, conv2d + Relu3, conv2d/Relu:0_pooling, conv2d_1 + Relu2, conv2d_1/Relu:0_pooling, conv2d_2 + Relu1, PushTranspose_0 + flatten + (Unnamed Layer* 12) [Shuffle], dense + (Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] + unsqueeze_node_after_(Unnamed Layer* 15) [Constant] + (Unnamed Layer* 16) [Shuffle] + Add1 + Relu, squeeze_after_Relu + (Unnamed Layer* 20) [Shuffle], dense_1 + (Unnamed Layer* 23) [Constant] + (Unnamed Layer* 24) [Shuffle] + unsqueeze_node_after_(Unnamed Layer* 23) [Constant] + (Unnamed Layer* 24) [Shuffle] + Add, copied_squeeze_after_Add, softmax, 
[06/28/2021-16:30:38] [V] [TRT] Constructing optimization profile number 0 [1/1].
[06/28/2021-16:30:38] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:38] [V] [TRT] Tactic: 1002 time 0.009896
[06/28/2021-16:30:38] [V] [TRT] Tactic: 0 time 0.00804
[06/28/2021-16:30:38] [V] [TRT] Fastest Tactic: 0 Time: 0.00804
[06/28/2021-16:30:38] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:38] [V] [TRT] Tactic: 1002 time 0.00762
[06/28/2021-16:30:38] [V] [TRT] Tactic: 0 time 0.007392
[06/28/2021-16:30:38] [V] [TRT] Fastest Tactic: 0 Time: 0.007392
[06/28/2021-16:30:38] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:38] [V] [TRT] Tactic: 1002 time 0.012532
[06/28/2021-16:30:38] [V] [TRT] Tactic: 0 time 0.005916
[06/28/2021-16:30:38] [V] [TRT] Fastest Tactic: 0 Time: 0.005916
[06/28/2021-16:30:38] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:38] [V] [TRT] Tactic: 1002 time 0.00754
[06/28/2021-16:30:38] [V] [TRT] Tactic: 0 time 0.00458
[06/28/2021-16:30:38] [V] [TRT] Fastest Tactic: 0 Time: 0.00458
[06/28/2021-16:30:38] [V] [TRT] *************** Autotuning format combination: Float(1,3,96,3072) -> Float(1,32,1024,3072) ***************
[06/28/2021-16:30:38] [V] [TRT] --------------- Timing Runner: Transpose6 (Shuffle)
[06/28/2021-16:30:38] [V] [TRT] Tactic: 0 time 0.004536
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1 time 0.006108
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004536
[06/28/2021-16:30:39] [V] [TRT] *************** Autotuning format combination: Float(1,3,96:32,96) -> Float(1,32,1024:32,1024) ***************
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: Transpose6 (Shuffle)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.006116
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1 time 0.009392
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.006116
[06/28/2021-16:30:39] [V] [TRT] *************** Autotuning format combination: Half(1,3,96,3072) -> Half(1,32,1024,3072) ***************
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: Transpose6 (Shuffle)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004496
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1 time 0.00624
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004496
[06/28/2021-16:30:39] [V] [TRT] *************** Autotuning format combination: Half(1,3,96:2,1536) -> Half(1,32,1024:2,2048) ***************
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: Transpose6 (Shuffle)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.005312
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1 time 0.010348
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.005312
[06/28/2021-16:30:39] [V] [TRT] *************** Autotuning format combination: Half(4,12,1:8,384) -> Half(1,32,1:8,1024) ***************
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: Transpose6 (Shuffle)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.005852
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1 time 0.01052
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.005852
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.004592
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.00488
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 1002 Time: 0.004592
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.009424
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.00458
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.00458
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.010936
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004448
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004448
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.010224
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004796
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004796
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.007592
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.00542
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.00542
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.007688
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004592
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004592
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.009568
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004472
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004472
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.009308
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004468
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004468
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.009304
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004728
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004728
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.004944
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004932
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004932
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.009176
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.00458
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.00458
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.010576
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.004492
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.004492
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:39] [V] [TRT] Tactic: 1002 time 0.01006
[06/28/2021-16:30:39] [V] [TRT] Tactic: 0 time 0.00448
[06/28/2021-16:30:39] [V] [TRT] Fastest Tactic: 0 Time: 0.00448
[06/28/2021-16:30:39] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.007624
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004608
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004608
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.00778
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004432
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004432
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.010084
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004504
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004504
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.010408
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004644
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004644
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.00944
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004564
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004564
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.009272
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004888
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004888
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.010392
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004508
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004508
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.00922
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004416
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004416
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.009204
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004476
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004476
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.009512
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004476
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004476
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.010292
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.00462
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.00462
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1002 time 0.009828
[06/28/2021-16:30:40] [V] [TRT] Tactic: 0 time 0.004572
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 0 Time: 0.004572
[06/28/2021-16:30:40] [V] [TRT] *************** Autotuning format combination: Float(1,32,1024,3072) -> Float(1,30,900,900) ***************
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (FusedConvActConvolution)
[06/28/2021-16:30:40] [V] [TRT] Tactic: 524287 time 0.009436
[06/28/2021-16:30:40] [V] [TRT] Tactic: 8585215 time 0.010652
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 524287 Time: 0.009436
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CaskConvolution)
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1754569683116234317 time 0.016372
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 1825138533642645384 time 0.015788
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 2733356012094739613 time 0.012668
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 2775507031594384867 time 0.00928
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 2842488832350522458 time 0.01434
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 3915320020053085238 time 0.015636
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 6448355332020552203 time 0.0162
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 6808617066150061604 time 0.013932
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: 9091006216302412844 time 0.013632
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: -8060443123034038864 time 0.014152
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: -4420849921117327522 time 0.009704
[06/28/2021-16:30:40] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[06/28/2021-16:30:40] [V] [TRT] Tactic: -3946921629105938337 time 0.012328
[06/28/2021-16:30:40] [V] [TRT] Fastest Tactic: 2775507031594384867 Time: 0.00928
[06/28/2021-16:30:40] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaConvolution)
[06/28/2021-16:30:41] [V] [TRT] Tactic: 0 time 0.019008
[06/28/2021-16:30:41] [V] [TRT] Tactic: 2 time 0.01892
[06/28/2021-16:30:41] [V] [TRT] Tactic: 5 time 0.050928
[06/28/2021-16:30:41] [V] [TRT] Tactic: 6 time 0.024464
[06/28/2021-16:30:41] [V] [TRT] Tactic: 57 time 0.019484
[06/28/2021-16:30:41] [V] [TRT] Fastest Tactic: 2 Time: 0.01892
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaDepthwiseConvolution)
[06/28/2021-16:30:41] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2775507031594384867
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:41] [V] [TRT] 
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:41] [V] [TRT] *************** Autotuning format combination: Half(1,32,1024,3072) -> Half(1,30,900,900) ***************
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (FusedConvActConvolution)
[06/28/2021-16:30:41] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CaskConvolution)
[06/28/2021-16:30:41] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaConvolution)
[06/28/2021-16:30:41] [V] [TRT] Tactic: 0 time 0.019292
[06/28/2021-16:30:41] [V] [TRT] Tactic: 1 time 0.019008
[06/28/2021-16:30:41] [V] [TRT] Tactic: 2 time 0.02002
[06/28/2021-16:30:41] [V] [TRT] Tactic: 4 time 0.042012
[06/28/2021-16:30:41] [V] [TRT] Tactic: 5 time 0.04676
[06/28/2021-16:30:41] [V] [TRT] Tactic: 6 time 0.112576
[06/28/2021-16:30:41] [V] [TRT] Fastest Tactic: 1 Time: 0.019008
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaDepthwiseConvolution)
[06/28/2021-16:30:41] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaConvolution Tactic: 1
[06/28/2021-16:30:41] [V] [TRT] 
[06/28/2021-16:30:41] [V] [TRT] *************** Autotuning format combination: Half(1,32,1024:2,2048) -> Half(1,30,900:2,900) ***************
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn_winograd) Set Tactic Name: volta_fp16x2_hcudnn_winograd_fp16x2_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (FusedConvActConvolution)
[06/28/2021-16:30:41] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CaskConvolution)
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: 1145226902788474763 time 0.012436
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: 2418518597804310654 time 0.010824
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: 8292881859266835088 time 0.012216
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: 8401509141903434922 time 0.01136
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -8654297089785671176 time 0.01226
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -7448936905981214224 time 0.012352
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -7140760933967189247 time 0.011264
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -4097850214384059472 time 0.011092
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -3689982367035295496 time 0.012744
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn_winograd) Set Tactic Name: volta_fp16x2_hcudnn_winograd_fp16x2_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -3140347171730126532 time 0.00912
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -2534402059426524406 time 0.012864
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -2027588946874785071 time 0.010988
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] Tactic: -245090590808296743 time 0.012776
[06/28/2021-16:30:41] [V] [TRT] Fastest Tactic: -3140347171730126532 Time: 0.00912
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaConvolution)
[06/28/2021-16:30:41] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaDepthwiseConvolution)
[06/28/2021-16:30:41] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -3140347171730126532
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn_winograd) Set Tactic Name: volta_fp16x2_hcudnn_winograd_fp16x2_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:41] [V] [TRT] 
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn_winograd) Set Tactic Name: volta_fp16x2_hcudnn_winograd_fp16x2_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1
[06/28/2021-16:30:41] [V] [TRT] conv2d + Relu3 (hcudnn_winograd) Set Tactic Name: volta_fp16x2_hcudnn_winograd_fp16x2_128x128_ldg1_ldg4_relu_tile148t_nt_v1
[06/28/2021-16:30:41] [V] [TRT] *************** Autotuning format combination: Half(1,32,1:4,1024) -> Half(1,30,1:8,900) ***************
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (FusedConvActConvolution)
[06/28/2021-16:30:41] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CaskConvolution)
[06/28/2021-16:30:41] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaConvolution)
[06/28/2021-16:30:41] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaDepthwiseConvolution)
[06/28/2021-16:30:41] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] *************** Autotuning format combination: Half(1,32,1:8,1024) -> Float(1,30,900,900) ***************
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (FusedConvActConvolution)
[06/28/2021-16:30:41] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CaskConvolution)
[06/28/2021-16:30:41] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaConvolution)
[06/28/2021-16:30:41] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaDepthwiseConvolution)
[06/28/2021-16:30:41] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:41] [V] [TRT] *************** Autotuning format combination: Half(1,32,1:8,1024) -> Half(1,30,1:8,900) ***************
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (FusedConvActConvolution)
[06/28/2021-16:30:42] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CaskConvolution)
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: 2105695814191699972 time 0.016976
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: 3754069740140581927 time 0.017064
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: 5666160310350604399 time 0.017132
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: 5925270497649423688 time 0.02354
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: 6680916730816870145 time 0.02564
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: 7158029511300006471 time 0.01716
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: 7859952145590271433 time 0.025368
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: 8283847742354150423 time 0.017156
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: -8391760416076885205 time 0.023116
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: -4534876761957424274 time 0.023672
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: -3237051169894153788 time 0.017084
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] Tactic: -2422160065350346448 time 0.025408
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 2105695814191699972 Time: 0.016976
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaConvolution)
[06/28/2021-16:30:42] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: conv2d + Relu3 (CudaDepthwiseConvolution)
[06/28/2021-16:30:42] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping
[06/28/2021-16:30:42] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2105695814191699972
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] 
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] conv2d + Relu3 (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.004616
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004564
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004564
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.00812
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.00462
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.00462
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.009304
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004356
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004356
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.00486
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004576
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004576
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.00782
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004556
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004556
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.009164
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004548
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004548
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.007556
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004584
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004584
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.0083
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004468
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004468
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.009012
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004472
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004472
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.006164
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004484
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004484
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.007788
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.00448
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.00448
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 1002 time 0.009384
[06/28/2021-16:30:42] [V] [TRT] Tactic: 0 time 0.004488
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: 0 Time: 0.004488
[06/28/2021-16:30:42] [V] [TRT] *************** Autotuning format combination: Float(1,30,900,900) -> Float(1,15,225,225) ***************
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: conv2d/Relu:0_pooling (Pooling)
[06/28/2021-16:30:42] [V] [TRT] Tactic: -1 time 0.005344
[06/28/2021-16:30:42] [V] [TRT] Fastest Tactic: -1 Time: 0.005344
[06/28/2021-16:30:42] [V] [TRT] --------------- Timing Runner: conv2d/Relu:0_pooling (TiledPooling)
[06/28/2021-16:30:42] [V] [TRT] Tactic: 5505281 time 0.004644
[06/28/2021-16:30:42] [V] [TRT] Tactic: 5570817 time 0.004492
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5636353 time 0.004656
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5701889 time 0.004548
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5767425 time 0.004452
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5832961 time 0.004532
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5898497 time 0.00468
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5964033 time 0.004724
[06/28/2021-16:30:43] [V] [TRT] Tactic: 6029569 time 0.00478
[06/28/2021-16:30:43] [V] [TRT] Tactic: 6095105 time 0.004592
[06/28/2021-16:30:43] [V] [TRT] Tactic: 6160641 time 0.004564
[06/28/2021-16:30:43] [V] [TRT] Tactic: 6226177 time 0.004708
[06/28/2021-16:30:43] [V] [TRT] Tactic: 6291713 time 0.004732
[06/28/2021-16:30:43] [V] [TRT] Tactic: 6357249 time 0.004648
[06/28/2021-16:30:43] [V] [TRT] Tactic: 6422785 time 0.004536
[06/28/2021-16:30:43] [V] [TRT] Tactic: 6488321 time 0.00474
[06/28/2021-16:30:43] [V] [TRT] Fastest Tactic: 5767425 Time: 0.004452
[06/28/2021-16:30:43] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: TiledPooling Tactic: 5767425
[06/28/2021-16:30:43] [V] [TRT] 
[06/28/2021-16:30:43] [V] [TRT] *************** Autotuning format combination: Half(1,30,900,900) -> Half(1,15,225,225) ***************
[06/28/2021-16:30:43] [V] [TRT] --------------- Timing Runner: conv2d/Relu:0_pooling (Pooling)
[06/28/2021-16:30:43] [V] [TRT] Tactic: -1 time 0.004664
[06/28/2021-16:30:43] [V] [TRT] Fastest Tactic: -1 Time: 0.004664
[06/28/2021-16:30:43] [V] [TRT] --------------- Timing Runner: conv2d/Relu:0_pooling (TiledPooling)
[06/28/2021-16:30:43] [V] [TRT] TiledPooling has no valid tactics for this config, skipping
[06/28/2021-16:30:43] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Pooling Tactic: -1
[06/28/2021-16:30:43] [V] [TRT] 
[06/28/2021-16:30:43] [V] [TRT] *************** Autotuning format combination: Half(1,30,900:2,900) -> Half(1,15,225:2,225) ***************
[06/28/2021-16:30:43] [V] [TRT] --------------- Timing Runner: conv2d/Relu:0_pooling (Pooling)
[06/28/2021-16:30:43] [V] [TRT] Tactic: -3 time 0.004588
[06/28/2021-16:30:43] [V] [TRT] Fastest Tactic: -3 Time: 0.004588
[06/28/2021-16:30:43] [V] [TRT] --------------- Timing Runner: conv2d/Relu:0_pooling (TiledPooling)
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5505281 time 0.004816
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5570817 time 0.004912
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5636353 time 0.0047
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5701889 time 0.004924
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5767425 time 0.004696
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5832961 time 0.0044
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5898497 time 0.004464
[06/28/2021-16:30:43] [V] [TRT] Tactic: 5964033 time 0.005296
[06/28/2021-16:30:44] [V] [TRT] Tactic: 6029569 time 0.004532
[06/28/2021-16:30:44] [V] [TRT] Tactic: 6095105 time 0.004492
[06/28/2021-16:30:44] [V] [TRT] Tactic: 6160641 time 0.004724
[06/28/2021-16:30:44] [V] [TRT] Tactic: 6226177 time 0.004768
[06/28/2021-16:30:44] [V] [TRT] Tactic: 6291713 time 0.004708
[06/28/2021-16:30:44] [V] [TRT] Tactic: 6357249 time 0.004452
[06/28/2021-16:30:44] [V] [TRT] Tactic: 6422785 time 0.00504
[06/28/2021-16:30:44] [V] [TRT] Tactic: 6488321 time 0.004932
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 5832961 Time: 0.0044
[06/28/2021-16:30:44] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: TiledPooling Tactic: 5832961
[06/28/2021-16:30:44] [V] [TRT] 
[06/28/2021-16:30:44] [V] [TRT] *************** Autotuning format combination: Half(1,30,1:8,900) -> Half(1,15,1:8,225) ***************
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: conv2d/Relu:0_pooling (Pooling)
[06/28/2021-16:30:44] [V] [TRT] Tactic: -2 time 0.004848
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: -2 Time: 0.004848
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: conv2d/Relu:0_pooling (TiledPooling)
[06/28/2021-16:30:44] [V] [TRT] TiledPooling has no valid tactics for this config, skipping
[06/28/2021-16:30:44] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Pooling Tactic: -2
[06/28/2021-16:30:44] [V] [TRT] 
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.004816
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.004488
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.004488
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.006104
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.004488
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.004488
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.0072
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.00454
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.00454
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.007516
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.00462
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.00462
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.00456
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.004444
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.004444
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.006416
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.00454
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.00454
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.007792
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.004624
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.004624
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.006216
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.004544
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.004544
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.007468
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.004612
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.004612
[06/28/2021-16:30:44] [V] [TRT] --------------- Timing Runner: <reformat> (Reformat)
[06/28/2021-16:30:44] [V] [TRT] Tactic: 1002 time 0.008212
[06/28/2021-16:30:44] [V] [TRT] Tactic: 0 time 0.00468
[06/28/2021-16:30:44] [V] [TRT] Fastest Tactic: 0 Time: 0.00468

[06/28/2021-16:31:01] [I] 
&&&& PASSED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=CNN_shaped.onnx --fp16 --workspace=2048 --saveEngine=CNN_shaped.trt --verbose --dumpProfile
5

@chakibdace,

Actually those are strides, not dimensions. TRT tries all different kinds of formats (which are represented by their strides).

Thank you.

6

Hi @spolisetty,

Thank you for your answer, in an operation like convolution or pooling, the stride is fixed to a value, but in the line of log below, i couldn’t know where is stride.
Can you please tell me in this line where are the strides in the line below or can you explain me this line ?

Autotuning format combination: Half(1,416,1:4,173056) → Half(2,832,1:8,346112)

Thanks

7

@chakibdace,

It represents Tensor Stride (not conv or pooling stride). We often see 1:4 and 1:8. It means that dimension is vectorized.
(1, 416, 1:4, 173056) This should be an HWC4 format.
Please refer API doc of “TensorFormat” for more explanation. Layer Base Classes — NVIDIA TensorRT Standard Python API Documentation 8.4.3 documentation

Thank you.

1 Like
8

Thank you @spolisetty