&&&& RUNNING TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --fp16 --workspace=64 --iterations=1 --warmUp=0 --duration=0 --verbose --dumpProfile --onnx=/home/shield/data/dla_correctness.models/release_prefix/model1.onnx --exportOutput=/tmp/tmpoa1umvwa.json [07/19/2021-12:42:45] [I] === Model Options === [07/19/2021-12:42:45] [I] Format: ONNX [07/19/2021-12:42:45] [I] Model: /home/shield/data/dla_correctness.models/release_prefix/model1.onnx [07/19/2021-12:42:45] [I] Output: [07/19/2021-12:42:45] [I] === Build Options === [07/19/2021-12:42:45] [I] Max batch: 1 [07/19/2021-12:42:45] [I] Workspace: 64 MB [07/19/2021-12:42:45] [I] minTiming: 1 [07/19/2021-12:42:45] [I] avgTiming: 8 [07/19/2021-12:42:45] [I] Precision: FP32+FP16 [07/19/2021-12:42:45] [I] Calibration: [07/19/2021-12:42:45] [I] Safe mode: Disabled [07/19/2021-12:42:45] [I] Save engine: [07/19/2021-12:42:45] [I] Load engine: [07/19/2021-12:42:45] [I] Builder Cache: Enabled [07/19/2021-12:42:45] [I] NVTX verbosity: 0 [07/19/2021-12:42:45] [I] Inputs format: fp32:CHW [07/19/2021-12:42:45] [I] Outputs format: fp32:CHW [07/19/2021-12:42:45] [I] Input build shapes: model [07/19/2021-12:42:45] [I] Input calibration shapes: model [07/19/2021-12:42:45] [I] === System Options === [07/19/2021-12:42:45] [I] Device: 0 [07/19/2021-12:42:45] [I] DLACore: [07/19/2021-12:42:45] [I] Plugins: [07/19/2021-12:42:45] [I] === Inference Options === [07/19/2021-12:42:45] [I] Batch: 1 [07/19/2021-12:42:45] [I] Input inference shapes: model [07/19/2021-12:42:45] [I] Iterations: 1 [07/19/2021-12:42:45] [I] Duration: 0s (+ 0ms warm up) [07/19/2021-12:42:45] [I] Sleep time: 0ms [07/19/2021-12:42:45] [I] Streams: 1 [07/19/2021-12:42:45] [I] ExposeDMA: Disabled [07/19/2021-12:42:45] [I] Spin-wait: Disabled [07/19/2021-12:42:45] [I] Multithreading: Disabled [07/19/2021-12:42:45] [I] CUDA Graph: Disabled [07/19/2021-12:42:45] [I] Skip inference: Disabled [07/19/2021-12:42:45] [I] Inputs: [07/19/2021-12:42:45] [I] === Reporting Options === [07/19/2021-12:42:45] [I] Verbose: Enabled [07/19/2021-12:42:45] [I] Averages: 10 inferences [07/19/2021-12:42:45] [I] Percentile: 99 [07/19/2021-12:42:45] [I] Dump output: Disabled [07/19/2021-12:42:45] [I] Profile: Enabled [07/19/2021-12:42:45] [I] Export timing to JSON file: [07/19/2021-12:42:45] [I] Export output to JSON file: /tmp/tmpoa1umvwa.json [07/19/2021-12:42:45] [I] Export profile to JSON file: [07/19/2021-12:42:45] [I] [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::Region_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::CropAndResize version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::Proposal version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::Split version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1 [07/19/2021-12:42:45] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1 ---------------------------------------------------------------- Input filename: /home/shield/data/dla_correctness.models/release_prefix/model1.onnx ONNX IR version: 0.0.4 Opset version: 7 Producer name: pytorch Producer version: 1.3 Domain: Model version: 0 Doc string: ---------------------------------------------------------------- [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::Proposal version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::Split version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:202: Adding network input: input.1 with dtype: float32, dimensions: (1, 2, 224, 224) [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: input.1 for ONNX tensor: input.1 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.0.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.bias [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.num_batches_tracked [07/19/2021-12:42:46] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.running_mean [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.running_var [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.0.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.bias [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.num_batches_tracked [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.running_mean [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.running_var [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.1.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.bias [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.num_batches_tracked [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.running_mean [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.running_var [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Conv] [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: input.1 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.0.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:125: [Conv] inputs: [input.1 -> (1, 2, 224, 224)], [0.0.weight -> (32, 2, 3, 3)], [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 2, 224, 224) [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 0) [Convolution] for ONNX node: [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (2, 2), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 32 [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 32, 112, 112) [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 19 for ONNX tensor: 19 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:179: [Conv] outputs: [19 -> (1, 32, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:103: Parsing node: [BatchNormalization] [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 19 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.1.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.1.bias [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.1.running_mean [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.1.running_var [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:125: [BatchNormalization] inputs: [19 -> (1, 32, 112, 112)], [0.1.weight -> (32)], [0.1.bias -> (32)], [0.1.running_mean -> (32)], [0.1.running_var -> (32)], [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 1) [Scale] for ONNX node: [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 20 for ONNX tensor: 20 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:179: [BatchNormalization] outputs: [20 -> (1, 32, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Clip] [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 20 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:125: [Clip] inputs: [20 -> (1, 32, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 2) [Activation] for ONNX node: [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 21 for ONNX tensor: 21 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:179: [Clip] outputs: [21 -> (1, 32, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Conv] [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 21 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.0.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:125: [Conv] inputs: [21 -> (1, 32, 112, 112)], [1.conv.0.0.weight -> (32, 1, 3, 3)], [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 32, 112, 112) [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 3) [Convolution] for ONNX node: [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 32 [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 32, 112, 112) [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 22 for ONNX tensor: 22 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:179: [Conv] outputs: [22 -> (1, 32, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:103: Parsing node: [BatchNormalization] [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 22 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.1.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.1.bias [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.1.running_mean [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.1.running_var [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:125: [BatchNormalization] inputs: [22 -> (1, 32, 112, 112)], [1.conv.0.1.weight -> (32)], [1.conv.0.1.bias -> (32)], [1.conv.0.1.running_mean -> (32)], [1.conv.0.1.running_var -> (32)], [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 4) [Scale] for ONNX node: [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 23 for ONNX tensor: 23 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:179: [BatchNormalization] outputs: [23 -> (1, 32, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Clip] [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 23 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:125: [Clip] inputs: [23 -> (1, 32, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 5) [Activation] for ONNX node: [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 24 for ONNX tensor: 24 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:179: [Clip] outputs: [24 -> (1, 32, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Conv] [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 24 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.1.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:125: [Conv] inputs: [24 -> (1, 32, 112, 112)], [1.conv.1.weight -> (16, 32, 1, 1)], [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 32, 112, 112) [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 6) [Convolution] for ONNX node: [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (1, 1), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 16 [07/19/2021-12:42:46] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 16, 112, 112) [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 25 for ONNX tensor: 25 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:179: [Conv] outputs: [25 -> (1, 16, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:103: Parsing node: [BatchNormalization] [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 25 [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.2.weight [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.2.bias [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.2.running_mean [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.2.running_var [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:125: [BatchNormalization] inputs: [25 -> (1, 16, 112, 112)], [1.conv.2.weight -> (16)], [1.conv.2.bias -> (16)], [1.conv.2.running_mean -> (16)], [1.conv.2.running_var -> (16)], [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 7) [Scale] for ONNX node: [07/19/2021-12:42:46] [V] [TRT] ImporterContext.hpp:116: Registering tensor: output_1 for ONNX tensor: output [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:179: [BatchNormalization] outputs: [output -> (1, 16, 112, 112)], [07/19/2021-12:42:46] [V] [TRT] ModelImporter.cpp:507: Marking output_1 as output: output ----- Parsing of ONNX model /home/shield/data/dla_correctness.models/release_prefix/model1.onnx is Done ---- [07/19/2021-12:42:46] [V] [TRT] Applying generic optimizations to the graph for inference. [07/19/2021-12:42:46] [V] [TRT] Original: 8 layers [07/19/2021-12:42:46] [V] [TRT] After dead-layer removal: 8 layers [07/19/2021-12:42:46] [V] [TRT] After Myelin optimization: 8 layers [07/19/2021-12:42:46] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 0) [Convolution] with scale (Unnamed Layer* 1) [Scale] [07/19/2021-12:42:46] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 3) [Convolution] with scale (Unnamed Layer* 4) [Scale] [07/19/2021-12:42:46] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 6) [Convolution] with scale (Unnamed Layer* 7) [Scale] [07/19/2021-12:42:46] [V] [TRT] After scale fusion: 5 layers [07/19/2021-12:42:46] [V] [TRT] Fusing (Unnamed Layer* 0) [Convolution] with (Unnamed Layer* 2) [Activation] [07/19/2021-12:42:46] [V] [TRT] Fusing (Unnamed Layer* 3) [Convolution] with (Unnamed Layer* 5) [Activation] [07/19/2021-12:42:46] [V] [TRT] After vertical fusions: 3 layers [07/19/2021-12:42:46] [V] [TRT] After final dead-layer removal: 3 layers [07/19/2021-12:42:46] [V] [TRT] After tensor merging: 3 layers [07/19/2021-12:42:46] [V] [TRT] After concat removal: 3 layers [07/19/2021-12:42:46] [V] [TRT] Graph construction and optimization completed in 0.00306237 seconds. [07/19/2021-12:42:46] [I] [TRT] [07/19/2021-12:42:46] [I] [TRT] --------------- Layers running on DLA: [07/19/2021-12:42:46] [I] [TRT] [07/19/2021-12:42:46] [I] [TRT] --------------- Layers running on GPU: [07/19/2021-12:42:46] [I] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation], (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation], (Unnamed Layer* 6) [Convolution], [07/19/2021-12:42:48] [V] [TRT] Constructing optimization profile number 0 [1/1]. [07/19/2021-12:42:48] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:48] [V] [TRT] Tactic: 1002 time 0.029316 [07/19/2021-12:42:48] [V] [TRT] Tactic: 0 time 0.04056 [07/19/2021-12:42:48] [V] [TRT] Fastest Tactic: 1002 Time: 0.029316 [07/19/2021-12:42:48] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:48] [V] [TRT] Tactic: 1002 time 0.422072 [07/19/2021-12:42:48] [V] [TRT] Tactic: 0 time 0.021404 [07/19/2021-12:42:48] [V] [TRT] Fastest Tactic: 0 Time: 0.021404 [07/19/2021-12:42:48] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:48] [V] [TRT] Tactic: 1002 time 0.527028 [07/19/2021-12:42:48] [V] [TRT] Tactic: 0 time 0.040296 [07/19/2021-12:42:48] [V] [TRT] Fastest Tactic: 0 Time: 0.040296 [07/19/2021-12:42:48] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:48] [V] [TRT] Tactic: 1002 time 0.53134 [07/19/2021-12:42:48] [V] [TRT] Tactic: 0 time 0.039796 [07/19/2021-12:42:48] [V] [TRT] Fastest Tactic: 0 Time: 0.039796 [07/19/2021-12:42:48] [V] [TRT] *************** Autotuning format combination: Float(1,224,50176,100352) -> Float(1,112,12544,401408) *************** [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:48] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:48] [V] [TRT] Tactic: 458751 time 0.471024 [07/19/2021-12:42:48] [V] [TRT] Fastest Tactic: 458751 Time: 0.471024 [07/19/2021-12:42:48] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:48] [V] [TRT] Tactic: 1825138533642645384 time 0.208248 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:48] [V] [TRT] Tactic: 2842488832350522458 time 0.128752 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:48] [V] [TRT] Tactic: 3915320020053085238 time 0.20676 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:48] [V] [TRT] Tactic: 6448355332020552203 time 0.304072 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:48] [V] [TRT] Tactic: 6808617066150061604 time 0.10854 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:48] [V] [TRT] Tactic: -8060443123034038864 time 0.111136 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:48] [V] [TRT] Tactic: -4420849921117327522 time 0.082256 [07/19/2021-12:42:48] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:48] [V] [TRT] Tactic: -3946921629105938337 time 0.0683 [07/19/2021-12:42:48] [V] [TRT] Fastest Tactic: -3946921629105938337 Time: 0.0683 [07/19/2021-12:42:48] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.269052 [07/19/2021-12:42:49] [V] [TRT] Tactic: 2 time 0.551376 [07/19/2021-12:42:49] [V] [TRT] Tactic: 5 time 6.36083 [07/19/2021-12:42:49] [V] [TRT] Tactic: 57 time 0.251804 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 57 Time: 0.251804 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -3946921629105938337 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] *************** Autotuning format combination: Half(1,224,50176,100352) -> Half(1,112,12544,401408) *************** [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:49] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:49] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.208208 [07/19/2021-12:42:49] [V] [TRT] Tactic: 1 time 0.206468 [07/19/2021-12:42:49] [V] [TRT] Tactic: 2 time 0.518164 [07/19/2021-12:42:49] [V] [TRT] Tactic: 5 time 6.29401 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 1 Time: 0.206468 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaConvolution Tactic: 1 [07/19/2021-12:42:49] [V] [TRT] [07/19/2021-12:42:49] [V] [TRT] *************** Autotuning format combination: Half(1,224,50176:2,50176) -> Half(1,112,12544:2,200704) *************** [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:49] [V] [TRT] Tactic: 458751 time 0.202296 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 458751 Time: 0.202296 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 1145226902788474763 time 0.0863 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 2418518597804310654 time 0.087212 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 8292881859266835088 time 0.06472 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 8401509141903434922 time 0.085168 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: -8654297089785671176 time 0.213172 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: -7448936905981214224 time 0.065852 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: -3689982367035295496 time 0.214984 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: -2027588946874785071 time 0.063668 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: -245090590808296743 time 0.216544 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: -2027588946874785071 Time: 0.063668 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -2027588946874785071 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:49] [V] [TRT] *************** Autotuning format combination: Half(1,224,1:4,50176) -> Half(4,448,1:8,50176) *************** [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:49] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:49] [V] [TRT] Tactic: -1886198213384953148 time 0.078072 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: -1886198213384953148 Time: 0.078072 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -1886198213384953148 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:49] [V] [TRT] [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:49] [V] [TRT] *************** Autotuning format combination: Half(1,224,1:8,50176) -> Float(1,112,12544,401408) *************** [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:49] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:49] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] *************** Autotuning format combination: Half(1,224,1:8,50176) -> Half(4,448,1:8,50176) *************** [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:49] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 3754069740140581927 time 0.265856 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 5925270497649423688 time 0.28724 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 6680916730816870145 time 0.320324 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 7158029511300006471 time 0.158288 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 7859952145590271433 time 0.313156 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: 8283847742354150423 time 0.270012 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: -4534876761957424274 time 0.281632 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] Tactic: -3237051169894153788 time 0.155732 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: -3237051169894153788 Time: 0.155732 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:49] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:49] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -3237051169894153788 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:49] [V] [TRT] Tactic: 1002 time 0.092272 [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.13428 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 1002 Time: 0.092272 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:49] [V] [TRT] Tactic: 1002 time 0.13896 [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.048312 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 0 Time: 0.048312 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:49] [V] [TRT] Tactic: 1002 time 0.097588 [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.120348 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 1002 Time: 0.097588 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:49] [V] [TRT] Tactic: 1002 time 0.140088 [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.044488 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 0 Time: 0.044488 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:49] [V] [TRT] Tactic: 1002 time 0.234412 [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.05762 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 0 Time: 0.05762 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:49] [V] [TRT] Tactic: 1002 time 0.237328 [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.04976 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 0 Time: 0.04976 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:49] [V] [TRT] Tactic: 1002 time 0.139248 [07/19/2021-12:42:49] [V] [TRT] Tactic: 0 time 0.040072 [07/19/2021-12:42:49] [V] [TRT] Fastest Tactic: 0 Time: 0.040072 [07/19/2021-12:42:49] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:49] [V] [TRT] Tactic: 1002 time 0.13912 [07/19/2021-12:42:50] [V] [TRT] Tactic: 0 time 0.047584 [07/19/2021-12:42:50] [V] [TRT] Fastest Tactic: 0 Time: 0.047584 [07/19/2021-12:42:50] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:50] [V] [TRT] Tactic: 1002 time 0.139588 [07/19/2021-12:42:50] [V] [TRT] Tactic: 0 time 0.035684 [07/19/2021-12:42:50] [V] [TRT] Fastest Tactic: 0 Time: 0.035684 [07/19/2021-12:42:50] [V] [TRT] *************** Autotuning format combination: Float(1,112,12544,401408) -> Float(1,112,12544,401408) *************** [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:50] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:50] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:50] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CaskConvolution) [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:50] [V] [TRT] Tactic: 1825138533642645384 time 5.60443 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:50] [V] [TRT] Tactic: 2842488832350522458 time 3.43542 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:50] [V] [TRT] Tactic: 3915320020053085238 time 5.55281 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:50] [V] [TRT] Tactic: 6448355332020552203 time 8.24422 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:50] [V] [TRT] Tactic: 6808617066150061604 time 2.87066 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:50] [V] [TRT] Tactic: -8060443123034038864 time 2.94144 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:50] [V] [TRT] Tactic: -4420849921117327522 time 1.83129 [07/19/2021-12:42:50] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:50] [V] [TRT] Tactic: -3946921629105938337 time 1.64932 [07/19/2021-12:42:50] [V] [TRT] Fastest Tactic: -3946921629105938337 Time: 1.64932 [07/19/2021-12:42:50] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaConvolution) [07/19/2021-12:42:50] [V] [TRT] Tactic: 0 time 0.284924 [07/19/2021-12:42:50] [V] [TRT] Tactic: 2 time 0.284976 [07/19/2021-12:42:51] [V] [TRT] Tactic: 5 time 25.8241 [07/19/2021-12:42:51] [V] [TRT] Tactic: 6 time 3.12849 [07/19/2021-12:42:51] [V] [TRT] Tactic: 57 time 0.275888 [07/19/2021-12:42:51] [V] [TRT] Fastest Tactic: 57 Time: 0.275888 [07/19/2021-12:42:51] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:51] [V] [TRT] Tactic: -1 time 0.113064 [07/19/2021-12:42:51] [V] [TRT] Fastest Tactic: -1 Time: 0.113064 [07/19/2021-12:42:51] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaDepthwiseConvolution Tactic: -1 [07/19/2021-12:42:51] [V] [TRT] [07/19/2021-12:42:51] [V] [TRT] *************** Autotuning format combination: Half(1,112,12544,401408) -> Half(1,112,12544,401408) *************** [07/19/2021-12:42:51] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:51] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:51] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CaskConvolution) [07/19/2021-12:42:51] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:51] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaConvolution) [07/19/2021-12:42:51] [V] [TRT] Tactic: 0 time 0.173648 [07/19/2021-12:42:51] [V] [TRT] Tactic: 1 time 0.174276 [07/19/2021-12:42:51] [V] [TRT] Tactic: 2 time 0.282344 [07/19/2021-12:42:51] [V] [TRT] Tactic: 4 time 14.7059 [07/19/2021-12:42:52] [V] [TRT] Tactic: 5 time 25.9266 [07/19/2021-12:42:52] [V] [TRT] Tactic: 6 time 8.39862 [07/19/2021-12:42:52] [V] [TRT] Fastest Tactic: 0 Time: 0.173648 [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:52] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:52] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaConvolution Tactic: 0 [07/19/2021-12:42:52] [V] [TRT] [07/19/2021-12:42:52] [V] [TRT] *************** Autotuning format combination: Half(4,448,1:8,50176) -> Float(1,112,12544,401408) *************** [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:52] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CaskConvolution) [07/19/2021-12:42:52] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaConvolution) [07/19/2021-12:42:52] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:52] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:52] [V] [TRT] *************** Autotuning format combination: Half(4,448,1:8,50176) -> Half(4,448,1:8,50176) *************** [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:52] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CaskConvolution) [07/19/2021-12:42:52] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaConvolution) [07/19/2021-12:42:52] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:52] [V] [TRT] Tactic: -2 time 0.061124 [07/19/2021-12:42:52] [V] [TRT] Fastest Tactic: -2 Time: 0.061124 [07/19/2021-12:42:52] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaDepthwiseConvolution Tactic: -2 [07/19/2021-12:42:52] [V] [TRT] [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:52] [V] [TRT] Tactic: 1002 time 0.222564 [07/19/2021-12:42:52] [V] [TRT] Tactic: 0 time 0.055912 [07/19/2021-12:42:52] [V] [TRT] Fastest Tactic: 0 Time: 0.055912 [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:52] [V] [TRT] Tactic: 1002 time 0.225416 [07/19/2021-12:42:52] [V] [TRT] Tactic: 0 time 0.052372 [07/19/2021-12:42:52] [V] [TRT] Fastest Tactic: 0 Time: 0.052372 [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:52] [V] [TRT] Tactic: 1002 time 0.21872 [07/19/2021-12:42:52] [V] [TRT] Tactic: 0 time 0.034524 [07/19/2021-12:42:52] [V] [TRT] Fastest Tactic: 0 Time: 0.034524 [07/19/2021-12:42:52] [V] [TRT] *************** Autotuning format combination: Float(1,112,12544,401408) -> Float(1,112,12544,200704) *************** [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1 [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1 [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:52] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:52] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:52] [V] [TRT] Tactic: 589823 time 0.267144 [07/19/2021-12:42:52] [V] [TRT] Tactic: 655359 time 0.28428 [07/19/2021-12:42:52] [V] [TRT] Tactic: 786431 time 0.228848 [07/19/2021-12:42:52] [V] [TRT] Tactic: 851967 time 0.4675 [07/19/2021-12:42:52] [V] [TRT] Tactic: 1179647 time 0.251808 [07/19/2021-12:42:52] [V] [TRT] Tactic: 1310719 time 0.343656 [07/19/2021-12:42:52] [V] [TRT] Tactic: 1376255 time 0.257732 [07/19/2021-12:42:52] [V] [TRT] Tactic: 1441791 time 0.447976 [07/19/2021-12:42:52] [V] [TRT] Tactic: 1507327 time 0.354796 [07/19/2021-12:42:52] [V] [TRT] Tactic: 1638399 time 0.21778 [07/19/2021-12:42:52] [V] [TRT] Tactic: 1835007 time 0.231464 [07/19/2021-12:42:52] [V] [TRT] Tactic: 1900543 time 0.349688 [07/19/2021-12:42:52] [V] [TRT] Tactic: 2097151 time 0.234632 [07/19/2021-12:42:52] [V] [TRT] Tactic: 2162687 time 0.255932 [07/19/2021-12:42:52] [V] [TRT] Tactic: 2293759 time 0.254412 [07/19/2021-12:42:52] [V] [TRT] Tactic: 2359295 time 0.205132 [07/19/2021-12:42:52] [V] [TRT] Tactic: 2686975 time 0.244068 [07/19/2021-12:42:52] [V] [TRT] Tactic: 3080191 time 0.32106 [07/19/2021-12:42:52] [V] [TRT] Tactic: 3342335 time 0.345044 [07/19/2021-12:42:52] [V] [TRT] Tactic: 3407871 time 0.227552 [07/19/2021-12:42:52] [V] [TRT] Tactic: 3538943 time 0.221896 [07/19/2021-12:42:52] [V] [TRT] Tactic: 3670015 time 0.324648 [07/19/2021-12:42:52] [V] [TRT] Tactic: 3932159 time 0.291068 [07/19/2021-12:42:52] [V] [TRT] Tactic: 3997695 time 0.223872 [07/19/2021-12:42:52] [V] [TRT] Tactic: 4063231 time 0.454076 [07/19/2021-12:42:52] [V] [TRT] Tactic: 4194303 time 0.215496 [07/19/2021-12:42:52] [V] [TRT] Tactic: 4259839 time 0.214352 [07/19/2021-12:42:52] [V] [TRT] Tactic: 4325375 time 0.203504 [07/19/2021-12:42:53] [V] [TRT] Tactic: 4521983 time 0.242276 [07/19/2021-12:42:53] [V] [TRT] Tactic: 4587519 time 0.208816 [07/19/2021-12:42:53] [V] [TRT] Tactic: 4653055 time 0.233604 [07/19/2021-12:42:53] [V] [TRT] Tactic: 4915199 time 0.211792 [07/19/2021-12:42:53] [V] [TRT] Tactic: 4980735 time 0.21526 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5177343 time 0.27272 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5242879 time 0.227848 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5373951 time 0.32824 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5439487 time 0.2847 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5570559 time 0.337684 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5636095 time 0.45372 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5701631 time 0.258852 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5767167 time 0.539436 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5832703 time 0.21674 [07/19/2021-12:42:53] [V] [TRT] Tactic: 5898239 time 0.27418 [07/19/2021-12:42:53] [V] [TRT] Tactic: 6029311 time 0.25016 [07/19/2021-12:42:53] [V] [TRT] Tactic: 6225919 time 0.226976 [07/19/2021-12:42:53] [V] [TRT] Tactic: 6291455 time 0.251604 [07/19/2021-12:42:53] [V] [TRT] Tactic: 6422527 time 0.294812 [07/19/2021-12:42:53] [V] [TRT] Tactic: 6750207 time 0.240808 [07/19/2021-12:42:53] [V] [TRT] Tactic: 6815743 time 0.270896 [07/19/2021-12:42:53] [V] [TRT] Tactic: 6946815 time 0.279224 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7012351 time 0.23414 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7077887 time 0.20362 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7143423 time 0.274992 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7208959 time 0.230856 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7340031 time 0.272176 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7405567 time 0.235492 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7536639 time 0.301912 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7602175 time 0.215816 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7733247 time 0.198616 [07/19/2021-12:42:53] [V] [TRT] Tactic: 7798783 time 0.231556 [07/19/2021-12:42:53] [V] [TRT] Tactic: 8191999 time 0.250556 [07/19/2021-12:42:53] [V] [TRT] Tactic: 8257535 time 0.212264 [07/19/2021-12:42:53] [V] [TRT] Tactic: 8323071 time 0.233092 [07/19/2021-12:42:53] [V] [TRT] Tactic: 8650751 time 0.240616 [07/19/2021-12:42:53] [V] [TRT] Tactic: 8716287 time 0.30202 [07/19/2021-12:42:53] [V] [TRT] Tactic: 9109503 time 0.24126 [07/19/2021-12:42:53] [V] [TRT] Tactic: 9568255 time 0.21152 [07/19/2021-12:42:53] [V] [TRT] Tactic: 9895935 time 0.216212 [07/19/2021-12:42:53] [V] [TRT] Tactic: 10223615 time 0.244556 [07/19/2021-12:42:53] [V] [TRT] Tactic: 10354687 time 0.200056 [07/19/2021-12:42:53] [V] [TRT] Tactic: 10551295 time 0.241736 [07/19/2021-12:42:53] [V] [TRT] Tactic: 10747903 time 0.205292 [07/19/2021-12:42:54] [V] [TRT] Tactic: 10944511 time 0.215672 [07/19/2021-12:42:54] [V] [TRT] Fastest Tactic: 7733247 Time: 0.198616 [07/19/2021-12:42:54] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1754569683116234317 time 0.236884 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1825138533642645384 time 0.239484 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: 2733356012094739613 time 0.083024 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: 3915320020053085238 time 0.237036 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: 6808617066150061604 time 0.124932 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: 9091006216302412844 time 0.122672 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: -8060443123034038864 time 0.126552 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: -4420849921117327522 time 0.097328 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] Tactic: -3946921629105938337 time 0.084812 [07/19/2021-12:42:54] [V] [TRT] Fastest Tactic: 2733356012094739613 Time: 0.083024 [07/19/2021-12:42:54] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:54] [V] [TRT] Tactic: 0 time 0.125676 [07/19/2021-12:42:54] [V] [TRT] Tactic: 2 time 0.226056 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5 time 0.348992 [07/19/2021-12:42:54] [V] [TRT] Tactic: 57 time 0.12656 [07/19/2021-12:42:54] [V] [TRT] Fastest Tactic: 0 Time: 0.125676 [07/19/2021-12:42:54] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:54] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:54] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2733356012094739613 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] *************** Autotuning format combination: Half(1,112,12544,401408) -> Half(1,112,12544,200704) *************** [07/19/2021-12:42:54] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:54] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:54] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:54] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:54] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:54] [V] [TRT] Tactic: 0 time 0.134188 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1 time 0.134088 [07/19/2021-12:42:54] [V] [TRT] Tactic: 2 time 0.207044 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4 skipped. Scratch requested: 72419328, available: 67108864 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5 time 0.347768 [07/19/2021-12:42:54] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [07/19/2021-12:42:54] [V] [TRT] Fastest Tactic: 1 Time: 0.134088 [07/19/2021-12:42:54] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:54] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:54] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaConvolution Tactic: 1 [07/19/2021-12:42:54] [V] [TRT] [07/19/2021-12:42:54] [V] [TRT] *************** Autotuning format combination: Half(1,112,12544:2,200704) -> Half(1,112,12544:2,100352) *************** [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1 [07/19/2021-12:42:54] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:54] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:54] [V] [TRT] Tactic: 589823 time 0.13954 [07/19/2021-12:42:54] [V] [TRT] Tactic: 655359 time 0.177136 [07/19/2021-12:42:54] [V] [TRT] Tactic: 786431 time 0.132204 [07/19/2021-12:42:54] [V] [TRT] Tactic: 851967 time 0.369556 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1179647 time 0.17988 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1310719 time 0.2432 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1376255 time 0.12356 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1441791 time 0.330392 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1507327 time 0.24668 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1638399 time 0.133276 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1835007 time 0.127812 [07/19/2021-12:42:54] [V] [TRT] Tactic: 1900543 time 0.216148 [07/19/2021-12:42:54] [V] [TRT] Tactic: 2162687 time 0.130692 [07/19/2021-12:42:54] [V] [TRT] Tactic: 2293759 time 0.131828 [07/19/2021-12:42:54] [V] [TRT] Tactic: 2359295 time 0.117912 [07/19/2021-12:42:54] [V] [TRT] Tactic: 2686975 time 0.130036 [07/19/2021-12:42:54] [V] [TRT] Tactic: 3080191 time 0.219632 [07/19/2021-12:42:54] [V] [TRT] Tactic: 3342335 time 0.23912 [07/19/2021-12:42:54] [V] [TRT] Tactic: 3407871 time 0.120148 [07/19/2021-12:42:54] [V] [TRT] Tactic: 3538943 time 0.135944 [07/19/2021-12:42:54] [V] [TRT] Tactic: 3670015 time 0.176816 [07/19/2021-12:42:54] [V] [TRT] Tactic: 3932159 time 0.176408 [07/19/2021-12:42:54] [V] [TRT] Tactic: 3997695 time 0.137136 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4063231 time 0.36496 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4194303 time 0.119664 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4325375 time 0.121208 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4521983 time 0.12688 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4587519 time 0.126928 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4653055 time 0.22454 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4915199 time 0.119388 [07/19/2021-12:42:54] [V] [TRT] Tactic: 4980735 time 0.121196 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5177343 time 0.18322 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5242879 time 0.115468 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5373951 time 0.20304 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5439487 time 0.148944 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5570559 time 0.231468 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5636095 time 0.365088 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5701631 time 0.140552 [07/19/2021-12:42:54] [V] [TRT] Tactic: 5767167 time 0.26094 [07/19/2021-12:42:55] [V] [TRT] Tactic: 5832703 time 0.113296 [07/19/2021-12:42:55] [V] [TRT] Tactic: 5898239 time 0.206364 [07/19/2021-12:42:55] [V] [TRT] Tactic: 6029311 time 0.132216 [07/19/2021-12:42:55] [V] [TRT] Tactic: 6225919 time 0.127648 [07/19/2021-12:42:55] [V] [TRT] Tactic: 6291455 time 0.179852 [07/19/2021-12:42:55] [V] [TRT] Tactic: 6422527 time 0.201752 [07/19/2021-12:42:55] [V] [TRT] Tactic: 6750207 time 0.13216 [07/19/2021-12:42:55] [V] [TRT] Tactic: 6815743 time 0.138608 [07/19/2021-12:42:55] [V] [TRT] Tactic: 6946815 time 0.15044 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7077887 time 0.124784 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7143423 time 0.139532 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7208959 time 0.118872 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7340031 time 0.207184 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7405567 time 0.175768 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7536639 time 0.16052 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7602175 time 0.125536 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7733247 time 0.130968 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7798783 time 0.131984 [07/19/2021-12:42:55] [V] [TRT] Tactic: 8191999 time 0.132824 [07/19/2021-12:42:55] [V] [TRT] Tactic: 8257535 time 0.12892 [07/19/2021-12:42:55] [V] [TRT] Tactic: 8323071 time 0.125408 [07/19/2021-12:42:55] [V] [TRT] Tactic: 8650751 time 0.133476 [07/19/2021-12:42:55] [V] [TRT] Tactic: 8716287 time 0.156296 [07/19/2021-12:42:55] [V] [TRT] Tactic: 9568255 time 0.11984 [07/19/2021-12:42:55] [V] [TRT] Tactic: 9895935 time 0.120224 [07/19/2021-12:42:55] [V] [TRT] Tactic: 10223615 time 0.129164 [07/19/2021-12:42:55] [V] [TRT] Tactic: 10354687 time 0.128556 [07/19/2021-12:42:55] [V] [TRT] Tactic: 10551295 time 0.127904 [07/19/2021-12:42:55] [V] [TRT] Tactic: 10747903 time 0.1318 [07/19/2021-12:42:55] [V] [TRT] Tactic: 10944511 time 0.12098 [07/19/2021-12:42:55] [V] [TRT] Fastest Tactic: 5832703 Time: 0.113296 [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 2418518597804310654 time 0.084672 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 8292881859266835088 time 0.06284 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 8401509141903434922 time 0.083868 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -8654297089785671176 time 0.210556 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -7140760933967189247 time 0.087244 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -4097850214384059472 time 0.053848 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -3689982367035295496 time 0.210984 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -2534402059426524406 time 0.145588 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -2027588946874785071 time 0.062388 [07/19/2021-12:42:55] [V] [TRT] Fastest Tactic: -4097850214384059472 Time: 0.053848 [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:55] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:55] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:55] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -4097850214384059472 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:55] [V] [TRT] [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:55] [V] [TRT] *************** Autotuning format combination: Half(4,448,1:8,50176) -> Float(1,112,12544,200704) *************** [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:55] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:55] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:55] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:55] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:55] [V] [TRT] *************** Autotuning format combination: Half(4,448,1:8,50176) -> Half(2,224,1:8,25088) *************** [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:55] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 2105695814191699972 time 0.059064 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 3754069740140581927 time 0.089416 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 5666160310350604399 time 0.08926 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 5925270497649423688 time 0.103644 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 6680916730816870145 time 0.108224 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7158029511300006471 time 0.059888 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 7859952145590271433 time 0.106916 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: 8283847742354150423 time 0.09002 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -8391760416076885205 time 0.101316 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -4534876761957424274 time 0.102668 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -3237051169894153788 time 0.059972 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Tactic: -2422160065350346448 time 0.105948 [07/19/2021-12:42:55] [V] [TRT] Fastest Tactic: 2105695814191699972 Time: 0.059064 [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:55] [V] [TRT] Tactic: 0 time 0.143336 [07/19/2021-12:42:55] [V] [TRT] Tactic: 1 time 0.143504 [07/19/2021-12:42:55] [V] [TRT] Tactic: 2 time 0.857004 [07/19/2021-12:42:55] [V] [TRT] Fastest Tactic: 0 Time: 0.143336 [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:55] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:55] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2105695814191699972 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:55] [V] [TRT] Tactic: 1002 time 0.053564 [07/19/2021-12:42:55] [V] [TRT] Tactic: 0 time 0.065228 [07/19/2021-12:42:55] [V] [TRT] Fastest Tactic: 1002 Time: 0.053564 [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:55] [V] [TRT] Tactic: 1002 time 0.121152 [07/19/2021-12:42:55] [V] [TRT] Tactic: 0 time 0.033196 [07/19/2021-12:42:55] [V] [TRT] Fastest Tactic: 0 Time: 0.033196 [07/19/2021-12:42:55] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:55] [V] [TRT] Tactic: 1002 time 0.181104 [07/19/2021-12:42:55] [V] [TRT] Tactic: 0 time 0.027264 [07/19/2021-12:42:55] [V] [TRT] Fastest Tactic: 0 Time: 0.027264 [07/19/2021-12:42:55] [V] [TRT] Adding reformat layer: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] reformatted input 0 (21) from Float(1,112,12544,401408) to Half(4,448,1:8,50176) [07/19/2021-12:42:55] [V] [TRT] Adding reformat layer: (Unnamed Layer* 6) [Convolution] output to be reformatted 0 (output) from Float(1,112,12544,200704) to Half(2,224,1:8,25088) [07/19/2021-12:42:55] [V] [TRT] For layer (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] a non-conforming implementation was chosen than was requested i.e. requested layer computation precision and output precision types were ignored because it resulted in faster network performance. Enable strict mode to try force choose a conforming implementation. [07/19/2021-12:42:55] [V] [TRT] Formats and tactics selection completed in 7.50308 seconds. [07/19/2021-12:42:55] [V] [TRT] After reformat layers: 5 layers [07/19/2021-12:42:55] [V] [TRT] Block size 67108864 [07/19/2021-12:42:55] [V] [TRT] Block size 1605632 [07/19/2021-12:42:55] [V] [TRT] Block size 802816 [07/19/2021-12:42:55] [V] [TRT] Total Activation Memory: 69517312 [07/19/2021-12:42:55] [I] [TRT] Detected 1 inputs and 1 output network tensors. [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:55] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:55] [V] [TRT] Layer: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] Weights: 0 HostPersistent: 1664 DevicePersistent: 77824 [07/19/2021-12:42:55] [V] [TRT] Layer: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] input reformatter 0 Weights: 0 HostPersistent: 0 DevicePersistent: 0 [07/19/2021-12:42:55] [V] [TRT] Layer: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] Weights: 576 HostPersistent: 0 DevicePersistent: 0 [07/19/2021-12:42:55] [V] [TRT] Layer: (Unnamed Layer* 6) [Convolution] Weights: 0 HostPersistent: 3200 DevicePersistent: 76800 [07/19/2021-12:42:55] [V] [TRT] Layer: (Unnamed Layer* 6) [Convolution] output reformatter 0 Weights: 0 HostPersistent: 0 DevicePersistent: 0 [07/19/2021-12:42:55] [V] [TRT] Total Host Persistent Memory: 4864 [07/19/2021-12:42:55] [V] [TRT] Total Device Persistent Memory: 154624 [07/19/2021-12:42:55] [V] [TRT] Total Weight Memory: 576 [07/19/2021-12:42:55] [V] [TRT] Builder timing cache: created 31 entries, 9 hit(s) [07/19/2021-12:42:55] [V] [TRT] Engine generation completed in 8.88287 seconds. [07/19/2021-12:42:55] [V] [TRT] Engine Layer Information: [07/19/2021-12:42:55] [V] [TRT] Layer(scudnn): (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation], Tactic: -3946921629105938337, input.1[Float(2,224,224)] -> 21[Float(32,112,112)] [07/19/2021-12:42:55] [V] [TRT] Layer(Reformat): (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] input reformatter 0, Tactic: 0, 21[Float(32,112,112)] -> (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] reformatted input 0[Half(32,112,112)] [07/19/2021-12:42:55] [V] [TRT] Layer(DepthwiseConvolution): (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation], Tactic: -2, (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] reformatted input 0[Half(32,112,112)] -> 24[Half(32,112,112)] [07/19/2021-12:42:55] [V] [TRT] Layer(h884cudnn): (Unnamed Layer* 6) [Convolution], Tactic: 2105695814191699972, 24[Half(32,112,112)] -> (Unnamed Layer* 6) [Convolution] output to be reformatted 0[Half(16,112,112)] [07/19/2021-12:42:55] [V] [TRT] Layer(Reformat): (Unnamed Layer* 6) [Convolution] output reformatter 0, Tactic: 0, (Unnamed Layer* 6) [Convolution] output to be reformatted 0[Half(16,112,112)] -> output[Float(16,112,112)] [07/19/2021-12:42:55] [I] Starting inference threads [07/19/2021-12:42:55] [I] Warmup completed 0 queries over 0 ms [07/19/2021-12:42:55] [I] Timing trace has 1 queries over 0.000925408 s [07/19/2021-12:42:55] [I] Trace averages of 10 runs: [07/19/2021-12:42:55] [I] Host Latency [07/19/2021-12:42:55] [I] min: 0.9008 ms (end to end 0.925408 ms) [07/19/2021-12:42:55] [I] max: 0.9008 ms (end to end 0.925408 ms) [07/19/2021-12:42:55] [I] mean: 0.9008 ms (end to end 0.925408 ms) [07/19/2021-12:42:55] [I] median: 0.9008 ms (end to end 0.925408 ms) [07/19/2021-12:42:55] [I] percentile: 0.9008 ms at 99% (end to end 0.925408 ms at 99%) [07/19/2021-12:42:55] [I] throughput: 1080.6 qps [07/19/2021-12:42:55] [I] walltime: 0.000925408 s [07/19/2021-12:42:55] [I] Enqueue Time [07/19/2021-12:42:55] [I] min: 0.693152 ms [07/19/2021-12:42:55] [I] max: 0.693152 ms [07/19/2021-12:42:55] [I] median: 0.693152 ms [07/19/2021-12:42:55] [I] GPU Compute [07/19/2021-12:42:55] [I] min: 0.70656 ms [07/19/2021-12:42:55] [I] max: 0.70656 ms [07/19/2021-12:42:55] [I] mean: 0.70656 ms [07/19/2021-12:42:55] [I] median: 0.70656 ms [07/19/2021-12:42:55] [I] percentile: 0.70656 ms at 99% [07/19/2021-12:42:55] [I] total compute time: 0.00070656 s [07/19/2021-12:42:55] [I] [07/19/2021-12:42:55] [I] === Profile (1 iterations ) === [07/19/2021-12:42:55] [I] Layer Time (ms) Avg. Time (ms) Time % [07/19/2021-12:42:55] [I] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] 0.36 0.36 62.2 [07/19/2021-12:42:55] [I] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] input reformatter 0 0.05 0.05 9.1 [07/19/2021-12:42:55] [I] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] 0.07 0.07 11.5 [07/19/2021-12:42:55] [I] (Unnamed Layer* 6) [Convolution] 0.07 0.07 12.2 [07/19/2021-12:42:55] [I] (Unnamed Layer* 6) [Convolution] output reformatter 0 0.03 0.03 5.0 [07/19/2021-12:42:55] [I] Total 0.57 0.57 100.0 [07/19/2021-12:42:55] [I] &&&& PASSED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --fp16 --workspace=64 --iterations=1 --warmUp=0 --duration=0 --verbose --dumpProfile --onnx=/home/shield/data/dla_correctness.models/release_prefix/model1.onnx --exportOutput=/tmp/tmpoa1umvwa.json