RUNNING TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --fp16 --workspace=64 --iterations=1 --warmUp=0 --duration=0 --verbose --dumpProfile --onnx=/home/shield/data/dla_correctness.models/release_prefix/model1.onnx --exportOutput=/tmp/tmpqnvl4hzz.json [07/19/2021-12:42:33] [I] === Model Options === [07/19/2021-12:42:33] [I] Format: ONNX [07/19/2021-12:42:33] [I] Model: /home/shield/data/dla_correctness.models/release_prefix/model1.onnx [07/19/2021-12:42:33] [I] Output: [07/19/2021-12:42:33] [I] === Build Options === [07/19/2021-12:42:33] [I] Max batch: 1 [07/19/2021-12:42:33] [I] Workspace: 64 MB [07/19/2021-12:42:33] [I] minTiming: 1 [07/19/2021-12:42:33] [I] avgTiming: 8 [07/19/2021-12:42:33] [I] Precision: FP32+FP16 [07/19/2021-12:42:33] [I] Calibration: [07/19/2021-12:42:33] [I] Safe mode: Disabled [07/19/2021-12:42:33] [I] Save engine: [07/19/2021-12:42:33] [I] Load engine: [07/19/2021-12:42:33] [I] Builder Cache: Enabled [07/19/2021-12:42:33] [I] NVTX verbosity: 0 [07/19/2021-12:42:33] [I] Inputs format: fp32:CHW [07/19/2021-12:42:33] [I] Outputs format: fp32:CHW [07/19/2021-12:42:33] [I] Input build shapes: model [07/19/2021-12:42:33] [I] Input calibration shapes: model [07/19/2021-12:42:33] [I] === System Options === [07/19/2021-12:42:33] [I] Device: 0 [07/19/2021-12:42:33] [I] DLACore: [07/19/2021-12:42:33] [I] Plugins: [07/19/2021-12:42:33] [I] === Inference Options === [07/19/2021-12:42:33] [I] Batch: 1 [07/19/2021-12:42:33] [I] Input inference shapes: model [07/19/2021-12:42:33] [I] Iterations: 1 [07/19/2021-12:42:33] [I] Duration: 0s (+ 0ms warm up) [07/19/2021-12:42:33] [I] Sleep time: 0ms [07/19/2021-12:42:33] [I] Streams: 1 [07/19/2021-12:42:33] [I] ExposeDMA: Disabled [07/19/2021-12:42:33] [I] Spin-wait: Disabled [07/19/2021-12:42:33] [I] Multithreading: Disabled [07/19/2021-12:42:33] [I] CUDA Graph: Disabled [07/19/2021-12:42:33] [I] Skip inference: Disabled [07/19/2021-12:42:33] [I] Inputs: [07/19/2021-12:42:33] [I] === Reporting Options === [07/19/2021-12:42:33] [I] Verbose: Enabled [07/19/2021-12:42:33] [I] Averages: 10 inferences [07/19/2021-12:42:33] [I] Percentile: 99 [07/19/2021-12:42:33] [I] Dump output: Disabled [07/19/2021-12:42:33] [I] Profile: Enabled [07/19/2021-12:42:33] [I] Export timing to JSON file: [07/19/2021-12:42:33] [I] Export output to JSON file: /tmp/tmpqnvl4hzz.json [07/19/2021-12:42:33] [I] Export profile to JSON file: [07/19/2021-12:42:33] [I] [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::Region_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::CropAndResize version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::Proposal version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::Split version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1 [07/19/2021-12:42:33] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1 ---------------------------------------------------------------- Input filename: /home/shield/data/dla_correctness.models/release_prefix/model1.onnx ONNX IR version: 0.0.4 Opset version: 7 Producer name: pytorch Producer version: 1.3 Domain: Model version: 0 Doc string: ---------------------------------------------------------------- [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::Proposal version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::Split version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:202: Adding network input: input.1 with dtype: float32, dimensions: (1, 2, 224, 224) [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: input.1 for ONNX tensor: input.1 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.0.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.bias [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.num_batches_tracked [07/19/2021-12:42:34] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.running_mean [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.running_var [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 0.1.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.0.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.bias [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.num_batches_tracked [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.running_mean [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.running_var [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.0.1.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.1.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.bias [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.num_batches_tracked [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.running_mean [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.running_var [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 1.conv.2.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Conv] [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: input.1 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.0.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:125: [Conv] inputs: [input.1 -> (1, 2, 224, 224)], [0.0.weight -> (32, 2, 3, 3)], [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 2, 224, 224) [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 0) [Convolution] for ONNX node: [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (2, 2), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 32 [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 32, 112, 112) [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 19 for ONNX tensor: 19 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:179: [Conv] outputs: [19 -> (1, 32, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:103: Parsing node: [BatchNormalization] [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 19 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.1.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.1.bias [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.1.running_mean [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 0.1.running_var [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:125: [BatchNormalization] inputs: [19 -> (1, 32, 112, 112)], [0.1.weight -> (32)], [0.1.bias -> (32)], [0.1.running_mean -> (32)], [0.1.running_var -> (32)], [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 1) [Scale] for ONNX node: [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 20 for ONNX tensor: 20 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:179: [BatchNormalization] outputs: [20 -> (1, 32, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Clip] [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 20 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:125: [Clip] inputs: [20 -> (1, 32, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 2) [Activation] for ONNX node: [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 21 for ONNX tensor: 21 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:179: [Clip] outputs: [21 -> (1, 32, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Conv] [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 21 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.0.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:125: [Conv] inputs: [21 -> (1, 32, 112, 112)], [1.conv.0.0.weight -> (32, 1, 3, 3)], [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 32, 112, 112) [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 3) [Convolution] for ONNX node: [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 32 [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 32, 112, 112) [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 22 for ONNX tensor: 22 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:179: [Conv] outputs: [22 -> (1, 32, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:103: Parsing node: [BatchNormalization] [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 22 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.1.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.1.bias [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.1.running_mean [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.0.1.running_var [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:125: [BatchNormalization] inputs: [22 -> (1, 32, 112, 112)], [1.conv.0.1.weight -> (32)], [1.conv.0.1.bias -> (32)], [1.conv.0.1.running_mean -> (32)], [1.conv.0.1.running_var -> (32)], [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 4) [Scale] for ONNX node: [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 23 for ONNX tensor: 23 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:179: [BatchNormalization] outputs: [23 -> (1, 32, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Clip] [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 23 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:125: [Clip] inputs: [23 -> (1, 32, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 5) [Activation] for ONNX node: [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 24 for ONNX tensor: 24 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:179: [Clip] outputs: [24 -> (1, 32, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Conv] [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 24 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.1.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:125: [Conv] inputs: [24 -> (1, 32, 112, 112)], [1.conv.1.weight -> (16, 32, 1, 1)], [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 32, 112, 112) [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 6) [Convolution] for ONNX node: [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (1, 1), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 16 [07/19/2021-12:42:34] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 16, 112, 112) [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 25 for ONNX tensor: 25 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:179: [Conv] outputs: [25 -> (1, 16, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:103: Parsing node: [BatchNormalization] [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 25 [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.2.weight [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.2.bias [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.2.running_mean [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:119: Searching for input: 1.conv.2.running_var [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:125: [BatchNormalization] inputs: [25 -> (1, 16, 112, 112)], [1.conv.2.weight -> (16)], [1.conv.2.bias -> (16)], [1.conv.2.running_mean -> (16)], [1.conv.2.running_var -> (16)], [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 7) [Scale] for ONNX node: [07/19/2021-12:42:34] [V] [TRT] ImporterContext.hpp:116: Registering tensor: output_1 for ONNX tensor: output [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:179: [BatchNormalization] outputs: [output -> (1, 16, 112, 112)], [07/19/2021-12:42:34] [V] [TRT] ModelImporter.cpp:507: Marking output_1 as output: output ----- Parsing of ONNX model /home/shield/data/dla_correctness.models/release_prefix/model1.onnx is Done ---- [07/19/2021-12:42:34] [V] [TRT] Applying generic optimizations to the graph for inference. [07/19/2021-12:42:34] [V] [TRT] Original: 8 layers [07/19/2021-12:42:34] [V] [TRT] After dead-layer removal: 8 layers [07/19/2021-12:42:34] [V] [TRT] After Myelin optimization: 8 layers [07/19/2021-12:42:34] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 0) [Convolution] with scale (Unnamed Layer* 1) [Scale] [07/19/2021-12:42:34] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 3) [Convolution] with scale (Unnamed Layer* 4) [Scale] [07/19/2021-12:42:34] [V] [TRT] Fusing convolution weights from (Unnamed Layer* 6) [Convolution] with scale (Unnamed Layer* 7) [Scale] [07/19/2021-12:42:34] [V] [TRT] After scale fusion: 5 layers [07/19/2021-12:42:34] [V] [TRT] Fusing (Unnamed Layer* 0) [Convolution] with (Unnamed Layer* 2) [Activation] [07/19/2021-12:42:34] [V] [TRT] Fusing (Unnamed Layer* 3) [Convolution] with (Unnamed Layer* 5) [Activation] [07/19/2021-12:42:34] [V] [TRT] After vertical fusions: 3 layers [07/19/2021-12:42:34] [V] [TRT] After final dead-layer removal: 3 layers [07/19/2021-12:42:34] [V] [TRT] After tensor merging: 3 layers [07/19/2021-12:42:34] [V] [TRT] After concat removal: 3 layers [07/19/2021-12:42:34] [V] [TRT] Graph construction and optimization completed in 0.00333536 seconds. [07/19/2021-12:42:34] [I] [TRT] [07/19/2021-12:42:34] [I] [TRT] --------------- Layers running on DLA: [07/19/2021-12:42:34] [I] [TRT] [07/19/2021-12:42:34] [I] [TRT] --------------- Layers running on GPU: [07/19/2021-12:42:34] [I] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation], (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation], (Unnamed Layer* 6) [Convolution], [07/19/2021-12:42:36] [V] [TRT] Constructing optimization profile number 0 [1/1]. [07/19/2021-12:42:36] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:36] [V] [TRT] Tactic: 1002 time 0.052224 [07/19/2021-12:42:36] [V] [TRT] Tactic: 0 time 0.084364 [07/19/2021-12:42:36] [V] [TRT] Fastest Tactic: 1002 Time: 0.052224 [07/19/2021-12:42:36] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:36] [V] [TRT] Tactic: 1002 time 0.891012 [07/19/2021-12:42:36] [V] [TRT] Tactic: 0 time 0.039412 [07/19/2021-12:42:36] [V] [TRT] Fastest Tactic: 0 Time: 0.039412 [07/19/2021-12:42:36] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:36] [V] [TRT] Tactic: 1002 time 1.11349 [07/19/2021-12:42:36] [V] [TRT] Tactic: 0 time 0.083224 [07/19/2021-12:42:36] [V] [TRT] Fastest Tactic: 0 Time: 0.083224 [07/19/2021-12:42:36] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:36] [V] [TRT] Tactic: 1002 time 1.11693 [07/19/2021-12:42:36] [V] [TRT] Tactic: 0 time 0.084228 [07/19/2021-12:42:36] [V] [TRT] Fastest Tactic: 0 Time: 0.084228 [07/19/2021-12:42:36] [V] [TRT] *************** Autotuning format combination: Float(1,224,50176,100352) -> Float(1,112,12544,401408) *************** [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:36] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:36] [V] [TRT] Tactic: 458751 time 0.999536 [07/19/2021-12:42:36] [V] [TRT] Fastest Tactic: 458751 Time: 0.999536 [07/19/2021-12:42:36] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:36] [V] [TRT] Tactic: 1825138533642645384 time 0.437508 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:36] [V] [TRT] Tactic: 2842488832350522458 time 0.266496 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:36] [V] [TRT] Tactic: 3915320020053085238 time 0.432652 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:36] [V] [TRT] Tactic: 6448355332020552203 time 0.635156 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:36] [V] [TRT] Tactic: 6808617066150061604 time 0.228628 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:36] [V] [TRT] Tactic: -8060443123034038864 time 0.23284 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:36] [V] [TRT] Tactic: -4420849921117327522 time 0.168068 [07/19/2021-12:42:36] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:36] [V] [TRT] Tactic: -3946921629105938337 time 0.144516 [07/19/2021-12:42:36] [V] [TRT] Fastest Tactic: -3946921629105938337 Time: 0.144516 [07/19/2021-12:42:36] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:37] [V] [TRT] Tactic: 0 time 0.515204 [07/19/2021-12:42:37] [V] [TRT] Tactic: 2 time 1.06662 [07/19/2021-12:42:37] [V] [TRT] Tactic: 5 time 12.9928 [07/19/2021-12:42:37] [V] [TRT] Tactic: 57 time 0.519824 [07/19/2021-12:42:37] [V] [TRT] Fastest Tactic: 0 Time: 0.515204 [07/19/2021-12:42:37] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:37] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:37] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -3946921629105938337 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:37] [V] [TRT] [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:37] [V] [TRT] *************** Autotuning format combination: Half(1,224,50176,100352) -> Half(1,112,12544,401408) *************** [07/19/2021-12:42:37] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:37] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:37] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:37] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:37] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:37] [V] [TRT] Tactic: 0 time 0.407948 [07/19/2021-12:42:37] [V] [TRT] Tactic: 1 time 0.41754 [07/19/2021-12:42:37] [V] [TRT] Tactic: 2 time 0.818284 [07/19/2021-12:42:37] [V] [TRT] Tactic: 5 time 9.85453 [07/19/2021-12:42:37] [V] [TRT] Fastest Tactic: 0 Time: 0.407948 [07/19/2021-12:42:37] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:37] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:37] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaConvolution Tactic: 0 [07/19/2021-12:42:37] [V] [TRT] [07/19/2021-12:42:37] [V] [TRT] *************** Autotuning format combination: Half(1,224,50176:2,50176) -> Half(1,112,12544:2,200704) *************** [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1 [07/19/2021-12:42:37] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:37] [V] [TRT] Tactic: 458751 time 0.198784 [07/19/2021-12:42:37] [V] [TRT] Fastest Tactic: 458751 Time: 0.198784 [07/19/2021-12:42:37] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1 [07/19/2021-12:42:37] [V] [TRT] Tactic: 1145226902788474763 time 0.08682 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:37] [V] [TRT] Tactic: 2418518597804310654 time 0.086564 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:37] [V] [TRT] Tactic: 8292881859266835088 time 0.064208 [07/19/2021-12:42:37] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 8401509141903434922 time 0.08434 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: -8654297089785671176 time 0.212716 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: -7448936905981214224 time 0.084368 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: -3689982367035295496 time 0.276584 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: -2027588946874785071 time 0.081504 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: -245090590808296743 time 0.27442 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 8292881859266835088 Time: 0.064208 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:38] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:38] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 8292881859266835088 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] *************** Autotuning format combination: Half(1,224,1:4,50176) -> Half(4,448,1:8,50176) *************** [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:38] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:38] [V] [TRT] Tactic: -1886198213384953148 time 0.099236 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: -1886198213384953148 Time: 0.099236 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:38] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:38] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -1886198213384953148 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:38] [V] [TRT] [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] () Set Tactic Name: volta_first_layer_filter3x3_fwd [07/19/2021-12:42:38] [V] [TRT] *************** Autotuning format combination: Half(1,224,1:8,50176) -> Float(1,112,12544,401408) *************** [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:38] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:38] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:38] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:38] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] *************** Autotuning format combination: Half(1,224,1:8,50176) -> Half(4,448,1:8,50176) *************** [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:38] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CaskConvolution) [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 3754069740140581927 time 0.341372 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 5925270497649423688 time 0.369892 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 6680916730816870145 time 0.414388 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 7158029511300006471 time 0.203124 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 7859952145590271433 time 0.404704 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 8283847742354150423 time 0.34866 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: -4534876761957424274 time 0.362788 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: -3237051169894153788 time 0.1997 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: -3237051169894153788 Time: 0.1997 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaConvolution) [07/19/2021-12:42:38] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:38] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -3237051169894153788 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.11862 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.17278 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 1002 Time: 0.11862 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.181228 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.062492 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 0 Time: 0.062492 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.124552 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.155464 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 1002 Time: 0.124552 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.181292 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.056708 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 0 Time: 0.056708 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.303992 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.074788 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 0 Time: 0.074788 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.303848 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.064224 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 0 Time: 0.064224 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.17926 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.05154 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 0 Time: 0.05154 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.178024 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.061516 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 0 Time: 0.061516 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:38] [V] [TRT] Tactic: 1002 time 0.179052 [07/19/2021-12:42:38] [V] [TRT] Tactic: 0 time 0.046356 [07/19/2021-12:42:38] [V] [TRT] Fastest Tactic: 0 Time: 0.046356 [07/19/2021-12:42:38] [V] [TRT] *************** Autotuning format combination: Float(1,112,12544,401408) -> Float(1,112,12544,401408) *************** [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:38] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:38] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CaskConvolution) [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 1825138533642645384 time 7.23522 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 2842488832350522458 time 4.43409 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 3915320020053085238 time 7.17263 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 6448355332020552203 time 10.6272 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:38] [V] [TRT] Tactic: 6808617066150061604 time 2.87848 [07/19/2021-12:42:38] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:39] [V] [TRT] Tactic: -8060443123034038864 time 2.94344 [07/19/2021-12:42:39] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:39] [V] [TRT] Tactic: -4420849921117327522 time 1.82843 [07/19/2021-12:42:39] [V] [TRT] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:39] [V] [TRT] Tactic: -3946921629105938337 time 1.64616 [07/19/2021-12:42:39] [V] [TRT] Fastest Tactic: -3946921629105938337 Time: 1.64616 [07/19/2021-12:42:39] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaConvolution) [07/19/2021-12:42:39] [V] [TRT] Tactic: 0 time 0.27676 [07/19/2021-12:42:39] [V] [TRT] Tactic: 2 time 0.27628 [07/19/2021-12:42:39] [V] [TRT] Tactic: 5 time 25.7721 [07/19/2021-12:42:39] [V] [TRT] Tactic: 6 time 3.12586 [07/19/2021-12:42:39] [V] [TRT] Tactic: 57 time 0.276092 [07/19/2021-12:42:39] [V] [TRT] Fastest Tactic: 57 Time: 0.276092 [07/19/2021-12:42:39] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:39] [V] [TRT] Tactic: -1 time 0.113984 [07/19/2021-12:42:39] [V] [TRT] Fastest Tactic: -1 Time: 0.113984 [07/19/2021-12:42:39] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaDepthwiseConvolution Tactic: -1 [07/19/2021-12:42:39] [V] [TRT] [07/19/2021-12:42:39] [V] [TRT] *************** Autotuning format combination: Half(1,112,12544,401408) -> Half(1,112,12544,401408) *************** [07/19/2021-12:42:39] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:39] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:39] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CaskConvolution) [07/19/2021-12:42:39] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:39] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaConvolution) [07/19/2021-12:42:39] [V] [TRT] Tactic: 0 time 0.175964 [07/19/2021-12:42:39] [V] [TRT] Tactic: 1 time 0.175384 [07/19/2021-12:42:39] [V] [TRT] Tactic: 2 time 0.284384 [07/19/2021-12:42:40] [V] [TRT] Tactic: 4 time 14.6303 [07/19/2021-12:42:40] [V] [TRT] Tactic: 5 time 25.9872 [07/19/2021-12:42:40] [V] [TRT] Tactic: 6 time 8.39918 [07/19/2021-12:42:40] [V] [TRT] Fastest Tactic: 1 Time: 0.175384 [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:40] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:40] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaConvolution Tactic: 1 [07/19/2021-12:42:40] [V] [TRT] [07/19/2021-12:42:40] [V] [TRT] *************** Autotuning format combination: Half(4,448,1:8,50176) -> Float(1,112,12544,401408) *************** [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:40] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CaskConvolution) [07/19/2021-12:42:40] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaConvolution) [07/19/2021-12:42:40] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:40] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:40] [V] [TRT] *************** Autotuning format combination: Half(4,448,1:8,50176) -> Half(4,448,1:8,50176) *************** [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (FusedConvActConvolution) [07/19/2021-12:42:40] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CaskConvolution) [07/19/2021-12:42:40] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaConvolution) [07/19/2021-12:42:40] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] (CudaDepthwiseConvolution) [07/19/2021-12:42:40] [V] [TRT] Tactic: -2 time 0.061264 [07/19/2021-12:42:40] [V] [TRT] Fastest Tactic: -2 Time: 0.061264 [07/19/2021-12:42:40] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaDepthwiseConvolution Tactic: -2 [07/19/2021-12:42:40] [V] [TRT] [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:40] [V] [TRT] Tactic: 1002 time 0.22812 [07/19/2021-12:42:40] [V] [TRT] Tactic: 0 time 0.056036 [07/19/2021-12:42:40] [V] [TRT] Fastest Tactic: 0 Time: 0.056036 [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:40] [V] [TRT] Tactic: 1002 time 0.218084 [07/19/2021-12:42:40] [V] [TRT] Tactic: 0 time 0.05184 [07/19/2021-12:42:40] [V] [TRT] Fastest Tactic: 0 Time: 0.05184 [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:40] [V] [TRT] Tactic: 1002 time 0.2126 [07/19/2021-12:42:40] [V] [TRT] Tactic: 0 time 0.034728 [07/19/2021-12:42:40] [V] [TRT] Fastest Tactic: 0 Time: 0.034728 [07/19/2021-12:42:40] [V] [TRT] *************** Autotuning format combination: Float(1,112,12544,401408) -> Float(1,112,12544,200704) *************** [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1 [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1 [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:40] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:40] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:40] [V] [TRT] Tactic: 589823 time 0.263052 [07/19/2021-12:42:40] [V] [TRT] Tactic: 655359 time 0.284632 [07/19/2021-12:42:40] [V] [TRT] Tactic: 786431 time 0.218812 [07/19/2021-12:42:40] [V] [TRT] Tactic: 851967 time 0.467232 [07/19/2021-12:42:40] [V] [TRT] Tactic: 1179647 time 0.249032 [07/19/2021-12:42:40] [V] [TRT] Tactic: 1310719 time 0.344848 [07/19/2021-12:42:40] [V] [TRT] Tactic: 1376255 time 0.254752 [07/19/2021-12:42:40] [V] [TRT] Tactic: 1441791 time 0.444612 [07/19/2021-12:42:40] [V] [TRT] Tactic: 1507327 time 0.354156 [07/19/2021-12:42:40] [V] [TRT] Tactic: 1638399 time 0.2006 [07/19/2021-12:42:40] [V] [TRT] Tactic: 1835007 time 0.218916 [07/19/2021-12:42:40] [V] [TRT] Tactic: 1900543 time 0.33724 [07/19/2021-12:42:41] [V] [TRT] Tactic: 2097151 time 0.222348 [07/19/2021-12:42:41] [V] [TRT] Tactic: 2162687 time 0.2504 [07/19/2021-12:42:41] [V] [TRT] Tactic: 2293759 time 0.252652 [07/19/2021-12:42:41] [V] [TRT] Tactic: 2359295 time 0.201288 [07/19/2021-12:42:41] [V] [TRT] Tactic: 2686975 time 0.241504 [07/19/2021-12:42:41] [V] [TRT] Tactic: 3080191 time 0.31986 [07/19/2021-12:42:41] [V] [TRT] Tactic: 3342335 time 0.343824 [07/19/2021-12:42:41] [V] [TRT] Tactic: 3407871 time 0.222984 [07/19/2021-12:42:41] [V] [TRT] Tactic: 3538943 time 0.217004 [07/19/2021-12:42:41] [V] [TRT] Tactic: 3670015 time 0.324684 [07/19/2021-12:42:41] [V] [TRT] Tactic: 3932159 time 0.28958 [07/19/2021-12:42:41] [V] [TRT] Tactic: 3997695 time 0.205616 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4063231 time 0.453176 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4194303 time 0.203984 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4259839 time 0.200688 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4325375 time 0.184388 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4521983 time 0.237836 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4587519 time 0.18946 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4653055 time 0.231268 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4915199 time 0.19074 [07/19/2021-12:42:41] [V] [TRT] Tactic: 4980735 time 0.197356 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5177343 time 0.269344 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5242879 time 0.22446 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5373951 time 0.326852 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5439487 time 0.273216 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5570559 time 0.33634 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5636095 time 0.453548 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5701631 time 0.255432 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5767167 time 0.533256 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5832703 time 0.211372 [07/19/2021-12:42:41] [V] [TRT] Tactic: 5898239 time 0.272232 [07/19/2021-12:42:41] [V] [TRT] Tactic: 6029311 time 0.246732 [07/19/2021-12:42:41] [V] [TRT] Tactic: 6225919 time 0.221312 [07/19/2021-12:42:41] [V] [TRT] Tactic: 6291455 time 0.248748 [07/19/2021-12:42:41] [V] [TRT] Tactic: 6422527 time 0.29448 [07/19/2021-12:42:41] [V] [TRT] Tactic: 6750207 time 0.221168 [07/19/2021-12:42:41] [V] [TRT] Tactic: 6815743 time 0.265716 [07/19/2021-12:42:41] [V] [TRT] Tactic: 6946815 time 0.265376 [07/19/2021-12:42:41] [V] [TRT] Tactic: 7012351 time 0.222852 [07/19/2021-12:42:41] [V] [TRT] Tactic: 7077887 time 0.198208 [07/19/2021-12:42:41] [V] [TRT] Tactic: 7143423 time 0.267604 [07/19/2021-12:42:41] [V] [TRT] Tactic: 7208959 time 0.225068 [07/19/2021-12:42:42] [V] [TRT] Tactic: 7340031 time 0.270648 [07/19/2021-12:42:42] [V] [TRT] Tactic: 7405567 time 0.231592 [07/19/2021-12:42:42] [V] [TRT] Tactic: 7536639 time 0.298972 [07/19/2021-12:42:42] [V] [TRT] Tactic: 7602175 time 0.202964 [07/19/2021-12:42:42] [V] [TRT] Tactic: 7733247 time 0.195008 [07/19/2021-12:42:42] [V] [TRT] Tactic: 7798783 time 0.21994 [07/19/2021-12:42:42] [V] [TRT] Tactic: 8191999 time 0.240064 [07/19/2021-12:42:42] [V] [TRT] Tactic: 8257535 time 0.189588 [07/19/2021-12:42:42] [V] [TRT] Tactic: 8323071 time 0.219852 [07/19/2021-12:42:42] [V] [TRT] Tactic: 8650751 time 0.226212 [07/19/2021-12:42:42] [V] [TRT] Tactic: 8716287 time 0.29762 [07/19/2021-12:42:42] [V] [TRT] Tactic: 9109503 time 0.22752 [07/19/2021-12:42:42] [V] [TRT] Tactic: 9568255 time 0.190708 [07/19/2021-12:42:42] [V] [TRT] Tactic: 9895935 time 0.202012 [07/19/2021-12:42:42] [V] [TRT] Tactic: 10223615 time 0.241584 [07/19/2021-12:42:42] [V] [TRT] Tactic: 10354687 time 0.1824 [07/19/2021-12:42:42] [V] [TRT] Tactic: 10551295 time 0.236848 [07/19/2021-12:42:42] [V] [TRT] Tactic: 10747903 time 0.200572 [07/19/2021-12:42:42] [V] [TRT] Tactic: 10944511 time 0.197964 [07/19/2021-12:42:42] [V] [TRT] Fastest Tactic: 10354687 Time: 0.1824 [07/19/2021-12:42:42] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1754569683116234317 time 0.234692 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1825138533642645384 time 0.240688 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: 2733356012094739613 time 0.082832 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: 3915320020053085238 time 0.237952 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: 6808617066150061604 time 0.124832 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: 9091006216302412844 time 0.122192 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: -8060443123034038864 time 0.127576 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: -4420849921117327522 time 0.09708 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] Tactic: -3946921629105938337 time 0.084396 [07/19/2021-12:42:42] [V] [TRT] Fastest Tactic: 2733356012094739613 Time: 0.082832 [07/19/2021-12:42:42] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:42] [V] [TRT] Tactic: 0 time 0.12532 [07/19/2021-12:42:42] [V] [TRT] Tactic: 2 time 0.225448 [07/19/2021-12:42:42] [V] [TRT] Tactic: 5 time 0.348536 [07/19/2021-12:42:42] [V] [TRT] Tactic: 57 time 0.12658 [07/19/2021-12:42:42] [V] [TRT] Fastest Tactic: 0 Time: 0.12532 [07/19/2021-12:42:42] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:42] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:42] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2733356012094739613 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] *************** Autotuning format combination: Half(1,112,12544,401408) -> Half(1,112,12544,200704) *************** [07/19/2021-12:42:42] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:42] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:42] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:42] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:42] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:42] [V] [TRT] Tactic: 0 time 0.134576 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1 time 0.134928 [07/19/2021-12:42:42] [V] [TRT] Tactic: 2 time 0.206068 [07/19/2021-12:42:42] [V] [TRT] Tactic: 4 skipped. Scratch requested: 72419328, available: 67108864 [07/19/2021-12:42:42] [V] [TRT] Tactic: 5 time 0.34736 [07/19/2021-12:42:42] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [07/19/2021-12:42:42] [V] [TRT] Fastest Tactic: 0 Time: 0.134576 [07/19/2021-12:42:42] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:42] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:42] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CudaConvolution Tactic: 0 [07/19/2021-12:42:42] [V] [TRT] [07/19/2021-12:42:42] [V] [TRT] *************** Autotuning format combination: Half(1,112,12544:2,200704) -> Half(1,112,12544:2,100352) *************** [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1 [07/19/2021-12:42:42] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:42] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:42] [V] [TRT] Tactic: 589823 time 0.135112 [07/19/2021-12:42:42] [V] [TRT] Tactic: 655359 time 0.178676 [07/19/2021-12:42:42] [V] [TRT] Tactic: 786431 time 0.128732 [07/19/2021-12:42:42] [V] [TRT] Tactic: 851967 time 0.369892 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1179647 time 0.180008 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1310719 time 0.242568 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1376255 time 0.120184 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1441791 time 0.330556 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1507327 time 0.246812 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1638399 time 0.130872 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1835007 time 0.12332 [07/19/2021-12:42:42] [V] [TRT] Tactic: 1900543 time 0.205096 [07/19/2021-12:42:42] [V] [TRT] Tactic: 2162687 time 0.126108 [07/19/2021-12:42:42] [V] [TRT] Tactic: 2293759 time 0.130208 [07/19/2021-12:42:42] [V] [TRT] Tactic: 2359295 time 0.117892 [07/19/2021-12:42:42] [V] [TRT] Tactic: 2686975 time 0.127672 [07/19/2021-12:42:42] [V] [TRT] Tactic: 3080191 time 0.219808 [07/19/2021-12:42:42] [V] [TRT] Tactic: 3342335 time 0.239348 [07/19/2021-12:42:42] [V] [TRT] Tactic: 3407871 time 0.120356 [07/19/2021-12:42:42] [V] [TRT] Tactic: 3538943 time 0.135952 [07/19/2021-12:42:43] [V] [TRT] Tactic: 3670015 time 0.173536 [07/19/2021-12:42:43] [V] [TRT] Tactic: 3932159 time 0.174484 [07/19/2021-12:42:43] [V] [TRT] Tactic: 3997695 time 0.134592 [07/19/2021-12:42:43] [V] [TRT] Tactic: 4063231 time 0.365664 [07/19/2021-12:42:43] [V] [TRT] Tactic: 4194303 time 0.114856 [07/19/2021-12:42:43] [V] [TRT] Tactic: 4325375 time 0.118084 [07/19/2021-12:42:43] [V] [TRT] Tactic: 4521983 time 0.124852 [07/19/2021-12:42:43] [V] [TRT] Tactic: 4587519 time 0.126064 [07/19/2021-12:42:43] [V] [TRT] Tactic: 4653055 time 0.224368 [07/19/2021-12:42:43] [V] [TRT] Tactic: 4915199 time 0.115024 [07/19/2021-12:42:43] [V] [TRT] Tactic: 4980735 time 0.120792 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5177343 time 0.183452 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5242879 time 0.114932 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5373951 time 0.202192 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5439487 time 0.145824 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5570559 time 0.231152 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5636095 time 0.365988 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5701631 time 0.135732 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5767167 time 0.256028 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5832703 time 0.112464 [07/19/2021-12:42:43] [V] [TRT] Tactic: 5898239 time 0.206016 [07/19/2021-12:42:43] [V] [TRT] Tactic: 6029311 time 0.128484 [07/19/2021-12:42:43] [V] [TRT] Tactic: 6225919 time 0.12646 [07/19/2021-12:42:43] [V] [TRT] Tactic: 6291455 time 0.180128 [07/19/2021-12:42:43] [V] [TRT] Tactic: 6422527 time 0.20182 [07/19/2021-12:42:43] [V] [TRT] Tactic: 6750207 time 0.126044 [07/19/2021-12:42:43] [V] [TRT] Tactic: 6815743 time 0.136828 [07/19/2021-12:42:43] [V] [TRT] Tactic: 6946815 time 0.147588 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7077887 time 0.123896 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7143423 time 0.13662 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7208959 time 0.117396 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7340031 time 0.207288 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7405567 time 0.175452 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7536639 time 0.157968 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7602175 time 0.12254 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7733247 time 0.130988 [07/19/2021-12:42:43] [V] [TRT] Tactic: 7798783 time 0.128976 [07/19/2021-12:42:43] [V] [TRT] Tactic: 8191999 time 0.130464 [07/19/2021-12:42:43] [V] [TRT] Tactic: 8257535 time 0.124568 [07/19/2021-12:42:43] [V] [TRT] Tactic: 8323071 time 0.121292 [07/19/2021-12:42:43] [V] [TRT] Tactic: 8650751 time 0.127976 [07/19/2021-12:42:43] [V] [TRT] Tactic: 8716287 time 0.153916 [07/19/2021-12:42:43] [V] [TRT] Tactic: 9568255 time 0.1153 [07/19/2021-12:42:43] [V] [TRT] Tactic: 9895935 time 0.11556 [07/19/2021-12:42:43] [V] [TRT] Tactic: 10223615 time 0.127436 [07/19/2021-12:42:43] [V] [TRT] Tactic: 10354687 time 0.126452 [07/19/2021-12:42:43] [V] [TRT] Tactic: 10551295 time 0.12694 [07/19/2021-12:42:43] [V] [TRT] Tactic: 10747903 time 0.131304 [07/19/2021-12:42:43] [V] [TRT] Tactic: 10944511 time 0.119516 [07/19/2021-12:42:43] [V] [TRT] Fastest Tactic: 5832703 Time: 0.112464 [07/19/2021-12:42:43] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: 2418518597804310654 time 0.08576 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: 8292881859266835088 time 0.065124 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: 8401509141903434922 time 0.084396 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: -8654297089785671176 time 0.211396 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: -7140760933967189247 time 0.088328 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: -4097850214384059472 time 0.0545 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: -3689982367035295496 time 0.212148 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: -2534402059426524406 time 0.145084 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:43] [V] [TRT] Tactic: -2027588946874785071 time 0.063832 [07/19/2021-12:42:43] [V] [TRT] Fastest Tactic: -4097850214384059472 Time: 0.0545 [07/19/2021-12:42:43] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:43] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:43] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:43] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:43] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -4097850214384059472 [07/19/2021-12:42:43] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:43] [V] [TRT] [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_medium_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_small_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_small_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x64_relu_interior_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_medium_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x128_relu_interior_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_small_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_interior_nn_v1 [07/19/2021-12:42:44] [V] [TRT] *************** Autotuning format combination: Half(4,448,1:8,50176) -> Float(1,112,12544,200704) *************** [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:44] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:44] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:44] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:44] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:44] [V] [TRT] *************** Autotuning format combination: Half(4,448,1:8,50176) -> Half(2,224,1:8,25088) *************** [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (FusedConvActConvolution) [07/19/2021-12:42:44] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CaskConvolution) [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: 2105695814191699972 time 0.059896 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: 3754069740140581927 time 0.089068 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: 5666160310350604399 time 0.088716 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: 5925270497649423688 time 0.103852 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: 6680916730816870145 time 0.108204 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: 7158029511300006471 time 0.060216 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: 7859952145590271433 time 0.10704 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: 8283847742354150423 time 0.090312 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: -8391760416076885205 time 0.101624 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: -4534876761957424274 time 0.1035 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: -3237051169894153788 time 0.059592 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Tactic: -2422160065350346448 time 0.106268 [07/19/2021-12:42:44] [V] [TRT] Fastest Tactic: -3237051169894153788 Time: 0.059592 [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaConvolution) [07/19/2021-12:42:44] [V] [TRT] Tactic: 0 time 0.143964 [07/19/2021-12:42:44] [V] [TRT] Tactic: 1 time 0.144016 [07/19/2021-12:42:44] [V] [TRT] Tactic: 2 time 0.853668 [07/19/2021-12:42:44] [V] [TRT] Fastest Tactic: 0 Time: 0.143964 [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 6) [Convolution] (CudaDepthwiseConvolution) [07/19/2021-12:42:44] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [07/19/2021-12:42:44] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -3237051169894153788 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_128x128_ldg8_relu_exp_medium_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x128_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_sliced1x2_ldg8_relu_exp_interior_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:44] [V] [TRT] Tactic: 1002 time 0.053432 [07/19/2021-12:42:44] [V] [TRT] Tactic: 0 time 0.065448 [07/19/2021-12:42:44] [V] [TRT] Fastest Tactic: 1002 Time: 0.053432 [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:44] [V] [TRT] Tactic: 1002 time 0.12142 [07/19/2021-12:42:44] [V] [TRT] Tactic: 0 time 0.033384 [07/19/2021-12:42:44] [V] [TRT] Fastest Tactic: 0 Time: 0.033384 [07/19/2021-12:42:44] [V] [TRT] --------------- Timing Runner: (Reformat) [07/19/2021-12:42:44] [V] [TRT] Tactic: 1002 time 0.17732 [07/19/2021-12:42:44] [V] [TRT] Tactic: 0 time 0.027468 [07/19/2021-12:42:44] [V] [TRT] Fastest Tactic: 0 Time: 0.027468 [07/19/2021-12:42:44] [V] [TRT] Adding reformat layer: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] reformatted input 0 (input.1) from Float(1,224,50176,100352) to Half(1,224,50176:2,50176) [07/19/2021-12:42:44] [V] [TRT] Adding reformat layer: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] reformatted input 0 (21) from Half(1,112,12544:2,200704) to Half(4,448,1:8,50176) [07/19/2021-12:42:44] [V] [TRT] Adding reformat layer: (Unnamed Layer* 6) [Convolution] output to be reformatted 0 (output) from Float(1,112,12544,200704) to Half(2,224,1:8,25088) [07/19/2021-12:42:44] [V] [TRT] Formats and tactics selection completed in 7.96245 seconds. [07/19/2021-12:42:44] [V] [TRT] After reformat layers: 6 layers [07/19/2021-12:42:44] [V] [TRT] Block size 67108864 [07/19/2021-12:42:44] [V] [TRT] Block size 802816 [07/19/2021-12:42:44] [V] [TRT] Block size 802816 [07/19/2021-12:42:44] [V] [TRT] Total Activation Memory: 68714496 [07/19/2021-12:42:44] [I] [TRT] Detected 1 inputs and 1 output network tensors. [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] (hcudnn) Set Tactic Name: volta_fp16x2_hcudnn_fp16x2_128x32_relu_medium_nn_v1 [07/19/2021-12:42:44] [V] [TRT] (Unnamed Layer* 6) [Convolution] (h884cudnn) Set Tactic Name: volta_h884cudnn_256x64_ldg8_relu_exp_small_nhwc_tn_v1 [07/19/2021-12:42:44] [V] [TRT] Layer: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] input reformatter 0 Weights: 0 HostPersistent: 0 DevicePersistent: 0 [07/19/2021-12:42:44] [V] [TRT] Layer: (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] Weights: 0 HostPersistent: 2176 DevicePersistent: 76800 [07/19/2021-12:42:44] [V] [TRT] Layer: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] input reformatter 0 Weights: 0 HostPersistent: 0 DevicePersistent: 0 [07/19/2021-12:42:44] [V] [TRT] Layer: (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] Weights: 576 HostPersistent: 0 DevicePersistent: 0 [07/19/2021-12:42:44] [V] [TRT] Layer: (Unnamed Layer* 6) [Convolution] Weights: 0 HostPersistent: 1664 DevicePersistent: 76800 [07/19/2021-12:42:44] [V] [TRT] Layer: (Unnamed Layer* 6) [Convolution] output reformatter 0 Weights: 0 HostPersistent: 0 DevicePersistent: 0 [07/19/2021-12:42:44] [V] [TRT] Total Host Persistent Memory: 3840 [07/19/2021-12:42:44] [V] [TRT] Total Device Persistent Memory: 153600 [07/19/2021-12:42:44] [V] [TRT] Total Weight Memory: 576 [07/19/2021-12:42:44] [V] [TRT] Builder timing cache: created 31 entries, 9 hit(s) [07/19/2021-12:42:44] [V] [TRT] Engine generation completed in 9.65129 seconds. [07/19/2021-12:42:44] [V] [TRT] Engine Layer Information: [07/19/2021-12:42:44] [V] [TRT] Layer(Reformat): (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] input reformatter 0, Tactic: 0, input.1[Float(2,224,224)] -> (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] reformatted input 0[Half(2,224,224)] [07/19/2021-12:42:44] [V] [TRT] Layer(hcudnn): (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation], Tactic: 8292881859266835088, (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] reformatted input 0[Half(2,224,224)] -> 21[Half(32,112,112)] [07/19/2021-12:42:44] [V] [TRT] Layer(Reformat): (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] input reformatter 0, Tactic: 0, 21[Half(32,112,112)] -> (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] reformatted input 0[Half(32,112,112)] [07/19/2021-12:42:44] [V] [TRT] Layer(DepthwiseConvolution): (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation], Tactic: -2, (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] reformatted input 0[Half(32,112,112)] -> 24[Half(32,112,112)] [07/19/2021-12:42:44] [V] [TRT] Layer(h884cudnn): (Unnamed Layer* 6) [Convolution], Tactic: -3237051169894153788, 24[Half(32,112,112)] -> (Unnamed Layer* 6) [Convolution] output to be reformatted 0[Half(16,112,112)] [07/19/2021-12:42:44] [V] [TRT] Layer(Reformat): (Unnamed Layer* 6) [Convolution] output reformatter 0, Tactic: 0, (Unnamed Layer* 6) [Convolution] output to be reformatted 0[Half(16,112,112)] -> output[Float(16,112,112)] [07/19/2021-12:42:44] [I] Starting inference threads [07/19/2021-12:42:44] [I] Warmup completed 0 queries over 0 ms [07/19/2021-12:42:44] [I] Timing trace has 1 queries over 0.00083552 s [07/19/2021-12:42:44] [I] Trace averages of 10 runs: [07/19/2021-12:42:44] [I] Host Latency [07/19/2021-12:42:44] [I] min: 0.820288 ms (end to end 0.83552 ms) [07/19/2021-12:42:44] [I] max: 0.820288 ms (end to end 0.83552 ms) [07/19/2021-12:42:44] [I] mean: 0.820288 ms (end to end 0.83552 ms) [07/19/2021-12:42:44] [I] median: 0.820288 ms (end to end 0.83552 ms) [07/19/2021-12:42:44] [I] percentile: 0.820288 ms at 99% (end to end 0.83552 ms at 99%) [07/19/2021-12:42:44] [I] throughput: 1196.86 qps [07/19/2021-12:42:44] [I] walltime: 0.00083552 s [07/19/2021-12:42:44] [I] Enqueue Time [07/19/2021-12:42:44] [I] min: 0.654272 ms [07/19/2021-12:42:44] [I] max: 0.654272 ms [07/19/2021-12:42:44] [I] median: 0.654272 ms [07/19/2021-12:42:44] [I] GPU Compute [07/19/2021-12:42:44] [I] min: 0.655648 ms [07/19/2021-12:42:44] [I] max: 0.655648 ms [07/19/2021-12:42:44] [I] mean: 0.655648 ms [07/19/2021-12:42:44] [I] median: 0.655648 ms [07/19/2021-12:42:44] [I] percentile: 0.655648 ms at 99% [07/19/2021-12:42:44] [I] total compute time: 0.000655648 s [07/19/2021-12:42:44] [I] [07/19/2021-12:42:44] [I] === Profile (1 iterations ) === [07/19/2021-12:42:44] [I] Layer Time (ms) Avg. Time (ms) Time % [07/19/2021-12:42:44] [I] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] input reformatter 0 0.31 0.31 51.4 [07/19/2021-12:42:44] [I] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] 0.09 0.09 14.5 [07/19/2021-12:42:44] [I] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] input reformatter 0 0.04 0.04 7.1 [07/19/2021-12:42:44] [I] (Unnamed Layer* 3) [Convolution] + (Unnamed Layer* 5) [Activation] 0.06 0.06 10.6 [07/19/2021-12:42:44] [I] (Unnamed Layer* 6) [Convolution] 0.07 0.07 11.6 [07/19/2021-12:42:44] [I] (Unnamed Layer* 6) [Convolution] output reformatter 0 0.03 0.03 4.8 [07/19/2021-12:42:44] [I] Total 0.60 0.60 100.0 [07/19/2021-12:42:44] [I] &&&& PASSED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --fp16 --workspace=64 --iterations=1 --warmUp=0 --duration=0 --verbose --dumpProfile --onnx=/home/shield/data/dla_correctness.models/release_prefix/model1.onnx --exportOutput=/tmp/tmpqnvl4hzz.json