&&&& RUNNING TensorRT.trtexec # trtexec --onnx=model_fcn.onnx --verbose [04/16/2020-17:19:55] [I] === Model Options === [04/16/2020-17:19:55] [I] Format: ONNX [04/16/2020-17:19:55] [I] Model: model_fcn.onnx [04/16/2020-17:19:55] [I] Output: [04/16/2020-17:19:55] [I] === Build Options === [04/16/2020-17:19:55] [I] Max batch: 1 [04/16/2020-17:19:55] [I] Workspace: 16 MB [04/16/2020-17:19:55] [I] minTiming: 1 [04/16/2020-17:19:55] [I] avgTiming: 8 [04/16/2020-17:19:55] [I] Precision: FP32 [04/16/2020-17:19:55] [I] Calibration: [04/16/2020-17:19:55] [I] Safe mode: Disabled [04/16/2020-17:19:55] [I] Save engine: [04/16/2020-17:19:55] [I] Load engine: [04/16/2020-17:19:55] [I] Inputs format: fp32:CHW [04/16/2020-17:19:55] [I] Outputs format: fp32:CHW [04/16/2020-17:19:55] [I] Input build shapes: model [04/16/2020-17:19:55] [I] === System Options === [04/16/2020-17:19:55] [I] Device: 0 [04/16/2020-17:19:55] [I] DLACore: [04/16/2020-17:19:55] [I] Plugins: [04/16/2020-17:19:55] [I] === Inference Options === [04/16/2020-17:19:55] [I] Batch: 1 [04/16/2020-17:19:55] [I] Iterations: 10 [04/16/2020-17:19:55] [I] Duration: 3s (+ 200ms warm up) [04/16/2020-17:19:55] [I] Sleep time: 0ms [04/16/2020-17:19:55] [I] Streams: 1 [04/16/2020-17:19:55] [I] ExposeDMA: Disabled [04/16/2020-17:19:55] [I] Spin-wait: Disabled [04/16/2020-17:19:55] [I] Multithreading: Disabled [04/16/2020-17:19:55] [I] CUDA Graph: Disabled [04/16/2020-17:19:55] [I] Skip inference: Disabled [04/16/2020-17:19:55] [I] Input inference shapes: model [04/16/2020-17:19:55] [I] Inputs: [04/16/2020-17:19:55] [I] === Reporting Options === [04/16/2020-17:19:55] [I] Verbose: Enabled [04/16/2020-17:19:55] [I] Averages: 10 inferences [04/16/2020-17:19:55] [I] Percentile: 99 [04/16/2020-17:19:55] [I] Dump output: Disabled [04/16/2020-17:19:55] [I] Profile: Disabled [04/16/2020-17:19:55] [I] Export timing to JSON file: [04/16/2020-17:19:55] [I] Export output to JSON file: [04/16/2020-17:19:55] [I] Export profile to JSON file: [04/16/2020-17:19:55] [I] [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::GridAnchor_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::NMS_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::Reorg_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::Region_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::Clip_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::LReLU_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::PriorBox_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::Normalize_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::RPROI_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::BatchedNMS_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::FlattenConcat_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::CropAndResize [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::DetectionLayer_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::Proposal [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::ProposalLayer_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::PyramidROIAlign_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::ResizeNearest_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::Split [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::SpecialSlice_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ::InstanceNormalization_TRT ---------------------------------------------------------------- Input filename: model_fcn.onnx ONNX IR version: 0.0.4 Opset version: 8 Producer name: tf2onnx Producer version: 1.5.6 Domain: Model version: 0 Doc string: ---------------------------------------------------------------- [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::GridAnchor_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::NMS_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Reorg_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Region_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Clip_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::LReLU_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::PriorBox_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Normalize_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::RPROI_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::BatchedNMS_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::FlattenConcat_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::CropAndResize [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::DetectionLayer_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Proposal [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::ProposalLayer_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::PyramidROIAlign_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::ResizeNearest_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::Split [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::SpecialSlice_TRT [04/16/2020-17:19:56] [V] [TRT] Plugin creator registration succeeded - ONNXTRT_NAMESPACE::InstanceNormalization_TRT [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:203: Adding network input: x:0 with dtype: float32, dimensions: (1, 256, 256, 3) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: x:0 for ONNX tensor: x:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: new_shape__29 [04/16/2020-17:19:56] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_7/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_7/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_6/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_6/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_5/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_5/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_4/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_4/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_3/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_3/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_2/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_2/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_1/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d_1/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:90: Importing initializer: model/conv2d/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d/Conv2D__5 [Transpose] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: x:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d/Conv2D__5 [Transpose] inputs: [x:0 -> (1, 256, 256, 3)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d/Conv2D__5 for ONNX node: model/conv2d/Conv2D__5 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d/Conv2D__5:0 for ONNX tensor: model/conv2d/Conv2D__5:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d/Conv2D__5 [Transpose] outputs: [model/conv2d/Conv2D__5:0 -> (1, 3, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: Conv__21 [Conv] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d/Conv2D__5:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: Conv__21 [Conv] inputs: [model/conv2d/Conv2D__5:0 -> (1, 3, 256, 256)], [model/conv2d/Conv2D/ReadVariableOp/resource:0 -> (64, 3, 3, 3)], [model/conv2d/BiasAdd/ReadVariableOp/resource:0 -> (64)], [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:442: Convolution input dimensions: (1, 3, 256, 256) [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (1, 1), dilations: (1, 1), numOutputs: 64 [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:525: Convolution output dimensions: (1, 64, 256, 256) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: Conv__21 for ONNX node: Conv__21 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Conv__21:0 for ONNX tensor: Conv__21:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: Conv__21 [Conv] outputs: [Conv__21:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d/Relu [Relu] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: Conv__21:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d/Relu [Relu] inputs: [Conv__21:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d/Relu for ONNX node: model/conv2d/Relu [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d/Relu:0 for ONNX tensor: model/conv2d/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d/Relu [Relu] outputs: [model/conv2d/Relu:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: Conv__22 [Conv] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_1/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_1/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: Conv__22 [Conv] inputs: [model/conv2d/Relu:0 -> (1, 64, 256, 256)], [model/conv2d_1/Conv2D/ReadVariableOp/resource:0 -> (128, 64, 3, 3)], [model/conv2d_1/BiasAdd/ReadVariableOp/resource:0 -> (128)], [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:442: Convolution input dimensions: (1, 64, 256, 256) [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (1, 1), dilations: (1, 1), numOutputs: 128 [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:525: Convolution output dimensions: (1, 128, 256, 256) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: Conv__22 for ONNX node: Conv__22 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Conv__22:0 for ONNX tensor: Conv__22:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: Conv__22 [Conv] outputs: [Conv__22:0 -> (1, 128, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d_1/Relu [Relu] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: Conv__22:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d_1/Relu [Relu] inputs: [Conv__22:0 -> (1, 128, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d_1/Relu for ONNX node: model/conv2d_1/Relu [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d_1/Relu:0 for ONNX tensor: model/conv2d_1/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d_1/Relu [Relu] outputs: [model/conv2d_1/Relu:0 -> (1, 128, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: Conv__23 [Conv] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_1/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_2/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_2/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: Conv__23 [Conv] inputs: [model/conv2d_1/Relu:0 -> (1, 128, 256, 256)], [model/conv2d_2/Conv2D/ReadVariableOp/resource:0 -> (128, 128, 3, 3)], [model/conv2d_2/BiasAdd/ReadVariableOp/resource:0 -> (128)], [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:442: Convolution input dimensions: (1, 128, 256, 256) [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (1, 1), dilations: (1, 1), numOutputs: 128 [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:525: Convolution output dimensions: (1, 128, 256, 256) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: Conv__23 for ONNX node: Conv__23 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Conv__23:0 for ONNX tensor: Conv__23:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: Conv__23 [Conv] outputs: [Conv__23:0 -> (1, 128, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d_2/Relu [Relu] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: Conv__23:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d_2/Relu [Relu] inputs: [Conv__23:0 -> (1, 128, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d_2/Relu for ONNX node: model/conv2d_2/Relu [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d_2/Relu:0 for ONNX tensor: model/conv2d_2/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d_2/Relu [Relu] outputs: [model/conv2d_2/Relu:0 -> (1, 128, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: Conv__24 [Conv] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_2/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_3/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_3/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: Conv__24 [Conv] inputs: [model/conv2d_2/Relu:0 -> (1, 128, 256, 256)], [model/conv2d_3/Conv2D/ReadVariableOp/resource:0 -> (64, 128, 3, 3)], [model/conv2d_3/BiasAdd/ReadVariableOp/resource:0 -> (64)], [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:442: Convolution input dimensions: (1, 128, 256, 256) [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (1, 1), dilations: (1, 1), numOutputs: 64 [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:525: Convolution output dimensions: (1, 64, 256, 256) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: Conv__24 for ONNX node: Conv__24 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Conv__24:0 for ONNX tensor: Conv__24:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: Conv__24 [Conv] outputs: [Conv__24:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d_3/Relu [Relu] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: Conv__24:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d_3/Relu [Relu] inputs: [Conv__24:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d_3/Relu for ONNX node: model/conv2d_3/Relu [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d_3/Relu:0 for ONNX tensor: model/conv2d_3/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d_3/Relu [Relu] outputs: [model/conv2d_3/Relu:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: Conv__25 [Conv] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_3/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_4/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_4/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: Conv__25 [Conv] inputs: [model/conv2d_3/Relu:0 -> (1, 64, 256, 256)], [model/conv2d_4/Conv2D/ReadVariableOp/resource:0 -> (64, 64, 3, 3)], [model/conv2d_4/BiasAdd/ReadVariableOp/resource:0 -> (64)], [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:442: Convolution input dimensions: (1, 64, 256, 256) [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (1, 1), dilations: (1, 1), numOutputs: 64 [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:525: Convolution output dimensions: (1, 64, 256, 256) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: Conv__25 for ONNX node: Conv__25 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Conv__25:0 for ONNX tensor: Conv__25:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: Conv__25 [Conv] outputs: [Conv__25:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d_4/Relu [Relu] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: Conv__25:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d_4/Relu [Relu] inputs: [Conv__25:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d_4/Relu for ONNX node: model/conv2d_4/Relu [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d_4/Relu:0 for ONNX tensor: model/conv2d_4/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d_4/Relu [Relu] outputs: [model/conv2d_4/Relu:0 -> (1, 64, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: Conv__26 [Conv] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_4/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_5/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_5/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: Conv__26 [Conv] inputs: [model/conv2d_4/Relu:0 -> (1, 64, 256, 256)], [model/conv2d_5/Conv2D/ReadVariableOp/resource:0 -> (32, 64, 3, 3)], [model/conv2d_5/BiasAdd/ReadVariableOp/resource:0 -> (32)], [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:442: Convolution input dimensions: (1, 64, 256, 256) [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (1, 1), dilations: (1, 1), numOutputs: 32 [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:525: Convolution output dimensions: (1, 32, 256, 256) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: Conv__26 for ONNX node: Conv__26 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Conv__26:0 for ONNX tensor: Conv__26:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: Conv__26 [Conv] outputs: [Conv__26:0 -> (1, 32, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d_5/Relu [Relu] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: Conv__26:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d_5/Relu [Relu] inputs: [Conv__26:0 -> (1, 32, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d_5/Relu for ONNX node: model/conv2d_5/Relu [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d_5/Relu:0 for ONNX tensor: model/conv2d_5/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d_5/Relu [Relu] outputs: [model/conv2d_5/Relu:0 -> (1, 32, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: Conv__27 [Conv] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_5/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_6/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_6/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: Conv__27 [Conv] inputs: [model/conv2d_5/Relu:0 -> (1, 32, 256, 256)], [model/conv2d_6/Conv2D/ReadVariableOp/resource:0 -> (16, 32, 3, 3)], [model/conv2d_6/BiasAdd/ReadVariableOp/resource:0 -> (16)], [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:442: Convolution input dimensions: (1, 32, 256, 256) [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (1, 1), dilations: (1, 1), numOutputs: 16 [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:525: Convolution output dimensions: (1, 16, 256, 256) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: Conv__27 for ONNX node: Conv__27 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Conv__27:0 for ONNX tensor: Conv__27:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: Conv__27 [Conv] outputs: [Conv__27:0 -> (1, 16, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d_6/Relu [Relu] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: Conv__27:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d_6/Relu [Relu] inputs: [Conv__27:0 -> (1, 16, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d_6/Relu for ONNX node: model/conv2d_6/Relu [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d_6/Relu:0 for ONNX tensor: model/conv2d_6/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d_6/Relu [Relu] outputs: [model/conv2d_6/Relu:0 -> (1, 16, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: Conv__28 [Conv] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_6/Relu:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_7/Conv2D/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_7/BiasAdd/ReadVariableOp/resource:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: Conv__28 [Conv] inputs: [model/conv2d_6/Relu:0 -> (1, 16, 256, 256)], [model/conv2d_7/Conv2D/ReadVariableOp/resource:0 -> (1, 16, 3, 3)], [model/conv2d_7/BiasAdd/ReadVariableOp/resource:0 -> (1)], [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:442: Convolution input dimensions: (1, 16, 256, 256) [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:524: Using kernel: (3, 3), strides: (1, 1), padding: (1, 1), dilations: (1, 1), numOutputs: 1 [04/16/2020-17:19:56] [V] [TRT] builtin_op_importers.cpp:525: Convolution output dimensions: (1, 1, 256, 256) [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: Conv__28 for ONNX node: Conv__28 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Conv__28:0 for ONNX tensor: Conv__28:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: Conv__28 [Conv] outputs: [Conv__28:0 -> (1, 1, 256, 256)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d_7/Conv2D__20 [Reshape] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: Conv__28:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: new_shape__29 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d_7/Conv2D__20 [Reshape] inputs: [Conv__28:0 -> (1, 1, 256, 256)], [new_shape__29 -> (4)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d_7/Conv2D__20 for ONNX node: model/conv2d_7/Conv2D__20 [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: model/conv2d_7/Conv2D__20:0 for ONNX tensor: model/conv2d_7/Conv2D__20:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d_7/Conv2D__20 [Reshape] outputs: [model/conv2d_7/Conv2D__20:0 -> (1, 256, 256, 1)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:107: Parsing node: model/conv2d_7/Sigmoid [Sigmoid] [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:123: Searching for input: model/conv2d_7/Conv2D__20:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:129: model/conv2d_7/Sigmoid [Sigmoid] inputs: [model/conv2d_7/Conv2D__20:0 -> (1, 256, 256, 1)], [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:122: Registering layer: model/conv2d_7/Sigmoid for ONNX node: model/conv2d_7/Sigmoid [04/16/2020-17:19:56] [V] [TRT] ImporterContext.hpp:97: Registering tensor: Identity:0_1 for ONNX tensor: Identity:0 [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:180: model/conv2d_7/Sigmoid [Sigmoid] outputs: [Identity:0 -> (1, 256, 256, 1)], [04/16/2020-17:19:56] [V] [TRT] ModelImporter.cpp:494: Marking Identity:0_1 as output: Identity:0 ----- Parsing of ONNX model model_fcn.onnx is Done ---- [04/16/2020-17:19:56] [V] [TRT] Applying generic optimizations to the graph for inference. [04/16/2020-17:19:56] [V] [TRT] Original: 18 layers [04/16/2020-17:19:56] [V] [TRT] After dead-layer removal: 18 layers [04/16/2020-17:19:56] [V] [TRT] After Myelin optimization: 18 layers [04/16/2020-17:19:56] [V] [TRT] After scale fusion: 18 layers [04/16/2020-17:19:56] [V] [TRT] Fusing Conv__21 with model/conv2d/Relu [04/16/2020-17:19:56] [V] [TRT] Fusing Conv__22 with model/conv2d_1/Relu [04/16/2020-17:19:56] [V] [TRT] Fusing Conv__23 with model/conv2d_2/Relu [04/16/2020-17:19:56] [V] [TRT] Fusing Conv__24 with model/conv2d_3/Relu [04/16/2020-17:19:56] [V] [TRT] Fusing Conv__25 with model/conv2d_4/Relu [04/16/2020-17:19:56] [V] [TRT] Fusing Conv__26 with model/conv2d_5/Relu [04/16/2020-17:19:56] [V] [TRT] Fusing Conv__27 with model/conv2d_6/Relu [04/16/2020-17:19:56] [V] [TRT] After vertical fusions: 11 layers [04/16/2020-17:19:56] [V] [TRT] After final dead-layer removal: 11 layers [04/16/2020-17:19:56] [V] [TRT] After tensor merging: 11 layers [04/16/2020-17:19:56] [V] [TRT] After concat removal: 11 layers [04/16/2020-17:19:56] [V] [TRT] Graph construction and optimization completed in 0.00424223 seconds. [04/16/2020-17:19:58] [V] [TRT] Constructing optimization profile number 0 out of 1 --------------- Timing Runner: (Reformat) [04/16/2020-17:19:58] [V] [TRT] Tactic: 0 time 0.042976 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 0 Time: 0.042976 [04/16/2020-17:19:58] [V] [TRT] *************** Autotuning format combination: Float(1,3,768,196608) -> Float(1,256,65536,196608) *************** [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: model/conv2d/Conv2D__5 (Shuffle) [04/16/2020-17:19:58] [V] [TRT] Tactic: 0 time 0.015872 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 0 Time: 0.015872 [04/16/2020-17:19:58] [V] [TRT] *************** Autotuning format combination: Float(1,3,768:32,6144) -> Float(1,256,65536:32,65536) *************** [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: model/conv2d/Conv2D__5 (Shuffle) [04/16/2020-17:19:58] [V] [TRT] Tactic: 0 time 0.157728 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 0 Time: 0.157728 [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: (Reformat) [04/16/2020-17:19:58] [V] [TRT] Tactic: 0 time 0.064512 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 0 Time: 0.064512 [04/16/2020-17:19:58] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,196608) -> Float(1,256,65536,4194304) *************** [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__21 + model/conv2d/Relu (LegacySASSConvolution) [04/16/2020-17:19:58] [V] [TRT] LegacySASSConvolution has no valid tactics for this config, skipping [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__21 + model/conv2d/Relu (FusedConvActConvolution) [04/16/2020-17:19:58] [V] [TRT] Tactic: 7 time 0.761856 [04/16/2020-17:19:58] [V] [TRT] Tactic: 29 time 0.760192 [04/16/2020-17:19:58] [V] [TRT] Tactic: 30 time 0.765888 [04/16/2020-17:19:58] [V] [TRT] Tactic: 43 time 0.741856 [04/16/2020-17:19:58] [V] [TRT] Tactic: 66 time 0.760832 [04/16/2020-17:19:58] [V] [TRT] Tactic: 90 time 0.759808 [04/16/2020-17:19:58] [V] [TRT] Tactic: 104 time 0.756736 [04/16/2020-17:19:58] [V] [TRT] Tactic: 130 time 0.753632 [04/16/2020-17:19:58] [V] [TRT] Tactic: 136 time 0.760832 [04/16/2020-17:19:58] [V] [TRT] Tactic: 144 time 0.759808 [04/16/2020-17:19:58] [V] [TRT] Tactic: 153 time 0.758752 [04/16/2020-17:19:58] [V] [TRT] Tactic: 156 time 0.761536 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 43 Time: 0.741856 [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__21 + model/conv2d/Relu (CaskConvolution) [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 1825138533642645384 time 0.294912 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 2775507031594384867 time 0.426976 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 2842488832350522458 time 0.174048 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 3915320020053085238 time 0.294944 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 6448355332020552203 time 0.41168 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 6808617066150061604 time 0.158688 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: -8060443123034038864 time 0.160704 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: -4420849921117327522 time 0.18528 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: -3946921629105938337 time 0.164352 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 6808617066150061604 Time: 0.158688 [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__21 + model/conv2d/Relu (CudaConvolution) [04/16/2020-17:19:58] [V] [TRT] Tactic: 0 time 0.578176 [04/16/2020-17:19:58] [V] [TRT] Tactic: 1 time 0.471008 [04/16/2020-17:19:58] [V] [TRT] Tactic: 2 time 0.636704 [04/16/2020-17:19:58] [V] [TRT] Tactic: 5 time 4.3609 [04/16/2020-17:19:58] [V] [TRT] Tactic: 6 time 0.73008 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 1 Time: 0.471008 [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__21 + model/conv2d/Relu (CudaDepthwiseConvolution) [04/16/2020-17:19:58] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [04/16/2020-17:19:58] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 6808617066150061604 [04/16/2020-17:19:58] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:19:58] [V] [TRT] [04/16/2020-17:19:58] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,4194304) -> Float(1,256,65536,8388608) *************** [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__22 + model/conv2d_1/Relu (LegacySASSConvolution) [04/16/2020-17:19:58] [V] [TRT] LegacySASSConvolution has no valid tactics for this config, skipping [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__22 + model/conv2d_1/Relu (FusedConvActConvolution) [04/16/2020-17:19:58] [V] [TRT] Tactic: 7 time 4.32438 [04/16/2020-17:19:58] [V] [TRT] Tactic: 10 time 2.78122 [04/16/2020-17:19:58] [V] [TRT] Tactic: 14 time 2.86272 [04/16/2020-17:19:58] [V] [TRT] Tactic: 15 time 2.82029 [04/16/2020-17:19:58] [V] [TRT] Tactic: 25 time 2.84307 [04/16/2020-17:19:58] [V] [TRT] Tactic: 26 time 3.60432 [04/16/2020-17:19:58] [V] [TRT] Tactic: 29 time 3.48198 [04/16/2020-17:19:58] [V] [TRT] Tactic: 30 time 3.41405 [04/16/2020-17:19:58] [V] [TRT] Tactic: 33 time 2.77168 [04/16/2020-17:19:58] [V] [TRT] Tactic: 36 time 3.08118 [04/16/2020-17:19:58] [V] [TRT] Tactic: 39 time 3.29523 [04/16/2020-17:19:58] [V] [TRT] Tactic: 41 time 2.77638 [04/16/2020-17:19:58] [V] [TRT] Tactic: 42 time 2.12794 [04/16/2020-17:19:58] [V] [TRT] Tactic: 43 time 2.20602 [04/16/2020-17:19:58] [V] [TRT] Tactic: 45 time 1.72998 [04/16/2020-17:19:58] [V] [TRT] Tactic: 47 time 1.71043 [04/16/2020-17:19:58] [V] [TRT] Tactic: 52 time 1.83008 [04/16/2020-17:19:58] [V] [TRT] Tactic: 54 time 1.74285 [04/16/2020-17:19:58] [V] [TRT] Tactic: 56 time 2.0439 [04/16/2020-17:19:58] [V] [TRT] Tactic: 66 time 2.23498 [04/16/2020-17:19:58] [V] [TRT] Tactic: 76 time 1.82493 [04/16/2020-17:19:58] [V] [TRT] Tactic: 90 time 2.21594 [04/16/2020-17:19:58] [V] [TRT] Tactic: 93 time 2.18522 [04/16/2020-17:19:58] [V] [TRT] Tactic: 98 time 1.87011 [04/16/2020-17:19:58] [V] [TRT] Tactic: 104 time 2.20397 [04/16/2020-17:19:58] [V] [TRT] Tactic: 110 time 2.12413 [04/16/2020-17:19:58] [V] [TRT] Tactic: 119 time 2.01146 [04/16/2020-17:19:58] [V] [TRT] Tactic: 121 time 1.75638 [04/16/2020-17:19:58] [V] [TRT] Tactic: 130 time 2.09485 [04/16/2020-17:19:58] [V] [TRT] Tactic: 134 time 2.54173 [04/16/2020-17:19:58] [V] [TRT] Tactic: 136 time 2.10291 [04/16/2020-17:19:58] [V] [TRT] Tactic: 137 time 1.73325 [04/16/2020-17:19:58] [V] [TRT] Tactic: 139 time 1.68365 [04/16/2020-17:19:58] [V] [TRT] Tactic: 144 time 2.15056 [04/16/2020-17:19:58] [V] [TRT] Tactic: 149 time 1.82266 [04/16/2020-17:19:58] [V] [TRT] Tactic: 151 time 2.31002 [04/16/2020-17:19:58] [V] [TRT] Tactic: 152 time 1.78413 [04/16/2020-17:19:58] [V] [TRT] Tactic: 153 time 2.19546 [04/16/2020-17:19:58] [V] [TRT] Tactic: 156 time 2.21626 [04/16/2020-17:19:58] [V] [TRT] Tactic: 159 time 1.75082 [04/16/2020-17:19:58] [V] [TRT] Tactic: 162 time 1.90259 [04/16/2020-17:19:58] [V] [TRT] Tactic: 164 time 1.74486 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 139 Time: 1.68365 [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__22 + model/conv2d_1/Relu (CaskConvolution) [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 1825138533642645384 time 1.60816 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 2775507031594384867 time 1.27418 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 2842488832350522458 time 1.63366 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 3915320020053085238 time 1.60339 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 6448355332020552203 time 1.62816 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: 6808617066150061604 time 1.59766 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: -8060443123034038864 time 1.61526 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: -4420849921117327522 time 1.9839 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [04/16/2020-17:19:58] [V] [TRT] Tactic: -3946921629105938337 time 1.73875 [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 2775507031594384867 Time: 1.27418 [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__22 + model/conv2d_1/Relu (CudaConvolution) [04/16/2020-17:19:58] [V] [TRT] Tactic: 0 time 2.65264 [04/16/2020-17:19:58] [V] [TRT] Tactic: 1 time 2.08115 [04/16/2020-17:19:58] [V] [TRT] Tactic: 2 skipped. Scratch requested: 150994944, available: 16777216 [04/16/2020-17:19:58] [V] [TRT] Tactic: 5 skipped. Scratch requested: 37371904, available: 16777216 [04/16/2020-17:19:58] [V] [TRT] Tactic: 6 time 1.73654 [04/16/2020-17:19:58] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [04/16/2020-17:19:58] [V] [TRT] Fastest Tactic: 6 Time: 1.73654 [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__22 + model/conv2d_1/Relu (CudaDepthwiseConvolution) [04/16/2020-17:19:58] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [04/16/2020-17:19:58] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2775507031594384867 [04/16/2020-17:19:58] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:19:58] [V] [TRT] [04/16/2020-17:19:58] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,8388608) -> Float(1,256,65536,8388608) *************** [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__23 + model/conv2d_2/Relu (LegacySASSConvolution) [04/16/2020-17:19:58] [V] [TRT] LegacySASSConvolution has no valid tactics for this config, skipping [04/16/2020-17:19:58] [V] [TRT] --------------- Timing Runner: Conv__23 + model/conv2d_2/Relu (FusedConvActConvolution) [04/16/2020-17:19:58] [V] [TRT] Tactic: 7 time 3.26698 [04/16/2020-17:19:58] [V] [TRT] Tactic: 10 time 3.0151 [04/16/2020-17:19:58] [V] [TRT] Tactic: 14 time 3.39766 [04/16/2020-17:19:58] [V] [TRT] Tactic: 15 time 3.49357 [04/16/2020-17:19:59] [V] [TRT] Tactic: 25 time 3.55443 [04/16/2020-17:19:59] [V] [TRT] Tactic: 26 time 4.14106 [04/16/2020-17:19:59] [V] [TRT] Tactic: 29 time 3.8137 [04/16/2020-17:19:59] [V] [TRT] Tactic: 30 time 3.72982 [04/16/2020-17:19:59] [V] [TRT] Tactic: 33 time 3.36262 [04/16/2020-17:19:59] [V] [TRT] Tactic: 36 time 3.69805 [04/16/2020-17:19:59] [V] [TRT] Tactic: 39 time 4.02394 [04/16/2020-17:19:59] [V] [TRT] Tactic: 41 time 3.58195 [04/16/2020-17:19:59] [V] [TRT] Tactic: 42 time 3.73997 [04/16/2020-17:19:59] [V] [TRT] Tactic: 43 time 3.74554 [04/16/2020-17:19:59] [V] [TRT] Tactic: 45 time 3.38058 [04/16/2020-17:19:59] [V] [TRT] Tactic: 47 time 3.36077 [04/16/2020-17:19:59] [V] [TRT] Tactic: 52 time 3.52272 [04/16/2020-17:19:59] [V] [TRT] Tactic: 54 time 3.51027 [04/16/2020-17:19:59] [V] [TRT] Tactic: 56 time 3.91962 [04/16/2020-17:19:59] [V] [TRT] Tactic: 66 time 4.15974 [04/16/2020-17:19:59] [V] [TRT] Tactic: 76 time 3.74374 [04/16/2020-17:19:59] [V] [TRT] Tactic: 90 time 4.04182 [04/16/2020-17:19:59] [V] [TRT] Tactic: 93 time 3.82701 [04/16/2020-17:19:59] [V] [TRT] Tactic: 98 time 3.58429 [04/16/2020-17:19:59] [V] [TRT] Tactic: 104 time 3.94909 [04/16/2020-17:19:59] [V] [TRT] Tactic: 110 time 3.78502 [04/16/2020-17:19:59] [V] [TRT] Tactic: 119 time 3.96442 [04/16/2020-17:19:59] [V] [TRT] Tactic: 121 time 3.57984 [04/16/2020-17:19:59] [V] [TRT] Tactic: 130 time 3.92426 [04/16/2020-17:19:59] [V] [TRT] Tactic: 134 time 4.36051 [04/16/2020-17:19:59] [V] [TRT] Tactic: 136 time 3.81984 [04/16/2020-17:19:59] [V] [TRT] Tactic: 137 time 3.38944 [04/16/2020-17:19:59] [V] [TRT] Tactic: 139 time 3.30531 [04/16/2020-17:19:59] [V] [TRT] Tactic: 144 time 3.8144 [04/16/2020-17:19:59] [V] [TRT] Tactic: 149 time 3.41197 [04/16/2020-17:19:59] [V] [TRT] Tactic: 151 time 3.92762 [04/16/2020-17:19:59] [V] [TRT] Tactic: 152 time 3.5185 [04/16/2020-17:19:59] [V] [TRT] Tactic: 153 time 3.9153 [04/16/2020-17:19:59] [V] [TRT] Tactic: 156 time 3.7929 [04/16/2020-17:19:59] [V] [TRT] Tactic: 159 time 3.42259 [04/16/2020-17:19:59] [V] [TRT] Tactic: 162 time 3.74134 [04/16/2020-17:19:59] [V] [TRT] Tactic: 164 time 3.65155 [04/16/2020-17:19:59] [V] [TRT] Fastest Tactic: 10 Time: 3.0151 [04/16/2020-17:19:59] [V] [TRT] --------------- Timing Runner: Conv__23 + model/conv2d_2/Relu (CaskConvolution) [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: 1825138533642645384 time 3.38736 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: 2775507031594384867 time 2.3104 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: 2842488832350522458 time 3.20509 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: 3915320020053085238 time 2.99213 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: 6448355332020552203 time 3.10272 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: 6808617066150061604 time 3.05117 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: -8060443123034038864 time 3.08262 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: -4420849921117327522 time 3.84042 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [04/16/2020-17:19:59] [V] [TRT] Tactic: -3946921629105938337 time 3.3848 [04/16/2020-17:19:59] [V] [TRT] Fastest Tactic: 2775507031594384867 Time: 2.3104 [04/16/2020-17:19:59] [V] [TRT] --------------- Timing Runner: Conv__23 + model/conv2d_2/Relu (CudaConvolution) [04/16/2020-17:19:59] [V] [TRT] Tactic: 0 time 4.78989 [04/16/2020-17:19:59] [V] [TRT] Tactic: 1 time 3.6104 [04/16/2020-17:19:59] [V] [TRT] Tactic: 2 skipped. Scratch requested: 301989888, available: 16777216 [04/16/2020-17:19:59] [V] [TRT] Tactic: 5 skipped. Scratch requested: 73580544, available: 16777216 [04/16/2020-17:19:59] [V] [TRT] Tactic: 6 time 2.67219 [04/16/2020-17:19:59] [V] [TRT] Fastest Tactic: 6 Time: 2.67219 [04/16/2020-17:19:59] [V] [TRT] --------------- Timing Runner: Conv__23 + model/conv2d_2/Relu (CudaDepthwiseConvolution) [04/16/2020-17:19:59] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [04/16/2020-17:19:59] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2775507031594384867 [04/16/2020-17:19:59] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:19:59] [V] [TRT] [04/16/2020-17:19:59] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,8388608) -> Float(1,256,65536,4194304) *************** [04/16/2020-17:19:59] [V] [TRT] --------------- Timing Runner: Conv__24 + model/conv2d_3/Relu (LegacySASSConvolution) [04/16/2020-17:19:59] [V] [TRT] LegacySASSConvolution has no valid tactics for this config, skipping [04/16/2020-17:19:59] [V] [TRT] --------------- Timing Runner: Conv__24 + model/conv2d_3/Relu (FusedConvActConvolution) [04/16/2020-17:19:59] [V] [TRT] Tactic: 7 time 1.77357 [04/16/2020-17:19:59] [V] [TRT] Tactic: 10 time 1.61965 [04/16/2020-17:19:59] [V] [TRT] Tactic: 14 time 1.63264 [04/16/2020-17:19:59] [V] [TRT] Tactic: 15 time 1.66608 [04/16/2020-17:20:00] [V] [TRT] Tactic: 25 time 1.67731 [04/16/2020-17:20:00] [V] [TRT] Tactic: 26 time 1.94698 [04/16/2020-17:20:00] [V] [TRT] Tactic: 29 time 1.96221 [04/16/2020-17:20:00] [V] [TRT] Tactic: 30 time 1.94346 [04/16/2020-17:20:00] [V] [TRT] Tactic: 33 time 1.70371 [04/16/2020-17:20:00] [V] [TRT] Tactic: 36 time 1.76742 [04/16/2020-17:20:00] [V] [TRT] Tactic: 39 time 1.86605 [04/16/2020-17:20:00] [V] [TRT] Tactic: 41 time 1.71162 [04/16/2020-17:20:00] [V] [TRT] Tactic: 42 time 3.34192 [04/16/2020-17:20:00] [V] [TRT] Tactic: 43 time 3.23597 [04/16/2020-17:20:00] [V] [TRT] Tactic: 45 time 1.64198 [04/16/2020-17:20:00] [V] [TRT] Tactic: 47 time 1.62845 [04/16/2020-17:20:00] [V] [TRT] Tactic: 52 time 3.16659 [04/16/2020-17:20:00] [V] [TRT] Tactic: 54 time 1.62592 [04/16/2020-17:20:00] [V] [TRT] Tactic: 56 time 3.17008 [04/16/2020-17:20:00] [V] [TRT] Tactic: 66 time 2.09334 [04/16/2020-17:20:00] [V] [TRT] Tactic: 76 time 1.79642 [04/16/2020-17:20:00] [V] [TRT] Tactic: 90 time 1.95174 [04/16/2020-17:20:00] [V] [TRT] Tactic: 93 time 1.91043 [04/16/2020-17:20:00] [V] [TRT] Tactic: 98 time 1.78381 [04/16/2020-17:20:00] [V] [TRT] Tactic: 104 time 1.89485 [04/16/2020-17:20:00] [V] [TRT] Tactic: 110 time 1.85686 [04/16/2020-17:20:00] [V] [TRT] Tactic: 119 time 1.84125 [04/16/2020-17:20:00] [V] [TRT] Tactic: 121 time 1.67885 [04/16/2020-17:20:00] [V] [TRT] Tactic: 130 time 1.93104 [04/16/2020-17:20:00] [V] [TRT] Tactic: 134 time 2.15366 [04/16/2020-17:20:00] [V] [TRT] Tactic: 136 time 1.88768 [04/16/2020-17:20:00] [V] [TRT] Tactic: 137 time 1.70224 [04/16/2020-17:20:00] [V] [TRT] Tactic: 139 time 1.66451 [04/16/2020-17:20:00] [V] [TRT] Tactic: 144 time 1.93776 [04/16/2020-17:20:00] [V] [TRT] Tactic: 149 time 3.10272 [04/16/2020-17:20:00] [V] [TRT] Tactic: 151 time 1.90608 [04/16/2020-17:20:00] [V] [TRT] Tactic: 152 time 1.68995 [04/16/2020-17:20:00] [V] [TRT] Tactic: 153 time 1.95789 [04/16/2020-17:20:00] [V] [TRT] Tactic: 156 time 1.85552 [04/16/2020-17:20:00] [V] [TRT] Tactic: 159 time 1.67146 [04/16/2020-17:20:00] [V] [TRT] Tactic: 162 time 1.74541 [04/16/2020-17:20:00] [V] [TRT] Tactic: 164 time 1.68128 [04/16/2020-17:20:00] [V] [TRT] Fastest Tactic: 10 Time: 1.61965 [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__24 + model/conv2d_3/Relu (CaskConvolution) [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 1825138533642645384 time 3.1288 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 2775507031594384867 time 1.12403 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 2842488832350522458 time 1.58106 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 3915320020053085238 time 2.94278 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 6448355332020552203 time 3.02938 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 6808617066150061604 time 1.55706 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: -8060443123034038864 time 1.51142 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: -4420849921117327522 time 1.90464 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: -3946921629105938337 time 1.67773 [04/16/2020-17:20:00] [V] [TRT] Fastest Tactic: 2775507031594384867 Time: 1.12403 [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__24 + model/conv2d_3/Relu (CudaConvolution) [04/16/2020-17:20:00] [V] [TRT] Tactic: 0 time 3.1273 [04/16/2020-17:20:00] [V] [TRT] Tactic: 1 time 1.74941 [04/16/2020-17:20:00] [V] [TRT] Tactic: 2 skipped. Scratch requested: 301989888, available: 16777216 [04/16/2020-17:20:00] [V] [TRT] Tactic: 5 skipped. Scratch requested: 37371904, available: 16777216 [04/16/2020-17:20:00] [V] [TRT] Tactic: 6 time 1.32493 [04/16/2020-17:20:00] [V] [TRT] Fastest Tactic: 6 Time: 1.32493 [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__24 + model/conv2d_3/Relu (CudaDepthwiseConvolution) [04/16/2020-17:20:00] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:00] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2775507031594384867 [04/16/2020-17:20:00] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:00] [V] [TRT] [04/16/2020-17:20:00] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,4194304) -> Float(1,256,65536,4194304) *************** [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__25 + model/conv2d_4/Relu (LegacySASSConvolution) [04/16/2020-17:20:00] [V] [TRT] LegacySASSConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__25 + model/conv2d_4/Relu (FusedConvActConvolution) [04/16/2020-17:20:00] [V] [TRT] Tactic: 7 time 0.991232 [04/16/2020-17:20:00] [V] [TRT] Tactic: 10 time 0.853856 [04/16/2020-17:20:00] [V] [TRT] Tactic: 14 time 0.869888 [04/16/2020-17:20:00] [V] [TRT] Tactic: 15 time 0.858496 [04/16/2020-17:20:00] [V] [TRT] Tactic: 25 time 0.856064 [04/16/2020-17:20:00] [V] [TRT] Tactic: 26 time 1.10387 [04/16/2020-17:20:00] [V] [TRT] Tactic: 29 time 1.05405 [04/16/2020-17:20:00] [V] [TRT] Tactic: 30 time 1.05526 [04/16/2020-17:20:00] [V] [TRT] Tactic: 33 time 0.856544 [04/16/2020-17:20:00] [V] [TRT] Tactic: 36 time 0.939392 [04/16/2020-17:20:00] [V] [TRT] Tactic: 39 time 1.00966 [04/16/2020-17:20:00] [V] [TRT] Tactic: 41 time 0.856064 [04/16/2020-17:20:00] [V] [TRT] Tactic: 42 time 1.67722 [04/16/2020-17:20:00] [V] [TRT] Tactic: 43 time 1.68755 [04/16/2020-17:20:00] [V] [TRT] Tactic: 45 time 0.851456 [04/16/2020-17:20:00] [V] [TRT] Tactic: 47 time 0.84416 [04/16/2020-17:20:00] [V] [TRT] Tactic: 52 time 1.60358 [04/16/2020-17:20:00] [V] [TRT] Tactic: 54 time 0.837632 [04/16/2020-17:20:00] [V] [TRT] Tactic: 56 time 1.57222 [04/16/2020-17:20:00] [V] [TRT] Tactic: 66 time 1.08893 [04/16/2020-17:20:00] [V] [TRT] Tactic: 76 time 0.87248 [04/16/2020-17:20:00] [V] [TRT] Tactic: 90 time 1.07174 [04/16/2020-17:20:00] [V] [TRT] Tactic: 93 time 1.02413 [04/16/2020-17:20:00] [V] [TRT] Tactic: 98 time 0.899072 [04/16/2020-17:20:00] [V] [TRT] Tactic: 104 time 1.06445 [04/16/2020-17:20:00] [V] [TRT] Tactic: 110 time 1.03568 [04/16/2020-17:20:00] [V] [TRT] Tactic: 119 time 0.988544 [04/16/2020-17:20:00] [V] [TRT] Tactic: 121 time 0.886304 [04/16/2020-17:20:00] [V] [TRT] Tactic: 130 time 1.0696 [04/16/2020-17:20:00] [V] [TRT] Tactic: 134 time 1.28819 [04/16/2020-17:20:00] [V] [TRT] Tactic: 136 time 1.08125 [04/16/2020-17:20:00] [V] [TRT] Tactic: 137 time 0.90688 [04/16/2020-17:20:00] [V] [TRT] Tactic: 139 time 0.870016 [04/16/2020-17:20:00] [V] [TRT] Tactic: 144 time 1.09542 [04/16/2020-17:20:00] [V] [TRT] Tactic: 149 time 1.58051 [04/16/2020-17:20:00] [V] [TRT] Tactic: 151 time 1.14522 [04/16/2020-17:20:00] [V] [TRT] Tactic: 152 time 0.856448 [04/16/2020-17:20:00] [V] [TRT] Tactic: 153 time 1.08512 [04/16/2020-17:20:00] [V] [TRT] Tactic: 156 time 1.03872 [04/16/2020-17:20:00] [V] [TRT] Tactic: 159 time 0.862592 [04/16/2020-17:20:00] [V] [TRT] Tactic: 162 time 0.929792 [04/16/2020-17:20:00] [V] [TRT] Tactic: 164 time 0.829824 [04/16/2020-17:20:00] [V] [TRT] Fastest Tactic: 164 Time: 0.829824 [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__25 + model/conv2d_4/Relu (CaskConvolution) [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 1825138533642645384 time 1.48275 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 2775507031594384867 time 0.606592 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 2842488832350522458 time 0.763904 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 3915320020053085238 time 1.46266 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 6448355332020552203 time 1.55216 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: 6808617066150061604 time 0.784032 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: -8060443123034038864 time 0.761632 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: -4420849921117327522 time 0.964416 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [04/16/2020-17:20:00] [V] [TRT] Tactic: -3946921629105938337 time 0.847872 [04/16/2020-17:20:00] [V] [TRT] Fastest Tactic: 2775507031594384867 Time: 0.606592 [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__25 + model/conv2d_4/Relu (CudaConvolution) [04/16/2020-17:20:00] [V] [TRT] Tactic: 0 time 1.70672 [04/16/2020-17:20:00] [V] [TRT] Tactic: 1 time 1.01616 [04/16/2020-17:20:00] [V] [TRT] Tactic: 2 skipped. Scratch requested: 150994944, available: 16777216 [04/16/2020-17:20:00] [V] [TRT] Tactic: 5 skipped. Scratch requested: 18989056, available: 16777216 [04/16/2020-17:20:00] [V] [TRT] Tactic: 6 time 0.861536 [04/16/2020-17:20:00] [V] [TRT] Fastest Tactic: 6 Time: 0.861536 [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__25 + model/conv2d_4/Relu (CudaDepthwiseConvolution) [04/16/2020-17:20:00] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:00] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2775507031594384867 [04/16/2020-17:20:00] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:00] [V] [TRT] [04/16/2020-17:20:00] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,4194304) -> Float(1,256,65536,2097152) *************** [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__26 + model/conv2d_5/Relu (LegacySASSConvolution) [04/16/2020-17:20:00] [V] [TRT] LegacySASSConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:00] [V] [TRT] --------------- Timing Runner: Conv__26 + model/conv2d_5/Relu (FusedConvActConvolution) [04/16/2020-17:20:00] [V] [TRT] Tactic: 7 time 0.49152 [04/16/2020-17:20:00] [V] [TRT] Tactic: 10 time 0.754368 [04/16/2020-17:20:00] [V] [TRT] Tactic: 14 time 0.723648 [04/16/2020-17:20:00] [V] [TRT] Tactic: 15 time 0.412128 [04/16/2020-17:20:00] [V] [TRT] Tactic: 25 time 0.410016 [04/16/2020-17:20:00] [V] [TRT] Tactic: 26 time 0.903168 [04/16/2020-17:20:00] [V] [TRT] Tactic: 29 time 0.786432 [04/16/2020-17:20:00] [V] [TRT] Tactic: 30 time 0.865696 [04/16/2020-17:20:00] [V] [TRT] Tactic: 33 time 0.448448 [04/16/2020-17:20:00] [V] [TRT] Tactic: 36 time 0.479232 [04/16/2020-17:20:00] [V] [TRT] Tactic: 39 time 0.498272 [04/16/2020-17:20:00] [V] [TRT] Tactic: 41 time 0.776448 [04/16/2020-17:20:01] [V] [TRT] Tactic: 42 time 1.53798 [04/16/2020-17:20:01] [V] [TRT] Tactic: 43 time 1.46291 [04/16/2020-17:20:01] [V] [TRT] Tactic: 45 time 0.417568 [04/16/2020-17:20:01] [V] [TRT] Tactic: 47 time 0.74592 [04/16/2020-17:20:01] [V] [TRT] Tactic: 52 time 1.4937 [04/16/2020-17:20:01] [V] [TRT] Tactic: 54 time 0.423936 [04/16/2020-17:20:01] [V] [TRT] Tactic: 56 time 1.49309 [04/16/2020-17:20:01] [V] [TRT] Tactic: 66 time 0.901728 [04/16/2020-17:20:01] [V] [TRT] Tactic: 76 time 0.442592 [04/16/2020-17:20:01] [V] [TRT] Tactic: 90 time 0.870912 [04/16/2020-17:20:01] [V] [TRT] Tactic: 93 time 0.503808 [04/16/2020-17:20:01] [V] [TRT] Tactic: 98 time 0.45184 [04/16/2020-17:20:01] [V] [TRT] Tactic: 104 time 0.843456 [04/16/2020-17:20:01] [V] [TRT] Tactic: 110 time 0.851968 [04/16/2020-17:20:01] [V] [TRT] Tactic: 119 time 0.470592 [04/16/2020-17:20:01] [V] [TRT] Tactic: 121 time 0.751168 [04/16/2020-17:20:01] [V] [TRT] Tactic: 130 time 0.504416 [04/16/2020-17:20:01] [V] [TRT] Tactic: 134 time 0.615616 [04/16/2020-17:20:01] [V] [TRT] Tactic: 136 time 0.820832 [04/16/2020-17:20:01] [V] [TRT] Tactic: 137 time 0.428352 [04/16/2020-17:20:01] [V] [TRT] Tactic: 139 time 0.415808 [04/16/2020-17:20:01] [V] [TRT] Tactic: 144 time 0.863296 [04/16/2020-17:20:01] [V] [TRT] Tactic: 149 time 1.4465 [04/16/2020-17:20:01] [V] [TRT] Tactic: 151 time 0.550912 [04/16/2020-17:20:01] [V] [TRT] Tactic: 152 time 0.421248 [04/16/2020-17:20:01] [V] [TRT] Tactic: 153 time 0.854016 [04/16/2020-17:20:01] [V] [TRT] Tactic: 156 time 0.8176 [04/16/2020-17:20:01] [V] [TRT] Tactic: 159 time 0.414304 [04/16/2020-17:20:01] [V] [TRT] Tactic: 162 time 0.452608 [04/16/2020-17:20:01] [V] [TRT] Tactic: 164 time 0.737696 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 25 Time: 0.410016 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__26 + model/conv2d_5/Relu (CaskConvolution) [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 1825138533642645384 time 1.40288 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2775507031594384867 time 0.328096 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2842488832350522458 time 0.724 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 3915320020053085238 time 1.35578 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6448355332020552203 time 1.43811 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6808617066150061604 time 0.714752 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -8060443123034038864 time 0.72496 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -4420849921117327522 time 0.491616 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -3946921629105938337 time 0.44032 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 2775507031594384867 Time: 0.328096 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__26 + model/conv2d_5/Relu (CudaConvolution) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.820928 [04/16/2020-17:20:01] [V] [TRT] Tactic: 1 time 0.576672 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2 skipped. Scratch requested: 150994944, available: 16777216 [04/16/2020-17:20:01] [V] [TRT] Tactic: 5 time 4.4848 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6 time 0.461856 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 6 Time: 0.461856 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__26 + model/conv2d_5/Relu (CudaDepthwiseConvolution) [04/16/2020-17:20:01] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:01] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2775507031594384867 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] [04/16/2020-17:20:01] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,2097152) -> Float(1,256,65536,1048576) *************** [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__27 + model/conv2d_6/Relu (LegacySASSConvolution) [04/16/2020-17:20:01] [V] [TRT] LegacySASSConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__27 + model/conv2d_6/Relu (FusedConvActConvolution) [04/16/2020-17:20:01] [V] [TRT] Tactic: 7 time 0.223712 [04/16/2020-17:20:01] [V] [TRT] Tactic: 10 time 0.36032 [04/16/2020-17:20:01] [V] [TRT] Tactic: 14 time 0.359072 [04/16/2020-17:20:01] [V] [TRT] Tactic: 15 time 0.202752 [04/16/2020-17:20:01] [V] [TRT] Tactic: 25 time 0.202496 [04/16/2020-17:20:01] [V] [TRT] Tactic: 26 time 0.498272 [04/16/2020-17:20:01] [V] [TRT] Tactic: 29 time 0.360672 [04/16/2020-17:20:01] [V] [TRT] Tactic: 30 time 0.368576 [04/16/2020-17:20:01] [V] [TRT] Tactic: 33 time 0.198656 [04/16/2020-17:20:01] [V] [TRT] Tactic: 36 time 0.223488 [04/16/2020-17:20:01] [V] [TRT] Tactic: 39 time 0.246272 [04/16/2020-17:20:01] [V] [TRT] Tactic: 41 time 0.360032 [04/16/2020-17:20:01] [V] [TRT] Tactic: 42 time 0.722912 [04/16/2020-17:20:01] [V] [TRT] Tactic: 43 time 0.675776 [04/16/2020-17:20:01] [V] [TRT] Tactic: 45 time 0.195072 [04/16/2020-17:20:01] [V] [TRT] Tactic: 47 time 0.356128 [04/16/2020-17:20:01] [V] [TRT] Tactic: 52 time 0.68992 [04/16/2020-17:20:01] [V] [TRT] Tactic: 54 time 0.192576 [04/16/2020-17:20:01] [V] [TRT] Tactic: 56 time 0.682496 [04/16/2020-17:20:01] [V] [TRT] Tactic: 66 time 0.379744 [04/16/2020-17:20:01] [V] [TRT] Tactic: 76 time 0.198656 [04/16/2020-17:20:01] [V] [TRT] Tactic: 90 time 0.36096 [04/16/2020-17:20:01] [V] [TRT] Tactic: 93 time 0.213632 [04/16/2020-17:20:01] [V] [TRT] Tactic: 98 time 0.202752 [04/16/2020-17:20:01] [V] [TRT] Tactic: 104 time 0.362496 [04/16/2020-17:20:01] [V] [TRT] Tactic: 110 time 0.414016 [04/16/2020-17:20:01] [V] [TRT] Tactic: 119 time 0.219136 [04/16/2020-17:20:01] [V] [TRT] Tactic: 121 time 0.344544 [04/16/2020-17:20:01] [V] [TRT] Tactic: 130 time 0.219136 [04/16/2020-17:20:01] [V] [TRT] Tactic: 134 time 0.329728 [04/16/2020-17:20:01] [V] [TRT] Tactic: 136 time 0.3608 [04/16/2020-17:20:01] [V] [TRT] Tactic: 137 time 0.199328 [04/16/2020-17:20:01] [V] [TRT] Tactic: 139 time 0.192512 [04/16/2020-17:20:01] [V] [TRT] Tactic: 144 time 0.37536 [04/16/2020-17:20:01] [V] [TRT] Tactic: 149 time 0.675872 [04/16/2020-17:20:01] [V] [TRT] Tactic: 151 time 0.296992 [04/16/2020-17:20:01] [V] [TRT] Tactic: 152 time 0.194816 [04/16/2020-17:20:01] [V] [TRT] Tactic: 153 time 0.379104 [04/16/2020-17:20:01] [V] [TRT] Tactic: 156 time 0.354784 [04/16/2020-17:20:01] [V] [TRT] Tactic: 159 time 0.192512 [04/16/2020-17:20:01] [V] [TRT] Tactic: 162 time 0.215552 [04/16/2020-17:20:01] [V] [TRT] Tactic: 164 time 0.339968 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 139 Time: 0.192512 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__27 + model/conv2d_6/Relu (CaskConvolution) [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 1825138533642645384 time 0.647168 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2775507031594384867 time 0.1664 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2842488832350522458 time 0.334336 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 3915320020053085238 time 0.643584 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6448355332020552203 time 0.70912 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6808617066150061604 time 0.339968 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -8060443123034038864 time 0.344384 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -4420849921117327522 time 0.224896 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -3946921629105938337 time 0.2088 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 2775507031594384867 Time: 0.1664 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__27 + model/conv2d_6/Relu (CudaConvolution) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.380928 [04/16/2020-17:20:01] [V] [TRT] Tactic: 1 time 0.26016 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2 skipped. Scratch requested: 75497472, available: 16777216 [04/16/2020-17:20:01] [V] [TRT] Tactic: 5 time 1.55341 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6 time 0.221184 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 6 Time: 0.221184 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__27 + model/conv2d_6/Relu (CudaDepthwiseConvolution) [04/16/2020-17:20:01] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:01] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2775507031594384867 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] [04/16/2020-17:20:01] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,1048576) -> Float(1,256,65536,65536) *************** [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__28 (LegacySASSConvolution) [04/16/2020-17:20:01] [V] [TRT] LegacySASSConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__28 (FusedConvActConvolution) [04/16/2020-17:20:01] [V] [TRT] Tactic: 7 time 0.100864 [04/16/2020-17:20:01] [V] [TRT] Tactic: 15 time 0.102464 [04/16/2020-17:20:01] [V] [TRT] Tactic: 25 time 0.102752 [04/16/2020-17:20:01] [V] [TRT] Tactic: 33 time 0.104288 [04/16/2020-17:20:01] [V] [TRT] Tactic: 36 time 0.125152 [04/16/2020-17:20:01] [V] [TRT] Tactic: 39 time 0.13568 [04/16/2020-17:20:01] [V] [TRT] Tactic: 45 time 0.1024 [04/16/2020-17:20:01] [V] [TRT] Tactic: 54 time 0.100416 [04/16/2020-17:20:01] [V] [TRT] Tactic: 76 time 0.10496 [04/16/2020-17:20:01] [V] [TRT] Tactic: 93 time 0.102592 [04/16/2020-17:20:01] [V] [TRT] Tactic: 98 time 0.106496 [04/16/2020-17:20:01] [V] [TRT] Tactic: 119 time 0.121312 [04/16/2020-17:20:01] [V] [TRT] Tactic: 130 time 0.106944 [04/16/2020-17:20:01] [V] [TRT] Tactic: 134 time 0.200288 [04/16/2020-17:20:01] [V] [TRT] Tactic: 137 time 0.106976 [04/16/2020-17:20:01] [V] [TRT] Tactic: 139 time 0.100416 [04/16/2020-17:20:01] [V] [TRT] Tactic: 151 time 0.182272 [04/16/2020-17:20:01] [V] [TRT] Tactic: 152 time 0.102912 [04/16/2020-17:20:01] [V] [TRT] Tactic: 159 time 0.10432 [04/16/2020-17:20:01] [V] [TRT] Tactic: 162 time 0.119296 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 54 Time: 0.100416 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__28 (CaskConvolution) [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 1825138533642645384 time 0.339936 [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2775507031594384867 time 0.108352 [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2842488832350522458 time 0.176128 [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 3915320020053085238 time 0.335872 [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6448355332020552203 time 0.39312 [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6808617066150061604 time 0.177952 [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -8060443123034038864 time 0.180128 [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -4420849921117327522 time 0.11264 [04/16/2020-17:20:01] [V] [TRT] Conv__28 (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Tactic: -3946921629105938337 time 0.10672 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: -3946921629105938337 Time: 0.10672 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__28 (CudaConvolution) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.172032 [04/16/2020-17:20:01] [V] [TRT] Tactic: 1 time 0.112384 [04/16/2020-17:20:01] [V] [TRT] Tactic: 2 skipped. Scratch requested: 37748736, available: 16777216 [04/16/2020-17:20:01] [V] [TRT] Tactic: 5 time 1.27763 [04/16/2020-17:20:01] [V] [TRT] Tactic: 6 time 0.111104 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 6 Time: 0.111104 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: Conv__28 (CudaDepthwiseConvolution) [04/16/2020-17:20:01] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [04/16/2020-17:20:01] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: FusedConvActConvolution Tactic: 54 [04/16/2020-17:20:01] [V] [TRT] [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: (Reformat) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.0528 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 0 Time: 0.0528 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: (Reformat) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.051712 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 0 Time: 0.051712 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: (Reformat) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.02304 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 0 Time: 0.02304 [04/16/2020-17:20:01] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536,65536) -> Float(1,1,256,65536) *************** [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: model/conv2d_7/Conv2D__20 (Shuffle) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.004256 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 0 Time: 0.004256 [04/16/2020-17:20:01] [V] [TRT] *************** Autotuning format combination: Float(1,256,65536:32,65536) -> Float(1,1,256:32,2048) *************** [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: model/conv2d_7/Conv2D__20 (Shuffle) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.024576 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 0 Time: 0.024576 [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: (Reformat) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 time 0.006656 [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 0 Time: 0.006656 [04/16/2020-17:20:01] [V] [TRT] *************** Autotuning format combination: Float(1,1,256,65536) -> Float(1,1,256,65536) *************** [04/16/2020-17:20:01] [V] [TRT] --------------- Timing Runner: model/conv2d_7/Sigmoid (Activation) [04/16/2020-17:20:01] [V] [TRT] Tactic: 0 is the only option, timing skipped [04/16/2020-17:20:01] [V] [TRT] Fastest Tactic: 0 Time: 0 [04/16/2020-17:20:01] [V] [TRT] Formats and tactics selection completed in 3.43894 seconds. [04/16/2020-17:20:01] [V] [TRT] After reformat layers: 11 layers [04/16/2020-17:20:01] [V] [TRT] Block size 33554432 [04/16/2020-17:20:01] [V] [TRT] Block size 33554432 [04/16/2020-17:20:01] [V] [TRT] Block size 16777216 [04/16/2020-17:20:01] [V] [TRT] Block size 0 [04/16/2020-17:20:01] [V] [TRT] Total Activation Memory: 83886080 [04/16/2020-17:20:01] [I] [TRT] Detected 1 inputs and 1 output network tensors. [04/16/2020-17:20:01] [V] [TRT] Conv__21 + model/conv2d/Relu (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1 [04/16/2020-17:20:01] [V] [TRT] Conv__22 + model/conv2d_1/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Conv__23 + model/conv2d_2/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Conv__24 + model/conv2d_3/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Conv__25 + model/conv2d_4/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Conv__26 + model/conv2d_5/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Conv__27 + model/conv2d_6/Relu (scudnn_winograd) Set Tactic Name: volta_scudnn_winograd_128x128_ldg1_ldg4_relu_tile148t_nt_v1 [04/16/2020-17:20:01] [V] [TRT] Engine generation completed in 4.78355 seconds. [04/16/2020-17:20:01] [V] [TRT] Engine Layer Information: [04/16/2020-17:20:01] [V] [TRT] Layer(Shuffle): model/conv2d/Conv2D__5, Tactic: 0, x:0[Float(256,256,3)] -> model/conv2d/Conv2D__5:0[Float(3,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(scudnn): Conv__21 + model/conv2d/Relu, Tactic: 6808617066150061604, model/conv2d/Conv2D__5:0[Float(3,256,256)] -> model/conv2d/Relu:0[Float(64,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(scudnn_winograd): Conv__22 + model/conv2d_1/Relu, Tactic: 2775507031594384867, model/conv2d/Relu:0[Float(64,256,256)] -> model/conv2d_1/Relu:0[Float(128,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(scudnn_winograd): Conv__23 + model/conv2d_2/Relu, Tactic: 2775507031594384867, model/conv2d_1/Relu:0[Float(128,256,256)] -> model/conv2d_2/Relu:0[Float(128,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(scudnn_winograd): Conv__24 + model/conv2d_3/Relu, Tactic: 2775507031594384867, model/conv2d_2/Relu:0[Float(128,256,256)] -> model/conv2d_3/Relu:0[Float(64,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(scudnn_winograd): Conv__25 + model/conv2d_4/Relu, Tactic: 2775507031594384867, model/conv2d_3/Relu:0[Float(64,256,256)] -> model/conv2d_4/Relu:0[Float(64,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(scudnn_winograd): Conv__26 + model/conv2d_5/Relu, Tactic: 2775507031594384867, model/conv2d_4/Relu:0[Float(64,256,256)] -> model/conv2d_5/Relu:0[Float(32,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(scudnn_winograd): Conv__27 + model/conv2d_6/Relu, Tactic: 2775507031594384867, model/conv2d_5/Relu:0[Float(32,256,256)] -> model/conv2d_6/Relu:0[Float(16,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(FusedConvActDirect): Conv__28, Tactic: 54, model/conv2d_6/Relu:0[Float(16,256,256)] -> Conv__28:0[Float(1,256,256)] [04/16/2020-17:20:01] [V] [TRT] Layer(Activation): model/conv2d_7/Sigmoid, Tactic: 0, model/conv2d_7/Conv2D__20:0[Float(256,256,1)] -> Identity:0[Float(256,256,1)] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:01] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:02] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:03] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [04/16/2020-17:20:04] [I] Warmup completed 29 queries over 200 ms [04/16/2020-17:20:04] [I] Timing trace has 409 queries over 3.02283 s [04/16/2020-17:20:04] [I] Trace averages of 10 runs: [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.21075 ms - Host latency: 7.40487 ms (end to end 14.3835 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.15258 ms - Host latency: 7.33177 ms (end to end 14.251 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.31736 ms - Host latency: 7.49698 ms (end to end 14.558 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.21465 ms - Host latency: 7.39471 ms (end to end 14.3797 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.34578 ms - Host latency: 7.52952 ms (end to end 14.6334 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.30971 ms - Host latency: 7.48977 ms (end to end 14.2462 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.31421 ms - Host latency: 7.49335 ms (end to end 14.5731 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.23838 ms - Host latency: 7.41831 ms (end to end 14.4207 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.45681 ms - Host latency: 7.65441 ms (end to end 14.7938 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 8.79316 ms - Host latency: 8.97216 ms (end to end 17.5294 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.35504 ms - Host latency: 7.54354 ms (end to end 14.7206 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.24847 ms - Host latency: 7.42831 ms (end to end 14.4192 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.28058 ms - Host latency: 7.46344 ms (end to end 14.523 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.19873 ms - Host latency: 7.37758 ms (end to end 13.7964 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.31703 ms - Host latency: 7.49788 ms (end to end 14.5914 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.24381 ms - Host latency: 7.42415 ms (end to end 14.4553 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.30476 ms - Host latency: 7.49374 ms (end to end 14.5377 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.29291 ms - Host latency: 7.47542 ms (end to end 14.5332 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.21696 ms - Host latency: 7.39562 ms (end to end 14.3617 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.30966 ms - Host latency: 7.48998 ms (end to end 14.5677 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.27642 ms - Host latency: 7.45736 ms (end to end 14.478 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.4176 ms - Host latency: 7.60219 ms (end to end 14.7771 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.70538 ms - Host latency: 7.88503 ms (end to end 15.3343 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.58495 ms - Host latency: 7.76975 ms (end to end 15.1434 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.44678 ms - Host latency: 7.63125 ms (end to end 14.8503 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.32954 ms - Host latency: 7.50752 ms (end to end 14.5992 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.29163 ms - Host latency: 7.47495 ms (end to end 14.5343 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.32466 ms - Host latency: 7.50876 ms (end to end 14.5729 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.29221 ms - Host latency: 7.47686 ms (end to end 14.5356 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.21196 ms - Host latency: 7.40002 ms (end to end 14.3586 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.35603 ms - Host latency: 7.54436 ms (end to end 14.6489 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.29658 ms - Host latency: 7.48569 ms (end to end 14.5486 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.23459 ms - Host latency: 7.42275 ms (end to end 14.4045 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.3613 ms - Host latency: 7.54543 ms (end to end 14.6774 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.38955 ms - Host latency: 7.57607 ms (end to end 14.7063 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.48752 ms - Host latency: 7.67263 ms (end to end 14.9197 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.55789 ms - Host latency: 7.74243 ms (end to end 15.0448 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.46082 ms - Host latency: 7.64648 ms (end to end 14.9113 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.35601 ms - Host latency: 7.54204 ms (end to end 14.6627 ms) [04/16/2020-17:20:04] [I] Average on 10 runs - GPU latency: 7.30955 ms - Host latency: 7.50366 ms (end to end 14.5322 ms) [04/16/2020-17:20:04] [I] Host latency [04/16/2020-17:20:04] [I] min: 7.26727 ms (end to end 9.31958 ms) [04/16/2020-17:20:04] [I] max: 10.249 ms (end to end 19.9075 ms) [04/16/2020-17:20:04] [I] mean: 7.55509 ms (end to end 14.665 ms) [04/16/2020-17:20:04] [I] median: 7.5061 ms (end to end 14.5858 ms) [04/16/2020-17:20:04] [I] percentile: 8.99866 ms at 99% (end to end 17.6307 ms at 99%) [04/16/2020-17:20:04] [I] throughput: 135.304 qps [04/16/2020-17:20:04] [I] walltime: 3.02283 s [04/16/2020-17:20:04] [I] GPU Compute [04/16/2020-17:20:04] [I] min: 7.0878 ms [04/16/2020-17:20:04] [I] max: 10.069 ms [04/16/2020-17:20:04] [I] mean: 7.37101 ms [04/16/2020-17:20:04] [I] median: 7.3252 ms [04/16/2020-17:20:04] [I] percentile: 8.82092 ms at 99% [04/16/2020-17:20:04] [I] total compute time: 3.01474 s &&&& PASSED TensorRT.trtexec # trtexec --onnx=model_fcn.onnx --verbose