---------------------------------------------------------------- [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::Proposal version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::Split version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1 [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:202: Adding network input: 52 with dtype: float32, dimensions: (1, 24, 224, 768) [11/18/2021-10:51:24] [V] [TRT] ImporterContext.hpp:120: Registering tensor: 52 for ONNX tensor: 52 [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv_seg_1.2.weight [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:90: Importing initializer: conv_seg_1.2.bias [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:103: Parsing node: [Conv] [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:119: Searching for input: 52 [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv_seg_1.2.weight [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:119: Searching for input: conv_seg_1.2.bias [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:125: [Conv] inputs: [52 -> (1, 24, 224, 768)], [conv_seg_1.2.weight -> (24, 24, 3, 3)], [conv_seg_1.2.bias -> (24)], [11/18/2021-10:51:24] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 24, 224, 768) [11/18/2021-10:51:24] [V] [TRT] ImporterContext.hpp:154: Registering layer: node_of_53 for ONNX node: node_of_53 [11/18/2021-10:51:24] [V] [TRT] builtin_op_importers.cpp:539: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 24 [11/18/2021-10:51:24] [V] [TRT] builtin_op_importers.cpp:540: Convolution output dimensions: (1, 24, 224, 768) [11/18/2021-10:51:24] [V] [TRT] ImporterContext.hpp:120: Registering tensor: 53_0 for ONNX tensor: 53 [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:179: [Conv] outputs: [53 -> (1, 24, 224, 768)], [11/18/2021-10:51:24] [V] [TRT] ModelImporter.cpp:510: Marking 53_0 as output: 53 ----- Parsing of ONNX model model_crash_during_inference.onnx is Done ---- [11/18/2021-10:51:24] [V] [TRT] Applying generic optimizations to the graph for inference. [11/18/2021-10:51:24] [V] [TRT] Original: 1 layers [11/18/2021-10:51:24] [V] [TRT] After dead-layer removal: 1 layers [11/18/2021-10:51:24] [V] [TRT] After Myelin optimization: 1 layers [11/18/2021-10:51:24] [V] [TRT] After scale fusion: 1 layers [11/18/2021-10:51:24] [V] [TRT] After vertical fusions: 1 layers [11/18/2021-10:51:24] [V] [TRT] After dupe layer removal: 1 layers [11/18/2021-10:51:24] [V] [TRT] After final dead-layer removal: 1 layers [11/18/2021-10:51:24] [V] [TRT] After tensor merging: 1 layers [11/18/2021-10:51:24] [V] [TRT] After concat removal: 1 layers [11/18/2021-10:51:24] [V] [TRT] Graph construction and optimization completed in 0.000436669 seconds. [11/18/2021-10:51:25] [V] [TRT] Constructing optimization profile number 0 [1/1]. [11/18/2021-10:51:25] [V] [TRT] --------------- Timing Runner: (Reformat) [11/18/2021-10:51:25] [V] [TRT] Tactic: 1002 time 0.05822 [11/18/2021-10:51:25] [V] [TRT] Tactic: 0 time 0.059296 [11/18/2021-10:51:25] [V] [TRT] Fastest Tactic: 1002 Time: 0.05822 [11/18/2021-10:51:25] [V] [TRT] --------------- Timing Runner: (Reformat) [11/18/2021-10:51:25] [V] [TRT] Tactic: 1002 time 0.093696 [11/18/2021-10:51:25] [V] [TRT] Tactic: 0 time 0.060264 [11/18/2021-10:51:25] [V] [TRT] Fastest Tactic: 0 Time: 0.060264 [11/18/2021-10:51:25] [V] [TRT] *************** Autotuning format combination: Float(1,768,172032,4128768) -> Float(1,768,172032,4128768) *************** [11/18/2021-10:51:25] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_small_nn_v1 [11/18/2021-10:51:25] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_small_nn_v1 [11/18/2021-10:51:25] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_xregs_large_nn_v1 [11/18/2021-10:51:25] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_xregs_large_nn_v1 [11/18/2021-10:51:25] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_relu_small_nn_v1 [11/18/2021-10:51:25] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_medium_nn_v1 [11/18/2021-10:51:25] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_medium_nn_v1 [11/18/2021-10:51:25] [V] [TRT] --------------- Timing Runner: node_of_53 (FusedConvActConvolution) [11/18/2021-10:51:25] [V] [TRT] Tactic: 524287 time 0.233332 [11/18/2021-10:51:25] [V] [TRT] Tactic: 720895 time 0.321948 [11/18/2021-10:51:25] [V] [TRT] Tactic: 983039 time 0.330652 [11/18/2021-10:51:25] [V] [TRT] Tactic: 1048575 time 0.208796 [11/18/2021-10:51:25] [V] [TRT] Tactic: 1703935 time 0.211332 [11/18/2021-10:51:25] [V] [TRT] Tactic: 1769471 time 0.481024 [11/18/2021-10:51:25] [V] [TRT] Tactic: 1966079 time 0.333448 [11/18/2021-10:51:25] [V] [TRT] Tactic: 2031615 time 0.3072 [11/18/2021-10:51:25] [V] [TRT] Tactic: 2228223 time 0.209852 [11/18/2021-10:51:25] [V] [TRT] Tactic: 2621439 time 0.235024 [11/18/2021-10:51:25] [V] [TRT] Tactic: 2752511 time 0.359068 [11/18/2021-10:51:25] [V] [TRT] Tactic: 2818047 time 0.645304 [11/18/2021-10:51:25] [V] [TRT] Tactic: 2883583 time 0.580096 [11/18/2021-10:51:25] [V] [TRT] Tactic: 3014655 time 0.175988 [11/18/2021-10:51:25] [V] [TRT] Tactic: 3145727 time 0.333696 [11/18/2021-10:51:25] [V] [TRT] Tactic: 3473407 time 0.521012 [11/18/2021-10:51:25] [V] [TRT] Tactic: 3604479 time 0.17536 [11/18/2021-10:51:25] [V] [TRT] Tactic: 3735551 time 0.466944 [11/18/2021-10:51:25] [V] [TRT] Tactic: 4390911 time 0.294784 [11/18/2021-10:51:25] [V] [TRT] Tactic: 5046271 time 0.187152 [11/18/2021-10:51:25] [V] [TRT] Tactic: 5963775 time 0.27512 [11/18/2021-10:51:25] [V] [TRT] Tactic: 6160383 time 0.203516 [11/18/2021-10:51:25] [V] [TRT] Tactic: 6488063 time 0.188928 [11/18/2021-10:51:26] [V] [TRT] Tactic: 6881279 time 0.26832 [11/18/2021-10:51:26] [V] [TRT] Tactic: 7274495 time 0.359168 [11/18/2021-10:51:26] [V] [TRT] Tactic: 7864319 time 0.202832 [11/18/2021-10:51:26] [V] [TRT] Tactic: 7995391 time 0.33818 [11/18/2021-10:51:26] [V] [TRT] Tactic: 8585215 time 0.220052 [11/18/2021-10:51:26] [V] [TRT] Tactic: 8847359 time 0.317732 [11/18/2021-10:51:26] [V] [TRT] Tactic: 8978431 time 0.270764 [11/18/2021-10:51:26] [V] [TRT] Tactic: 9043967 time 0.189196 [11/18/2021-10:51:26] [V] [TRT] Tactic: 9175039 time 0.176664 [11/18/2021-10:51:26] [V] [TRT] Tactic: 9502719 time 0.335992 [11/18/2021-10:51:26] [V] [TRT] Tactic: 9830399 time 0.496512 [11/18/2021-10:51:26] [V] [TRT] Tactic: 10027007 time 0.183176 [11/18/2021-10:51:26] [V] [TRT] Tactic: 10092543 time 0.29648 [11/18/2021-10:51:26] [V] [TRT] Tactic: 10289151 time 0.294528 [11/18/2021-10:51:26] [V] [TRT] Tactic: 10485759 time 0.183808 [11/18/2021-10:51:26] [V] [TRT] Tactic: 10682367 time 0.199808 [11/18/2021-10:51:26] [V] [TRT] Tactic: 10813439 time 0.314496 [11/18/2021-10:51:26] [V] [TRT] Fastest Tactic: 3604479 Time: 0.17536 [11/18/2021-10:51:26] [V] [TRT] --------------- Timing Runner: node_of_53 (CaskConvolution) [11/18/2021-10:51:26] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_small_nn_v1 [11/18/2021-10:51:26] [V] [TRT] Tactic: 4549827808004681195 time 0.282368 [11/18/2021-10:51:26] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_small_nn_v1 [11/18/2021-10:51:26] [V] [TRT] Tactic: 5779835512569528575 time 0.502016 [11/18/2021-10:51:26] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_xregs_large_nn_v1 [11/18/2021-10:51:26] [V] [TRT] Tactic: 6053873026024413720 time 0.569472 [11/18/2021-10:51:26] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_xregs_large_nn_v1 [11/18/2021-10:51:26] [V] [TRT] Tactic: 6767548733843469815 time 0.276784 [11/18/2021-10:51:26] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_relu_small_nn_v1 [11/18/2021-10:51:26] [V] [TRT] Tactic: -6313876406580483184 time 0.164096 [11/18/2021-10:51:26] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_medium_nn_v1 [11/18/2021-10:51:26] [V] [TRT] Tactic: -1123676555321336786 time 0.505128 [11/18/2021-10:51:26] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_medium_nn_v1 [11/18/2021-10:51:26] [V] [TRT] Tactic: -701551393537224327 time 0.28754 [11/18/2021-10:51:26] [V] [TRT] Fastest Tactic: -6313876406580483184 Time: 0.164096 [11/18/2021-10:51:26] [V] [TRT] --------------- Timing Runner: node_of_53 (CudaConvolution) [11/18/2021-10:51:27] [V] [TRT] Tactic: 0 time 0.27874 [11/18/2021-10:51:27] [V] [TRT] Tactic: 1 time 0.21762 [11/18/2021-10:51:27] [V] [TRT] Tactic: 2 skipped. Scratch requested: 148635648, available: 16777216 [11/18/2021-10:51:27] [V] [TRT] Tactic: 5 time 2.84686 [11/18/2021-10:51:27] [V] [TRT] Tactic: 6 time 0.171776 [11/18/2021-10:51:27] [V] [TRT] Tactic: 56 time 0.27874 [11/18/2021-10:51:27] [V] [TRT] Tactic: 57 time 0.217664 [11/18/2021-10:51:27] [V] [TRT] Tactic: 58 skipped. Scratch requested: 148635648, available: 16777216 [11/18/2021-10:51:27] [V] [TRT] Tactic: 61 time 2.7981 [11/18/2021-10:51:27] [V] [TRT] Tactic: 62 time 0.172012 [11/18/2021-10:51:27] [V] [TRT] Tactic: 112 time 0.278704 [11/18/2021-10:51:27] [V] [TRT] Tactic: 113 time 0.217904 [11/18/2021-10:51:27] [V] [TRT] Tactic: 114 skipped. Scratch requested: 148635648, available: 16777216 [11/18/2021-10:51:27] [V] [TRT] Tactic: 117 time 2.824 [11/18/2021-10:51:27] [V] [TRT] Tactic: 118 time 0.171904 [11/18/2021-10:51:27] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [11/18/2021-10:51:27] [V] [TRT] Fastest Tactic: 6 Time: 0.171776 [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CudaDepthwiseConvolution) [11/18/2021-10:51:27] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CublasConvolution) [11/18/2021-10:51:27] [V] [TRT] CublasConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -6313876406580483184 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_relu_small_nn_v1 [11/18/2021-10:51:27] [V] [TRT] [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_small_nn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_small_nn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_xregs_large_nn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_xregs_large_nn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_relu_small_nn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_medium_nn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_relu_medium_nn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_relu_small_nn_v1 [11/18/2021-10:51:27] [V] [TRT] *************** Autotuning format combination: Float(24,18432,1,4128768) -> Float(24,18432,1,4128768) *************** [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (FusedConvActConvolution) [11/18/2021-10:51:27] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CaskConvolution) [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 2860655430572478466 time 0.374952 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 4474630279712975759 time 0.223232 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 4479823862704990365 time 0.217752 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 4696204239951173149 time 0.374912 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 5778138195697110003 time 0.492824 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 7155825427510256858 time 0.48064 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 8918020581761223752 time 0.476196 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: -4756382386362004279 time 0.371204 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: -3855385237722507464 time 0.49908 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: -2809379259463049391 time 0.497408 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: -504296718212024303 time 0.476728 [11/18/2021-10:51:27] [V] [TRT] Fastest Tactic: 4479823862704990365 Time: 0.217752 [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CudaConvolution) [11/18/2021-10:51:27] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CudaDepthwiseConvolution) [11/18/2021-10:51:27] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CublasConvolution) [11/18/2021-10:51:27] [V] [TRT] CublasConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 4479823862704990365 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] *************** Autotuning format combination: Float(6,4608,1:4,1032192) -> Float(6,4608,1:4,1032192) *************** [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop_indexed) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_indexed_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_1x3x3_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (FusedConvActConvolution) [11/18/2021-10:51:27] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CaskConvolution) [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 2860655430572478466 time 0.37468 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 4474630279712975759 time 0.22336 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 4479823862704990365 time 0.217856 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 4696204239951173149 time 0.37504 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 5778138195697110003 time 0.492728 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop_indexed) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_indexed_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] Tactic: 6642501658177097719 time 0.500908 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] Tactic: 6670616668161295401 time 0.497792 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 7155825427510256858 time 0.480512 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: 8918020581761223752 time 0.476672 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_1x3x3_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] Tactic: -8844488725130067278 time 0.476288 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: -4756382386362004279 time 0.371044 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: -3855385237722507464 time 0.498708 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: -2809379259463049391 time 0.497488 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] Tactic: -504296718212024303 time 0.476672 [11/18/2021-10:51:27] [V] [TRT] Fastest Tactic: 4479823862704990365 Time: 0.217856 [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CudaConvolution) [11/18/2021-10:51:27] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CudaDepthwiseConvolution) [11/18/2021-10:51:27] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: node_of_53 (CublasConvolution) [11/18/2021-10:51:27] [V] [TRT] CublasConvolution has no valid tactics for this config, skipping [11/18/2021-10:51:27] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 4479823862704990365 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop_indexed) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_indexed_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (fprop) Set Tactic Name: trt_ampere_xmma_implicit_gemm_f32f32f32_nhwc_nhwc_fprop_1x3x3_128x128x16x1_16x8x8_4stages_tf32 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x64_sliced1x2_ldg4_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_large_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_relu_exp_medium_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1 [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: (Reformat) [11/18/2021-10:51:27] [V] [TRT] Tactic: 1002 time 0.054784 [11/18/2021-10:51:27] [V] [TRT] Tactic: 0 time 0.214084 [11/18/2021-10:51:27] [V] [TRT] Fastest Tactic: 1002 Time: 0.054784 [11/18/2021-10:51:27] [V] [TRT] --------------- Timing Runner: (Reformat) [11/18/2021-10:51:27] [V] [TRT] Tactic: 1002 time 0.1056 [11/18/2021-10:51:27] [V] [TRT] Tactic: 0 time 0.214036 [11/18/2021-10:51:27] [V] [TRT] Fastest Tactic: 1002 Time: 0.1056 [11/18/2021-10:51:27] [V] [TRT] Formats and tactics selection completed in 2.34765 seconds. [11/18/2021-10:51:27] [V] [TRT] After reformat layers: 1 layers [11/18/2021-10:51:27] [V] [TRT] Block size 16777216 [11/18/2021-10:51:27] [V] [TRT] Total Activation Memory: 16777216 [11/18/2021-10:51:27] [I] [TRT] Detected 1 inputs and 1 output network tensors. [11/18/2021-10:51:27] [V] [TRT] node_of_53 (scudnn) Set Tactic Name: ampere_scudnn_128x32_relu_small_nn_v1 [11/18/2021-10:51:27] [V] [TRT] Layer: node_of_53 Weights: 0 HostPersistent: 1664 DevicePersistent: 1053184 [11/18/2021-10:51:27] [V] [TRT] Total Host Persistent Memory: 1664 [11/18/2021-10:51:27] [V] [TRT] Total Device Persistent Memory: 1053184 [11/18/2021-10:51:27] [V] [TRT] Total Weight Memory: 0 [11/18/2021-10:51:27] [V] [TRT] Engine generation completed in 3.10626 seconds. [11/18/2021-10:51:27] [V] [TRT] Builder timing cache: created 7 entries, 0 hit(s) [11/18/2021-10:51:27] [V] [TRT] Engine Layer Information: [11/18/2021-10:51:27] [V] [TRT] Layer(scudnn): node_of_53, Tactic: -6313876406580483184, 52[Float(24,224,768)] -> 53[Float(24,224,768)] [11/18/2021-10:51:27] [I] Engine built in 3.38253 sec. [11/18/2021-10:51:27] [V] [TRT] Allocated persistent device memory of size 1053184 [11/18/2021-10:51:27] [V] [TRT] Allocated activation device memory of size 0 [11/18/2021-10:51:27] [V] [TRT] Assigning persistent memory blocks for various profiles [11/18/2021-10:51:27] [I] Starting inference