Tensorrt select algorithm tactic?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I use TensorRT python api build 3 layer network, and why tensorrt not select the best tactic for (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] , the best tactic (i8816cudnn) latency is 0.54 ms while in fact it select the icudnn, its latency is 0.73 ms

Environment

TensorRT Version: 7.1.3.0-1+cuda10.2
GPU Type: Jetson Xavier
Nvidia Driver Version: 10.2
CUDA Version: 10.2
CUDNN Version: 8.0.0.180-1
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): python3.6
TensorFlow Version (if applicable): N/A
PyTorch Version (if applicable): N/A
Baremetal or Container (if container which image + tag):

Relevant Files

icudnn kernel

截屏2020-10-19 下午2.39.17
截屏2020-10-19 下午2.39.172532×762 297 KB

i8816cudnn kernel

截屏2020-10-19 下午2.38.56
截屏2020-10-19 下午2.38.562814×1066 650 KB

Steps To Reproduce

final select kernel

截屏2020-10-19 下午2.39.27
截屏2020-10-19 下午2.39.272560×422 150 KB

log verbose

[TensorRT] VERBOSE: Original: 6 layers
[TensorRT] VERBOSE: After dead-layer removal: 4 layers
[TensorRT] VERBOSE: After scale fusion: 4 layers
[TensorRT] VERBOSE: After vertical fusions: 4 layers
[TensorRT] VERBOSE: After final dead-layer removal: 4 layers
[TensorRT] VERBOSE: After concat removal: 4 layers
[TensorRT] VERBOSE: After tensor merging: 4 layers
[TensorRT] VERBOSE: Constructing optimization profile number 0 [1/1].
[TensorRT] VERBOSE: *************** Autotuning format combination: Float(1,1,18000,2304000,2304000) → Float(1,1,18000,2304000,2304000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 1825138533642645384 time 166.087
[TensorRT] VERBOSE: Tactic: 1825138533642645384 A valid calibration tactic is found. Rest of the timing is skipped.
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 1825138533642645384
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Float(1,1,18000,2304000,2304000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 1754569683116234317 time 3.66944
[TensorRT] VERBOSE: Tactic: 1754569683116234317 A valid calibration tactic is found. Rest of the timing is skipped.
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 1754569683116234317
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Float(1,1,18000,1152000,1152000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 3) [Activation] (Activation)
[TensorRT] VERBOSE: Tactic: 0 is the only option, timing skipped
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0
[TensorRT] VERBOSE: *************** Autotuning format combination: Float(1,1,18000,1152000,1152000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 1754569683116234317 time 2.04813
[TensorRT] VERBOSE: Tactic: 1754569683116234317 A valid calibration tactic is found. Rest of the timing is skipped.
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 1754569683116234317
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Formats and tactics selection completed in 1.0126 seconds.
[TensorRT] VERBOSE: After reformat layers: 4 layers
[TensorRT] VERBOSE: Block size 4294967296
[TensorRT] VERBOSE: Block size 73728000
[TensorRT] VERBOSE: Block size 36864000
[TensorRT] VERBOSE: Total Activation Memory: 4405559296
[TensorRT] INFO: Detected 1 inputs and 1 output network tensors.
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Layer: (Unnamed Layer* 0) [Convolution] Weights: 0 HostPersistent: 2176 DevicePersistent: 3319808
[TensorRT] VERBOSE: Layer: (Unnamed Layer* 2) [Convolution] Weights: 0 HostPersistent: 3200 DevicePersistent: 141312
[TensorRT] VERBOSE: Layer: (Unnamed Layer* 3) [Activation] Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: (Unnamed Layer* 4) [Convolution] Weights: 0 HostPersistent: 3200 DevicePersistent: 124928
[TensorRT] VERBOSE: Total Host Persistent Memory: 8576
[TensorRT] VERBOSE: Total Device Persistent Memory: 3586048
[TensorRT] VERBOSE: Total Weight Memory: 0
[TensorRT] VERBOSE: Builder timing cache: created 3 entries, 0 hit(s)
[TensorRT] VERBOSE: Engine generation completed in 3.08988 seconds.
[TensorRT] VERBOSE: Calculating Maxima
[TensorRT] INFO: Starting Calibration with batch size 8.
[TensorRT] INFO: Calibrated batch 0 in 0.425352 seconds.
[TensorRT] INFO: Calibrated batch 1 in 0.420408 seconds.
[TensorRT] INFO: Calibrated batch 2 in 0.421317 seconds.
[TensorRT] INFO: Post Processing Calibration data in 0.126398 seconds.
[TensorRT] INFO: Calibration completed in 4.49954 seconds.
[TensorRT] VERBOSE: INT8 Inference Tensor Scales: input range [-1.00024,1.00024]
[TensorRT] VERBOSE: INT8 Inference Tensor Scales: (Unnamed Layer* 0) [Convolution]_output range [-897.219,897.219]
[TensorRT] VERBOSE: INT8 Inference Tensor Scales: (Unnamed Layer* 2) [Convolution]_output range [-114845,114845]
[TensorRT] VERBOSE: INT8 Inference Tensor Scales: (Unnamed Layer* 3) [Activation]_output range [-114845,114845]
[TensorRT] VERBOSE: INT8 Inference Tensor Scales: (Unnamed Layer* 4) [Convolution]_output range [-7.35008e+06,7.35008e+06]
[TensorRT] VERBOSE: Original: 6 layers
[TensorRT] VERBOSE: After dead-layer removal: 4 layers
[TensorRT] VERBOSE: After Myelin optimization: 4 layers
[TensorRT] VERBOSE: After scale fusion: 4 layers
[TensorRT] VERBOSE: Fusing (Unnamed Layer* 2) [Convolution] with (Unnamed Layer* 3) [Activation]
[TensorRT] VERBOSE: After vertical fusions: 3 layers
[TensorRT] VERBOSE: After final dead-layer removal: 3 layers
[TensorRT] VERBOSE: After tensor merging: 3 layers
[TensorRT] VERBOSE: After concat removal: 3 layers
[TensorRT] INFO: Writing Calibration Cache for calibrator: TRT-7103-EntropyCalibration2
[TensorRT] VERBOSE: Configuring builder for Int8 Mode completed in 4.50267 seconds.
[TensorRT] VERBOSE: Graph construction and optimization completed in 4.50296 seconds.
[TensorRT] INFO:
[TensorRT] INFO: --------------- Layers running on DLA:
[TensorRT] INFO:
[TensorRT] INFO: --------------- Layers running on GPU:
[TensorRT] INFO: (Unnamed Layer* 0) [Convolution], (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation], (Unnamed Layer* 4) [Convolution],
[TensorRT] VERBOSE: Constructing optimization profile number 0 [1/1].
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.533888
[TensorRT] VERBOSE: Tactic: 0 time 0.114624
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.114624
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.524256
[TensorRT] VERBOSE: Tactic: 0 time 1.07094
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.524256
[TensorRT] VERBOSE: *************** Autotuning format combination: Float(1,1,18000,2304000,2304000) → Float(1,1,18000,2304000,2304000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 1825138533642645384 time 21.211
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: Tactic: 2842488832350522458 time 21.3847
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: Tactic: 6448355332020552203 time 21.4836
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: -8060443123034038864 time 21.902
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: -4420849921117327522 time 21.9039
[TensorRT] VERBOSE: Fastest Tactic: 1825138533642645384 Time: 21.211
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: Tactic: 0 time 26.3857
[TensorRT] VERBOSE: Tactic: 2 time 29.5598
[TensorRT] VERBOSE: Tactic: 5 time 485.517
[TensorRT] VERBOSE: Tactic: 57 time 22.5912
[TensorRT] VERBOSE: Fastest Tactic: 57 Time: 22.5912
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 1825138533642645384
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:4,576000,576000) → Float(1,1,18000,2304000,2304000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 892787096507693407 time 5.3456
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 1204440019753223942 time 5.70883
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 2057291331119027912 time 5.50803
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 5623454780463195174 time 5.98013
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: Tactic: 8930254200803946944 time 5.41168
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: Tactic: -1228371230285617088 time 5.49232
[TensorRT] VERBOSE: Fastest Tactic: 892787096507693407 Time: 5.3456
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 892787096507693407
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:4,576000,576000) → Int8(1,1,18000:4,576000,576000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 4438325421691896755 time 5.49306
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 4934335053031119367 time 5.6441
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 6797040896965118050 time 5.31875
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 8006952294591770973 time 5.84842
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: Tactic: -5026383765466876607 time 5.43536
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: Tactic: -1370999262391786833 time 5.36595
[TensorRT] VERBOSE: Fastest Tactic: 6797040896965118050 Time: 5.31875
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 6797040896965118050
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_xregs_large_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:4,576000,576000) → Int8(1,1,18000:32,72000,72000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_xregs_large_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_xregs_large_c32_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_c32_nn_v1
[TensorRT] VERBOSE: Tactic: 1213457772632185722 time 5.74682
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: Tactic: 1713441381477652893 time 5.77066
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_xregs_large_c32_nn_v1
[TensorRT] VERBOSE: Tactic: 7125598890155666458 time 5.36691
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -3566249366964946311 time 5.49235
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -2002418013575043687 time 5.32246
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_xregs_large_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -1659631603542281459 time 5.43981
[TensorRT] VERBOSE: Fastest Tactic: -2002418013575043687 Time: 5.32246
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -2002418013575043687
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_c32_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_xregs_large_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_xregs_large_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:32,72000,72000) → Float(1,1,18000:32,72000,72000) ***************
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:32,72000,72000) → Int8(1,1,18000:32,72000,72000) ***************
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 0) [Convolution] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.907296
[TensorRT] VERBOSE: Tactic: 0 time 0.965408
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.907296
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.544064
[TensorRT] VERBOSE: Tactic: 0 time 0.95632
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.544064
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.522496
[TensorRT] VERBOSE: Tactic: 0 time 1.92362
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.522496
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.539008
[TensorRT] VERBOSE: Tactic: 0 time 0.139968
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.139968
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.525024
[TensorRT] VERBOSE: Tactic: 0 time 0.090464
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.090464
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.538976
[TensorRT] VERBOSE: Tactic: 0 time 0.550976
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.538976
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.512192
[TensorRT] VERBOSE: Tactic: 0 time 0.320704
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.320704
[TensorRT] VERBOSE: *************** Autotuning format combination: Float(1,1,18000,2304000,2304000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 1754569683116234317 time 0.474624
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 1825138533642645384 time 0.47424
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 2733356012094739613 time 0.301664
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: 3915320020053085238 time 0.473856
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: 6808617066150061604 time 0.25056
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 9091006216302412844 time 0.245408
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: -8060443123034038864 time 0.257856
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: -4420849921117327522 time 0.341664
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: -3946921629105938337 time 0.314816
[TensorRT] VERBOSE: Fastest Tactic: 9091006216302412844 Time: 0.245408
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaConvolution)
[TensorRT] VERBOSE: Tactic: 0 time 0.565344
[TensorRT] VERBOSE: Tactic: 2 time 0.788736
[TensorRT] VERBOSE: Tactic: 5 time 7.4072
[TensorRT] VERBOSE: Tactic: 57 time 1.01741
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.565344
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 9091006216302412844
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:4,576000,576000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 892787096507693407 time 0.16272
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 1204440019753223942 time 0.140672
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: 1659301557717208403 time 0.11376
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 2057291331119027912 time 0.092928
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: Tactic: 3275977259705528576 time 0.137536
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 5623454780463195174 time 0.124352
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: -9204333525109552344 time 0.091232
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -7924103240988931433 time 0.095296
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -7489650117016530013 time 0.14352
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: -4973811344878172338 time 0.16288
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -3908975881807046106 time 0.11728
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -1765942417666394360 time 0.162784
[TensorRT] VERBOSE: Fastest Tactic: -9204333525109552344 Time: 0.091232
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -9204333525109552344
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:4,576000,576000) → Int8(1,1,18000:4,288000,288000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 3145259992339075399 time 0.073312
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 4000990898022781625 time 0.139072
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 4438325421691896755 time 0.076096
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: 4581732244273465060 time 0.076512
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 4934335053031119367 time 0.107328
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 6797040896965118050 time 0.1368
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 8006952294591770973 time 0.093632
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 8097855305881829878 time 0.097088
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: -7210942453088153035 time 0.139328
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: -6282183216199417697 time 0.089568
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: Tactic: -5016725782072253841 time 0.095616
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -1543391652455542154 time 0.094016
[TensorRT] VERBOSE: Fastest Tactic: 3145259992339075399 Time: 0.073312
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 3145259992339075399
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:4,576000,576000) → Int8(1,1,18000:32,36000,36000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_small_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_small_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_interior_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_small_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_small_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: Tactic: 1025026069226666066 time 0.135104
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_c32_nn_v1
[TensorRT] VERBOSE: Tactic: 1213457772632185722 time 0.102912
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: Tactic: 1713441381477652893 time 0.092384
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: Tactic: 2339361327868109050 time 0.0736
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_small_c32_nn_v1
[TensorRT] VERBOSE: Tactic: 8047041638267142825 time 0.093792
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_small_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -7846982807478255793 time 0.075712
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_interior_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -7686150779628967382 time 0.094912
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_small_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -6459719113600909000 time 0.099136
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_small_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -4573925292554651334 time 0.133152
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -4208188808979933945 time 0.089408
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -3566249366964946311 time 0.076032
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: Tactic: -2002418013575043687 time 0.134144
[TensorRT] VERBOSE: Fastest Tactic: 2339361327868109050 Time: 0.0736
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 2339361327868109050
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_c32_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_small_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_small_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_interior_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_xregs_small_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_small_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x32_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_c32_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_c32_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:32,72000,72000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CaskConvolution)
[TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:32,72000,72000) → Float(1,1,18000:32,36000,36000) ***************
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CaskConvolution)
[TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:32,72000,72000) → Int8(1,1,18000:32,36000,36000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_medium_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_small_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_interior_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: Tactic: 66319348402778770 time 0.080992
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: Tactic: 2271687430539765460 time 0.09296
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: Tactic: 5754467717466343388 time 0.078112
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: Tactic: 7039764449991095921 time 0.056064
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: Tactic: 7584772692956718645 time 0.094848
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_medium_nt_v1
[TensorRT] VERBOSE: Tactic: -9114895246540757312 time 0.055136
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: Tactic: -7274936339335021260 time 0.05664
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_small_nt_v1
[TensorRT] VERBOSE: Tactic: -2102888629196925141 time 0.054464
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_interior_nt_v1
[TensorRT] VERBOSE: Tactic: -1383447415429797909 time 0.054656
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: Tactic: -743032628982127825 time 0.057632
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: Tactic: -674235064782459186 time 0.092288
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: Tactic: -182858804213663094 time 0.07808
[TensorRT] VERBOSE: Fastest Tactic: -2102888629196925141 Time: 0.054464
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -2102888629196925141
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_small_nt_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_medium_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_medium_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_small_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_interior_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_interior_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x128_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_128x128_ldg16_relu_small_nt_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (i8816cudnn) Set Tactic Name: volta_int8_i8816cudnn_int8_256x64_ldg16_relu_singleBuffer_small_nt_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.283616
[TensorRT] VERBOSE: Tactic: 0 time 0.052352
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.052352
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.278816
[TensorRT] VERBOSE: Tactic: 0 time 0.489088
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.278816
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.45952
[TensorRT] VERBOSE: Tactic: 0 time 0.470208
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.45952
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.285728
[TensorRT] VERBOSE: Tactic: 0 time 0.527168
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.285728
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.277248
[TensorRT] VERBOSE: Tactic: 0 time 0.999872
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.277248
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.281696
[TensorRT] VERBOSE: Tactic: 0 time 0.07184
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.07184
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.277856
[TensorRT] VERBOSE: Tactic: 0 time 0.033632
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.033632
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.28704
[TensorRT] VERBOSE: Tactic: 0 time 0.269792
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.269792
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.27216
[TensorRT] VERBOSE: Tactic: 0 time 0.061792
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.061792
[TensorRT] VERBOSE: *************** Autotuning format combination: Float(1,1,18000,1152000,1152000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 1754569683116234317 time 0.262816
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 1825138533642645384 time 0.260672
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 2733356012094739613 time 0.169632
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: 3915320020053085238 time 0.268544
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: 6808617066150061604 time 0.14064
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: 9091006216302412844 time 0.135712
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: -8060443123034038864 time 0.14288
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: -4420849921117327522 time 0.210176
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: -3946921629105938337 time 0.174144
[TensorRT] VERBOSE: Fastest Tactic: 9091006216302412844 Time: 0.135712
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: Tactic: 0 time 0.336512
[TensorRT] VERBOSE: Tactic: 2 time 0.468832
[TensorRT] VERBOSE: Tactic: 5 time 4.39472
[TensorRT] VERBOSE: Tactic: 57 time 0.73328
[TensorRT] VERBOSE: Fastest Tactic: 0 Time: 0.336512
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: 9091006216302412844
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (scudnn) Set Tactic Name: volta_scudnn_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:4,288000,288000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 892787096507693407 time 0.132832
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 1204440019753223942 time 0.104736
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: 1659301557717208403 time 0.089664
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 2057291331119027912 time 0.087744
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: Tactic: 3275977259705528576 time 0.106144
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: Tactic: 5623454780463195174 time 0.088736
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: -9204333525109552344 time 0.090944
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -7924103240988931433 time 0.082656
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -7489650117016530013 time 0.100608
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: Tactic: -4973811344878172338 time 0.128192
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -3908975881807046106 time 0.087712
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: Tactic: -1765942417666394360 time 0.125184
[TensorRT] VERBOSE: Fastest Tactic: -7924103240988931433 Time: 0.082656
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -7924103240988931433
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE:
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_xregs_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_small_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x32_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x128_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:32,36000,36000) → Float(1,1,18000,1152000,1152000) ***************
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: *************** Autotuning format combination: Int8(1,1,18000:32,36000,36000) → Float(1,1,18000:32,36000,36000) ***************
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (FusedConvActConvolution)
[TensorRT] VERBOSE: FusedConvActConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CaskConvolution)
[TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaConvolution)
[TensorRT] VERBOSE: CudaConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaDepthwiseConvolution)
[TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping
[TensorRT] VERBOSE: --------------- Timing Runner: (Unnamed Layer* 4) [Convolution] (CudaGroupConvolution)
[TensorRT] VERBOSE: CudaGroupConvolution has no valid tactics for this config, skipping
[TensorRT] WARNING: No implementation of layer (Unnamed Layer* 4) [Convolution] obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] VERBOSE: --------------- Timing Runner: (Reformat)
[TensorRT] VERBOSE: Tactic: 1002 time 0.462592
[TensorRT] VERBOSE: Tactic: 0 time 0.46928
[TensorRT] VERBOSE: Fastest Tactic: 1002 Time: 0.462592
[TensorRT] VERBOSE: Reformatting format: [in] Float(1,1,18000,2304000,2304000), [out] Int8(1,1,18000:4,576000,576000)
[TensorRT] WARNING: No implementation obeys reformatting-free rules, at least 1 reformatting nodes are needed, now picking the fastest path instead.
[TensorRT] VERBOSE: Adding reformat layer: (Unnamed Layer* 0) [Convolution] reformatted input 0 (input) from Float(1,1,18000,2304000,2304000) to Int8(1,1,18000:4,576000,576000)
[TensorRT] VERBOSE: Formats and tactics selection completed in 6.66481 seconds.
[TensorRT] VERBOSE: After reformat layers: 4 layers
[TensorRT] VERBOSE: Block size 4294967296
[TensorRT] VERBOSE: Block size 2304000
[TensorRT] VERBOSE: Block size 2304000
[TensorRT] VERBOSE: Total Activation Memory: 4299575296
[TensorRT] INFO: Detected 1 inputs and 1 output network tensors.
[TensorRT] VERBOSE: (Unnamed Layer* 0) [Convolution] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x128_relu_medium_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] (icudnn) Set Tactic Name: volta_int8x4_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: (Unnamed Layer* 4) [Convolution] (icudnn) Set Tactic Name: volta_fp32_icudnn_int8x4_128x64_relu_interior_nn_v1
[TensorRT] VERBOSE: Layer: (Unnamed Layer* 0) [Convolution] input reformatter 0 Weights: 0 HostPersistent: 0 DevicePersistent: 0
[TensorRT] VERBOSE: Layer: (Unnamed Layer* 0) [Convolution] Weights: 0 HostPersistent: 2176 DevicePersistent: 913408
[TensorRT] VERBOSE: Layer: (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] Weights: 0 HostPersistent: 3200 DevicePersistent: 117760
[TensorRT] VERBOSE: Layer: (Unnamed Layer* 4) [Convolution] Weights: 0 HostPersistent: 3200 DevicePersistent: 113664
[TensorRT] VERBOSE: Total Host Persistent Memory: 8576
[TensorRT] VERBOSE: Total Device Persistent Memory: 1144832
[TensorRT] VERBOSE: Total Weight Memory: 0
[TensorRT] VERBOSE: Builder timing cache: created 30 entries, 23 hit(s)
[TensorRT] VERBOSE: Engine generation completed in 7.4088 seconds.
[TensorRT] VERBOSE: Engine Layer Information:
[TensorRT] VERBOSE: Layer(Reformat): (Unnamed Layer* 0) [Convolution] input reformatter 0, Tactic: 0, input[Float(1,128,18000,1)] → (Unnamed Layer* 0) [Convolution] reformatted input 0[Int8(1,128,18000,1)]
[TensorRT] VERBOSE: Layer(icudnn): (Unnamed Layer* 0) [Convolution], Tactic: 6797040896965118050, (Unnamed Layer* 0) [Convolution] reformatted input 0[Int8(1,128,18000,1)] → (Unnamed Layer* 0) [Convolution]_output[Int8(1,128,18000,1)]
[TensorRT] VERBOSE: Layer(icudnn): (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation], Tactic: 3145259992339075399, (Unnamed Layer* 0) [Convolution]_output[Int8(1,128,18000,1)] → (Unnamed Layer* 3) [Activation]_output[Int8(1,64,18000,1)]
[TensorRT] VERBOSE: Layer(icudnn): (Unnamed Layer* 4) [Convolution], Tactic: -7924103240988931433, (Unnamed Layer* 3) [Activation]_output[Int8(1,64,18000,1)] → (Unnamed Layer* 4) [Convolution]_output[Float(1,64,18000,1)]
run in int8
===== builder.platform_has_fast_int8 : True
input
size 2304000, dtype <class ‘numpy.float32’>
(Unnamed Layer* 4) [Convolution]_output
size 1152000, dtype <class ‘numpy.float32’>
run time 5.6121630859375 ms
run time 5.71154833984375 ms
Layer (Unnamed Layer* 0) [Convolution] input reformatter 0 → Latency 0.13327087998390197
Layer (Unnamed Layer* 0) [Convolution] → Latency 5.3387031679153445
Layer (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] → Latency 0.07858729606121778
Layer (Unnamed Layer* 4) [Convolution] → Latency 0.0917734399959445
idx 0 result 0.13327087998390197 profile Layer (Unnamed Layer* 0) [Convolution] input reformatter 0 → Latency 0.13327087998390197
idx 1 result 5.3387031679153445 profile Layer (Unnamed Layer* 0) [Convolution] → Latency 5.3387031679153445
idx 2 result 0.07858729606121778 profile Layer (Unnamed Layer* 2) [Convolution] + (Unnamed Layer* 3) [Activation] → Latency 0.07858729606121778
idx 3 result 0.0917734399959445 profile Layer (Unnamed Layer* 4) [Convolution] → Latency 0.0917734399959445
=== valid_idx 2 latency 0.07858729606121778

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered
2

Hi @erfeng.jef,
TensorRT optimizes for minimum timing of the whole network, possibly departing from locally greedy choices in exchange for less reformatting overhead.

Thanks

3

Can I choose a specific tactic for a layer manually?

4

Hi,
Please check the below links, as they might answer your concerns.

Thanks!

7

Thanks for your reply. But I am still confused. I am using TensorRT to deploy a network on Jetson Xavier NX. In NVDLA document, Winograd algorithm is mentioned. So I want to use this algorithm for all network layers on DLA device. Is it possible using TensorRT?