&&&& RUNNING TensorRT.trtexec # trtexec --loadEngine=resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine --fp16
[06/25/2021-18:13:46] [I] === Model Options ===
[06/25/2021-18:13:46] [I] Format: *
[06/25/2021-18:13:46] [I] Model: 
[06/25/2021-18:13:46] [I] Output:
[06/25/2021-18:13:46] [I] === Build Options ===
[06/25/2021-18:13:46] [I] Max batch: 1
[06/25/2021-18:13:46] [I] Workspace: 16 MiB
[06/25/2021-18:13:46] [I] minTiming: 1
[06/25/2021-18:13:46] [I] avgTiming: 8
[06/25/2021-18:13:46] [I] Precision: FP32+FP16
[06/25/2021-18:13:46] [I] Calibration: 
[06/25/2021-18:13:46] [I] Refit: Disabled
[06/25/2021-18:13:46] [I] Safe mode: Disabled
[06/25/2021-18:13:46] [I] Save engine: 
[06/25/2021-18:13:46] [I] Load engine: resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine
[06/25/2021-18:13:46] [I] Builder Cache: Enabled
[06/25/2021-18:13:46] [I] NVTX verbosity: 0
[06/25/2021-18:13:46] [I] Tactic sources: Using default tactic sources
[06/25/2021-18:13:46] [I] Input(s)s format: fp32:CHW
[06/25/2021-18:13:46] [I] Output(s)s format: fp32:CHW
[06/25/2021-18:13:46] [I] Input build shapes: model
[06/25/2021-18:13:46] [I] Input calibration shapes: model
[06/25/2021-18:13:46] [I] === System Options ===
[06/25/2021-18:13:46] [I] Device: 0
[06/25/2021-18:13:46] [I] DLACore: 
[06/25/2021-18:13:46] [I] Plugins:
[06/25/2021-18:13:46] [I] === Inference Options ===
[06/25/2021-18:13:46] [I] Batch: 1
[06/25/2021-18:13:46] [I] Input inference shapes: model
[06/25/2021-18:13:46] [I] Iterations: 10
[06/25/2021-18:13:46] [I] Duration: 3s (+ 200ms warm up)
[06/25/2021-18:13:46] [I] Sleep time: 0ms
[06/25/2021-18:13:46] [I] Streams: 1
[06/25/2021-18:13:46] [I] ExposeDMA: Disabled
[06/25/2021-18:13:46] [I] Data transfers: Enabled
[06/25/2021-18:13:46] [I] Spin-wait: Disabled
[06/25/2021-18:13:46] [I] Multithreading: Disabled
[06/25/2021-18:13:46] [I] CUDA Graph: Disabled
[06/25/2021-18:13:46] [I] Separate profiling: Disabled
[06/25/2021-18:13:46] [I] Skip inference: Disabled
[06/25/2021-18:13:46] [I] Inputs:
[06/25/2021-18:13:46] [I] === Reporting Options ===
[06/25/2021-18:13:46] [I] Verbose: Disabled
[06/25/2021-18:13:46] [I] Averages: 10 inferences
[06/25/2021-18:13:46] [I] Percentile: 99
[06/25/2021-18:13:46] [I] Dump refittable layers:Disabled
[06/25/2021-18:13:46] [I] Dump output: Disabled
[06/25/2021-18:13:46] [I] Profile: Disabled
[06/25/2021-18:13:46] [I] Export timing to JSON file: 
[06/25/2021-18:13:46] [I] Export output to JSON file: 
[06/25/2021-18:13:46] [I] Export profile to JSON file: 
[06/25/2021-18:13:46] [I] 
[06/25/2021-18:13:46] [I] === Device Information ===
[06/25/2021-18:13:46] [I] Selected Device: GeForce RTX 2080 SUPER
[06/25/2021-18:13:46] [I] Compute Capability: 7.5
[06/25/2021-18:13:46] [I] SMs: 48
[06/25/2021-18:13:46] [I] Compute Clock Rate: 1.815 GHz
[06/25/2021-18:13:46] [I] Device Global Memory: 7959 MiB
[06/25/2021-18:13:46] [I] Shared Memory per SM: 64 KiB
[06/25/2021-18:13:46] [I] Memory Bus Width: 256 bits (ECC disabled)
[06/25/2021-18:13:46] [I] Memory Clock Rate: 7.751 GHz
[06/25/2021-18:13:46] [I] 
[06/25/2021-18:13:48] [I] Engine loaded in 1.8451 sec.
[06/25/2021-18:13:48] [I] Starting inference
[06/25/2021-18:13:51] [I] Warmup completed 128 queries over 200 ms
[06/25/2021-18:13:51] [I] Timing trace has 2256 queries over 3.00382 s
[06/25/2021-18:13:51] [I] Trace averages of 10 runs:
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.53435 ms - Host latency: 2.07944 ms (end to end 2.97976 ms, enqueue 0.719878 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.53379 ms - Host latency: 2.07748 ms (end to end 2.97912 ms, enqueue 0.715819 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.53367 ms - Host latency: 2.07756 ms (end to end 2.98025 ms, enqueue 0.715765 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.42269 ms - Host latency: 1.96767 ms (end to end 2.77733 ms, enqueue 0.724438 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31507 ms - Host latency: 1.85983 ms (end to end 2.54297 ms, enqueue 0.706659 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31519 ms - Host latency: 1.86345 ms (end to end 2.52973 ms, enqueue 0.714557 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31565 ms - Host latency: 1.85858 ms (end to end 2.5462 ms, enqueue 0.540939 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31615 ms - Host latency: 1.86147 ms (end to end 2.53923 ms, enqueue 0.556497 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31535 ms - Host latency: 1.85949 ms (end to end 2.5476 ms, enqueue 0.541303 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31602 ms - Host latency: 1.8583 ms (end to end 2.54952 ms, enqueue 0.536121 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3144 ms - Host latency: 1.8576 ms (end to end 2.54727 ms, enqueue 0.535501 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31262 ms - Host latency: 1.85664 ms (end to end 2.54279 ms, enqueue 0.535248 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31289 ms - Host latency: 1.85498 ms (end to end 2.54261 ms, enqueue 0.532422 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31262 ms - Host latency: 1.85616 ms (end to end 2.53393 ms, enqueue 0.536536 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31039 ms - Host latency: 1.85565 ms (end to end 2.5373 ms, enqueue 0.544989 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31154 ms - Host latency: 1.85551 ms (end to end 2.5414 ms, enqueue 0.542709 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31328 ms - Host latency: 1.85631 ms (end to end 2.54395 ms, enqueue 0.53877 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31421 ms - Host latency: 1.85817 ms (end to end 2.54444 ms, enqueue 0.537668 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31158 ms - Host latency: 1.8545 ms (end to end 2.53926 ms, enqueue 0.533917 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31308 ms - Host latency: 1.85678 ms (end to end 2.53342 ms, enqueue 0.537506 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31215 ms - Host latency: 1.85588 ms (end to end 2.53863 ms, enqueue 0.53392 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31884 ms - Host latency: 1.86245 ms (end to end 2.55694 ms, enqueue 0.539081 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31822 ms - Host latency: 1.86145 ms (end to end 2.55227 ms, enqueue 0.536758 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31904 ms - Host latency: 1.86279 ms (end to end 2.48235 ms, enqueue 0.550406 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32386 ms - Host latency: 1.86716 ms (end to end 2.5683 ms, enqueue 0.532581 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32192 ms - Host latency: 1.86432 ms (end to end 2.56111 ms, enqueue 0.535602 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32544 ms - Host latency: 1.86732 ms (end to end 2.56823 ms, enqueue 0.534106 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32463 ms - Host latency: 1.86801 ms (end to end 2.56578 ms, enqueue 0.536096 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32226 ms - Host latency: 1.8636 ms (end to end 2.56186 ms, enqueue 0.531219 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32383 ms - Host latency: 1.86688 ms (end to end 2.5562 ms, enqueue 0.537073 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32194 ms - Host latency: 1.86442 ms (end to end 2.56135 ms, enqueue 0.535071 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32198 ms - Host latency: 1.86522 ms (end to end 2.56079 ms, enqueue 0.536188 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32046 ms - Host latency: 1.86218 ms (end to end 2.55547 ms, enqueue 0.533691 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32042 ms - Host latency: 1.86221 ms (end to end 2.56024 ms, enqueue 0.538153 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31961 ms - Host latency: 1.86245 ms (end to end 2.55575 ms, enqueue 0.53822 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31919 ms - Host latency: 1.86246 ms (end to end 2.55161 ms, enqueue 0.545789 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31979 ms - Host latency: 1.86216 ms (end to end 2.55695 ms, enqueue 0.553027 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31974 ms - Host latency: 1.86288 ms (end to end 2.54562 ms, enqueue 0.555493 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31999 ms - Host latency: 1.86361 ms (end to end 2.55404 ms, enqueue 0.537537 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31939 ms - Host latency: 1.86204 ms (end to end 2.5592 ms, enqueue 0.536469 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31807 ms - Host latency: 1.86102 ms (end to end 2.55114 ms, enqueue 0.537146 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31982 ms - Host latency: 1.86263 ms (end to end 2.55618 ms, enqueue 0.538672 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31845 ms - Host latency: 1.86185 ms (end to end 2.53506 ms, enqueue 0.543469 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3186 ms - Host latency: 1.85977 ms (end to end 2.52333 ms, enqueue 0.536804 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31816 ms - Host latency: 1.86021 ms (end to end 2.54211 ms, enqueue 0.544513 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31914 ms - Host latency: 1.8603 ms (end to end 2.55493 ms, enqueue 0.541412 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31777 ms - Host latency: 1.85984 ms (end to end 2.56121 ms, enqueue 0.544958 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31851 ms - Host latency: 1.86302 ms (end to end 2.55151 ms, enqueue 0.561639 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31853 ms - Host latency: 1.86278 ms (end to end 2.57311 ms, enqueue 0.546442 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31849 ms - Host latency: 1.86098 ms (end to end 2.56065 ms, enqueue 0.535767 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3177 ms - Host latency: 1.86176 ms (end to end 2.55272 ms, enqueue 0.550714 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31777 ms - Host latency: 1.8627 ms (end to end 2.54189 ms, enqueue 0.554388 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31727 ms - Host latency: 1.8621 ms (end to end 2.55109 ms, enqueue 0.544409 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31993 ms - Host latency: 1.86544 ms (end to end 2.55268 ms, enqueue 0.541071 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31693 ms - Host latency: 1.859 ms (end to end 2.55304 ms, enqueue 0.543488 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31737 ms - Host latency: 1.86093 ms (end to end 2.55251 ms, enqueue 0.548346 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31882 ms - Host latency: 1.86163 ms (end to end 2.55533 ms, enqueue 0.543793 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31813 ms - Host latency: 1.86382 ms (end to end 2.54678 ms, enqueue 0.546747 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31774 ms - Host latency: 1.8613 ms (end to end 2.55048 ms, enqueue 0.542957 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31871 ms - Host latency: 1.86316 ms (end to end 2.55003 ms, enqueue 0.543268 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31876 ms - Host latency: 1.86122 ms (end to end 2.55456 ms, enqueue 0.545642 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31848 ms - Host latency: 1.8629 ms (end to end 2.54578 ms, enqueue 0.542322 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31809 ms - Host latency: 1.86146 ms (end to end 2.55079 ms, enqueue 0.551538 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31843 ms - Host latency: 1.86483 ms (end to end 2.54194 ms, enqueue 0.573767 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31916 ms - Host latency: 1.86311 ms (end to end 2.55215 ms, enqueue 0.576355 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32075 ms - Host latency: 1.86512 ms (end to end 2.55563 ms, enqueue 0.571533 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31836 ms - Host latency: 1.86815 ms (end to end 2.5238 ms, enqueue 0.629651 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31782 ms - Host latency: 1.86652 ms (end to end 2.53118 ms, enqueue 0.521814 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31921 ms - Host latency: 1.86346 ms (end to end 2.54979 ms, enqueue 0.542554 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31829 ms - Host latency: 1.8627 ms (end to end 2.54854 ms, enqueue 0.552563 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31831 ms - Host latency: 1.86289 ms (end to end 2.55527 ms, enqueue 0.543762 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3179 ms - Host latency: 1.86198 ms (end to end 2.55024 ms, enqueue 0.542419 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31904 ms - Host latency: 1.8634 ms (end to end 2.55104 ms, enqueue 0.545691 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31887 ms - Host latency: 1.86274 ms (end to end 2.54662 ms, enqueue 0.543701 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31805 ms - Host latency: 1.86304 ms (end to end 2.55596 ms, enqueue 0.539294 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31787 ms - Host latency: 1.86008 ms (end to end 2.56296 ms, enqueue 0.541187 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31761 ms - Host latency: 1.86008 ms (end to end 2.56383 ms, enqueue 0.543665 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31823 ms - Host latency: 1.86232 ms (end to end 2.56552 ms, enqueue 0.54303 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31953 ms - Host latency: 1.86169 ms (end to end 2.56205 ms, enqueue 0.541455 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31888 ms - Host latency: 1.86323 ms (end to end 2.56002 ms, enqueue 0.539307 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31676 ms - Host latency: 1.85886 ms (end to end 2.54987 ms, enqueue 0.536914 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31888 ms - Host latency: 1.8623 ms (end to end 2.55399 ms, enqueue 0.537219 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31802 ms - Host latency: 1.8603 ms (end to end 2.55103 ms, enqueue 0.551331 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32 ms - Host latency: 1.86527 ms (end to end 2.55397 ms, enqueue 0.550769 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31851 ms - Host latency: 1.86145 ms (end to end 2.55231 ms, enqueue 0.53844 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31843 ms - Host latency: 1.86254 ms (end to end 2.55748 ms, enqueue 0.545557 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31844 ms - Host latency: 1.86244 ms (end to end 2.55441 ms, enqueue 0.544824 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32051 ms - Host latency: 1.86537 ms (end to end 2.55708 ms, enqueue 0.544312 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31815 ms - Host latency: 1.8597 ms (end to end 2.55308 ms, enqueue 0.541162 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31942 ms - Host latency: 1.86434 ms (end to end 2.54192 ms, enqueue 0.542053 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3166 ms - Host latency: 1.86062 ms (end to end 2.54939 ms, enqueue 0.538623 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31876 ms - Host latency: 1.86279 ms (end to end 2.54434 ms, enqueue 0.539319 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31758 ms - Host latency: 1.85977 ms (end to end 2.54554 ms, enqueue 0.537036 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31935 ms - Host latency: 1.86431 ms (end to end 2.55452 ms, enqueue 0.539416 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31898 ms - Host latency: 1.86199 ms (end to end 2.55254 ms, enqueue 0.539221 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31797 ms - Host latency: 1.86017 ms (end to end 2.55438 ms, enqueue 0.537476 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31884 ms - Host latency: 1.86284 ms (end to end 2.55518 ms, enqueue 0.544543 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31953 ms - Host latency: 1.86204 ms (end to end 2.55663 ms, enqueue 0.544067 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32 ms - Host latency: 1.86274 ms (end to end 2.55854 ms, enqueue 0.541724 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31678 ms - Host latency: 1.85934 ms (end to end 2.55314 ms, enqueue 0.536877 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31823 ms - Host latency: 1.86162 ms (end to end 2.55457 ms, enqueue 0.538684 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31853 ms - Host latency: 1.86031 ms (end to end 2.55416 ms, enqueue 0.53385 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32043 ms - Host latency: 1.86224 ms (end to end 2.55339 ms, enqueue 0.537219 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31874 ms - Host latency: 1.86176 ms (end to end 2.5509 ms, enqueue 0.541748 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32048 ms - Host latency: 1.86508 ms (end to end 2.55513 ms, enqueue 0.539624 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31841 ms - Host latency: 1.86147 ms (end to end 2.54409 ms, enqueue 0.537646 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31827 ms - Host latency: 1.8606 ms (end to end 2.55348 ms, enqueue 0.542749 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31766 ms - Host latency: 1.86069 ms (end to end 2.55334 ms, enqueue 0.54342 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31754 ms - Host latency: 1.86155 ms (end to end 2.55463 ms, enqueue 0.5427 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31852 ms - Host latency: 1.86066 ms (end to end 2.55458 ms, enqueue 0.541882 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31904 ms - Host latency: 1.86132 ms (end to end 2.55647 ms, enqueue 0.53844 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31862 ms - Host latency: 1.86235 ms (end to end 2.55006 ms, enqueue 0.544153 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31929 ms - Host latency: 1.86235 ms (end to end 2.5615 ms, enqueue 0.539282 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32029 ms - Host latency: 1.86343 ms (end to end 2.5436 ms, enqueue 0.559753 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31826 ms - Host latency: 1.86453 ms (end to end 2.54241 ms, enqueue 0.561426 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32041 ms - Host latency: 1.86295 ms (end to end 2.5436 ms, enqueue 0.544971 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31842 ms - Host latency: 1.86584 ms (end to end 2.54111 ms, enqueue 0.577612 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31956 ms - Host latency: 1.86299 ms (end to end 2.55042 ms, enqueue 0.55332 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31748 ms - Host latency: 1.86298 ms (end to end 2.55222 ms, enqueue 0.552893 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31681 ms - Host latency: 1.85857 ms (end to end 2.55251 ms, enqueue 0.542114 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31801 ms - Host latency: 1.8619 ms (end to end 2.55352 ms, enqueue 0.542749 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31874 ms - Host latency: 1.86116 ms (end to end 2.55682 ms, enqueue 0.538586 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31726 ms - Host latency: 1.86014 ms (end to end 2.5522 ms, enqueue 0.542688 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32032 ms - Host latency: 1.86462 ms (end to end 2.55376 ms, enqueue 0.540588 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31836 ms - Host latency: 1.86079 ms (end to end 2.55166 ms, enqueue 0.536182 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31851 ms - Host latency: 1.86149 ms (end to end 2.55018 ms, enqueue 0.538464 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31887 ms - Host latency: 1.8632 ms (end to end 2.55321 ms, enqueue 0.543958 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31689 ms - Host latency: 1.86005 ms (end to end 2.55312 ms, enqueue 0.542566 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31874 ms - Host latency: 1.86093 ms (end to end 2.55399 ms, enqueue 0.539429 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31875 ms - Host latency: 1.86589 ms (end to end 2.55765 ms, enqueue 0.544348 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31843 ms - Host latency: 1.86193 ms (end to end 2.55524 ms, enqueue 0.543347 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31873 ms - Host latency: 1.86136 ms (end to end 2.55482 ms, enqueue 0.535083 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31782 ms - Host latency: 1.86013 ms (end to end 2.55568 ms, enqueue 0.534082 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3188 ms - Host latency: 1.86295 ms (end to end 2.55509 ms, enqueue 0.538538 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31891 ms - Host latency: 1.86345 ms (end to end 2.55391 ms, enqueue 0.54856 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31801 ms - Host latency: 1.86168 ms (end to end 2.55197 ms, enqueue 0.536316 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3182 ms - Host latency: 1.86023 ms (end to end 2.55322 ms, enqueue 0.535376 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31758 ms - Host latency: 1.86232 ms (end to end 2.55416 ms, enqueue 0.543921 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.59758 ms - Host latency: 2.14563 ms (end to end 2.76078 ms, enqueue 1.0197 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31682 ms - Host latency: 1.85891 ms (end to end 2.53992 ms, enqueue 0.533936 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32012 ms - Host latency: 1.86348 ms (end to end 2.55598 ms, enqueue 0.539844 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31926 ms - Host latency: 1.86238 ms (end to end 2.55374 ms, enqueue 0.539209 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31877 ms - Host latency: 1.8615 ms (end to end 2.55432 ms, enqueue 0.53833 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31958 ms - Host latency: 1.86138 ms (end to end 2.55571 ms, enqueue 0.540039 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31709 ms - Host latency: 1.85896 ms (end to end 2.55405 ms, enqueue 0.537915 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31895 ms - Host latency: 1.86035 ms (end to end 2.55686 ms, enqueue 0.541333 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31885 ms - Host latency: 1.86292 ms (end to end 2.55894 ms, enqueue 0.538061 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31843 ms - Host latency: 1.86111 ms (end to end 2.55393 ms, enqueue 0.534497 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31824 ms - Host latency: 1.86223 ms (end to end 2.55181 ms, enqueue 0.538892 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31934 ms - Host latency: 1.86226 ms (end to end 2.55684 ms, enqueue 0.548706 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31777 ms - Host latency: 1.86201 ms (end to end 2.55408 ms, enqueue 0.538501 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31858 ms - Host latency: 1.86079 ms (end to end 2.55503 ms, enqueue 0.534546 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31938 ms - Host latency: 1.8626 ms (end to end 2.5531 ms, enqueue 0.540601 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31614 ms - Host latency: 1.85806 ms (end to end 2.55247 ms, enqueue 0.541528 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3196 ms - Host latency: 1.86318 ms (end to end 2.55574 ms, enqueue 0.541943 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31885 ms - Host latency: 1.86191 ms (end to end 2.55608 ms, enqueue 0.540039 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31816 ms - Host latency: 1.86345 ms (end to end 2.55569 ms, enqueue 0.547119 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31851 ms - Host latency: 1.86082 ms (end to end 2.55498 ms, enqueue 0.535693 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31836 ms - Host latency: 1.86123 ms (end to end 2.55054 ms, enqueue 0.538477 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31646 ms - Host latency: 1.85815 ms (end to end 2.5502 ms, enqueue 0.535181 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31863 ms - Host latency: 1.86042 ms (end to end 2.55273 ms, enqueue 0.538501 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31858 ms - Host latency: 1.8603 ms (end to end 2.55444 ms, enqueue 0.535767 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31968 ms - Host latency: 1.8645 ms (end to end 2.55579 ms, enqueue 0.536523 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31851 ms - Host latency: 1.86172 ms (end to end 2.55605 ms, enqueue 0.543164 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3168 ms - Host latency: 1.85828 ms (end to end 2.55144 ms, enqueue 0.541772 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31792 ms - Host latency: 1.85967 ms (end to end 2.55073 ms, enqueue 0.542529 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31963 ms - Host latency: 1.86125 ms (end to end 2.55854 ms, enqueue 0.537451 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31995 ms - Host latency: 1.86335 ms (end to end 2.5541 ms, enqueue 0.539111 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31851 ms - Host latency: 1.86094 ms (end to end 2.55359 ms, enqueue 0.534668 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31934 ms - Host latency: 1.86323 ms (end to end 2.55601 ms, enqueue 0.540259 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31807 ms - Host latency: 1.86084 ms (end to end 2.55088 ms, enqueue 0.535181 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31975 ms - Host latency: 1.8634 ms (end to end 2.55554 ms, enqueue 0.544922 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31748 ms - Host latency: 1.86111 ms (end to end 2.55034 ms, enqueue 0.536255 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31924 ms - Host latency: 1.86284 ms (end to end 2.55615 ms, enqueue 0.541504 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31853 ms - Host latency: 1.85967 ms (end to end 2.55593 ms, enqueue 0.538428 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31919 ms - Host latency: 1.86331 ms (end to end 2.55723 ms, enqueue 0.542871 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31716 ms - Host latency: 1.86057 ms (end to end 2.54409 ms, enqueue 0.540674 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31831 ms - Host latency: 1.8625 ms (end to end 2.55439 ms, enqueue 0.542212 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31836 ms - Host latency: 1.86018 ms (end to end 2.55469 ms, enqueue 0.536377 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31897 ms - Host latency: 1.86201 ms (end to end 2.55652 ms, enqueue 0.543506 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32021 ms - Host latency: 1.86316 ms (end to end 2.55295 ms, enqueue 0.537891 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31736 ms - Host latency: 1.86067 ms (end to end 2.55234 ms, enqueue 0.535864 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31748 ms - Host latency: 1.85938 ms (end to end 2.55115 ms, enqueue 0.536572 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31677 ms - Host latency: 1.86143 ms (end to end 2.55039 ms, enqueue 0.546948 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31975 ms - Host latency: 1.8624 ms (end to end 2.55791 ms, enqueue 0.540601 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3187 ms - Host latency: 1.85984 ms (end to end 2.55564 ms, enqueue 0.541797 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31641 ms - Host latency: 1.85969 ms (end to end 2.55332 ms, enqueue 0.547998 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31895 ms - Host latency: 1.86104 ms (end to end 2.55718 ms, enqueue 0.53667 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31892 ms - Host latency: 1.86074 ms (end to end 2.55596 ms, enqueue 0.534473 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31821 ms - Host latency: 1.8613 ms (end to end 2.55525 ms, enqueue 0.539478 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31707 ms - Host latency: 1.86052 ms (end to end 2.546 ms, enqueue 0.540625 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31851 ms - Host latency: 1.86111 ms (end to end 2.55334 ms, enqueue 0.536914 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31741 ms - Host latency: 1.86284 ms (end to end 2.53259 ms, enqueue 0.540405 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32024 ms - Host latency: 1.86482 ms (end to end 2.55576 ms, enqueue 0.540088 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31965 ms - Host latency: 1.86226 ms (end to end 2.56018 ms, enqueue 0.539453 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31804 ms - Host latency: 1.86013 ms (end to end 2.55286 ms, enqueue 0.544214 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31865 ms - Host latency: 1.86211 ms (end to end 2.55608 ms, enqueue 0.543701 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31719 ms - Host latency: 1.86118 ms (end to end 2.55322 ms, enqueue 0.539844 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3188 ms - Host latency: 1.8616 ms (end to end 2.55623 ms, enqueue 0.542749 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31833 ms - Host latency: 1.86184 ms (end to end 2.54321 ms, enqueue 0.536108 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31848 ms - Host latency: 1.86204 ms (end to end 2.55229 ms, enqueue 0.537988 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3188 ms - Host latency: 1.86062 ms (end to end 2.554 ms, enqueue 0.534351 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31692 ms - Host latency: 1.85977 ms (end to end 2.55229 ms, enqueue 0.540283 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31787 ms - Host latency: 1.86316 ms (end to end 2.53679 ms, enqueue 0.560425 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31912 ms - Host latency: 1.86265 ms (end to end 2.54998 ms, enqueue 0.553345 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31721 ms - Host latency: 1.85957 ms (end to end 2.5541 ms, enqueue 0.538672 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3178 ms - Host latency: 1.85984 ms (end to end 2.55557 ms, enqueue 0.541992 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31829 ms - Host latency: 1.8614 ms (end to end 2.55234 ms, enqueue 0.540112 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3187 ms - Host latency: 1.86072 ms (end to end 2.5552 ms, enqueue 0.532715 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31821 ms - Host latency: 1.8625 ms (end to end 2.55576 ms, enqueue 0.536255 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.32039 ms - Host latency: 1.86311 ms (end to end 2.55532 ms, enqueue 0.537231 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31797 ms - Host latency: 1.86147 ms (end to end 2.55298 ms, enqueue 0.538257 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31814 ms - Host latency: 1.86035 ms (end to end 2.55332 ms, enqueue 0.537378 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31821 ms - Host latency: 1.86206 ms (end to end 2.55261 ms, enqueue 0.535815 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31851 ms - Host latency: 1.86082 ms (end to end 2.55549 ms, enqueue 0.539551 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31658 ms - Host latency: 1.86086 ms (end to end 2.55107 ms, enqueue 0.541357 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31892 ms - Host latency: 1.86216 ms (end to end 2.54382 ms, enqueue 0.540918 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.3178 ms - Host latency: 1.85925 ms (end to end 2.55271 ms, enqueue 0.535962 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31804 ms - Host latency: 1.86016 ms (end to end 2.5564 ms, enqueue 0.536401 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31824 ms - Host latency: 1.86018 ms (end to end 2.55449 ms, enqueue 0.534009 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31853 ms - Host latency: 1.86191 ms (end to end 2.55564 ms, enqueue 0.537061 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31741 ms - Host latency: 1.8606 ms (end to end 2.54932 ms, enqueue 0.546436 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31631 ms - Host latency: 1.86138 ms (end to end 2.54978 ms, enqueue 0.539941 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31797 ms - Host latency: 1.86301 ms (end to end 2.55251 ms, enqueue 0.537012 ms)
[06/25/2021-18:13:51] [I] Average on 10 runs - GPU latency: 1.31846 ms - Host latency: 1.86326 ms (end to end 2.5553 ms, enqueue 0.54397 ms)
[06/25/2021-18:13:51] [I] Host Latency
[06/25/2021-18:13:51] [I] min: 1.84375 ms (end to end 1.88159 ms)
[06/25/2021-18:13:51] [I] max: 4.6886 ms (end to end 5.38684 ms)
[06/25/2021-18:13:51] [I] mean: 1.86625 ms (end to end 2.55938 ms)
[06/25/2021-18:13:51] [I] median: 1.86111 ms (end to end 2.5542 ms)
[06/25/2021-18:13:51] [I] percentile: 2.07727 ms at 99% (end to end 2.97946 ms at 99%)
[06/25/2021-18:13:51] [I] throughput: 751.043 qps
[06/25/2021-18:13:51] [I] walltime: 3.00382 s
[06/25/2021-18:13:51] [I] Enqueue Time
[06/25/2021-18:13:51] [I] min: 0.443115 ms
[06/25/2021-18:13:51] [I] max: 5.28906 ms
[06/25/2021-18:13:51] [I] median: 0.536987 ms
[06/25/2021-18:13:51] [I] GPU Compute
[06/25/2021-18:13:51] [I] min: 1.30249 ms
[06/25/2021-18:13:51] [I] max: 4.11279 ms
[06/25/2021-18:13:51] [I] mean: 1.32287 ms
[06/25/2021-18:13:51] [I] median: 1.3186 ms
[06/25/2021-18:13:51] [I] percentile: 1.53337 ms at 99%
[06/25/2021-18:13:51] [I] total compute time: 2.98438 s
&&&& PASSED TensorRT.trtexec # trtexec --loadEngine=resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine --fp16