[07/05/2021-16:10:19] [I] === Model Options ===
[07/05/2021-16:10:19] [I] Format: *
[07/05/2021-16:10:19] [I] Model:
[07/05/2021-16:10:19] [I] Output:
[07/05/2021-16:10:19] [I] === Build Options ===
[07/05/2021-16:10:19] [I] Max batch: 1
[07/05/2021-16:10:19] [I] Workspace: 16 MiB
[07/05/2021-16:10:19] [I] minTiming: 1
[07/05/2021-16:10:19] [I] avgTiming: 8
[07/05/2021-16:10:19] [I] Precision: FP32
[07/05/2021-16:10:19] [I] Calibration:
[07/05/2021-16:10:19] [I] Refit: Disabled
[07/05/2021-16:10:19] [I] Safe mode: Disabled
[07/05/2021-16:10:19] [I] Save engine:
[07/05/2021-16:10:19] [I] Load engine: Facevisa_DtyStainSidebottomcenterDetection.bin
[07/05/2021-16:10:19] [I] Builder Cache: Enabled
[07/05/2021-16:10:19] [I] NVTX verbosity: 0
[07/05/2021-16:10:19] [I] Tactic sources: Using default tactic sources
[07/05/2021-16:10:19] [I] Input(s)s format: fp32:CHW
[07/05/2021-16:10:19] [I] Output(s)s format: fp32:CHW
[07/05/2021-16:10:19] [I] Input build shapes: model
[07/05/2021-16:10:19] [I] Input calibration shapes: model
[07/05/2021-16:10:19] [I] === System Options ===
[07/05/2021-16:10:19] [I] Device: 0
[07/05/2021-16:10:19] [I] DLACore:
[07/05/2021-16:10:19] [I] Plugins:
[07/05/2021-16:10:19] [I] === Inference Options ===
[07/05/2021-16:10:19] [I] Batch: 1
[07/05/2021-16:10:19] [I] Input inference shapes: model
[07/05/2021-16:10:19] [I] Iterations: 10
[07/05/2021-16:10:19] [I] Duration: 3s (+ 200ms warm up)
[07/05/2021-16:10:19] [I] Sleep time: 0ms
[07/05/2021-16:10:19] [I] Streams: 1
[07/05/2021-16:10:19] [I] ExposeDMA: Disabled
[07/05/2021-16:10:19] [I] Data transfers: Enabled
[07/05/2021-16:10:19] [I] Spin-wait: Disabled
[07/05/2021-16:10:19] [I] Multithreading: Disabled
[07/05/2021-16:10:19] [I] CUDA Graph: Disabled
[07/05/2021-16:10:19] [I] Separate profiling: Disabled
[07/05/2021-16:10:19] [I] Skip inference: Disabled
[07/05/2021-16:10:19] [I] Inputs:
[07/05/2021-16:10:19] [I] === Reporting Options ===
[07/05/2021-16:10:19] [I] Verbose: Enabled
[07/05/2021-16:10:19] [I] Averages: 10 inferences
[07/05/2021-16:10:19] [I] Percentile: 99
[07/05/2021-16:10:19] [I] Dump refittable layers:Disabled
[07/05/2021-16:10:19] [I] Dump output: Disabled
[07/05/2021-16:10:19] [I] Profile: Disabled
[07/05/2021-16:10:19] [I] Export timing to JSON file:
[07/05/2021-16:10:19] [I] Export output to JSON file:
[07/05/2021-16:10:19] [I] Export profile to JSON file:
[07/05/2021-16:10:19] [I]
[07/05/2021-16:10:19] [I] === Device Information ===
[07/05/2021-16:10:19] [I] Selected Device: GeForce RTX 2070
[07/05/2021-16:10:19] [I] Compute Capability: 7.5
[07/05/2021-16:10:19] [I] SMs: 36
[07/05/2021-16:10:19] [I] Compute Clock Rate: 1.71 GHz
[07/05/2021-16:10:19] [I] Device Global Memory: 8192 MiB
[07/05/2021-16:10:19] [I] Shared Memory per SM: 64 KiB
[07/05/2021-16:10:19] [I] Memory Bus Width: 256 bits (ECC disabled)
[07/05/2021-16:10:19] [I] Memory Clock Rate: 7.001 GHz
[07/05/2021-16:10:19] [I]
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::Proposal version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::Split version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[07/05/2021-16:10:19] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[07/05/2021-16:10:21] [W] [TRT] TensorRT was linked against cuBLAS/cuBLAS LT 11.3.0 but loaded cuBLAS/cuBLAS LT 11.2.1
[07/05/2021-16:10:21] [V] [TRT] Deserialize required 1201612 microseconds.
[07/05/2021-16:10:21] [I] Engine loaded in 1.80721 sec.
[07/05/2021-16:10:21] [W] [TRT] TensorRT was linked against cuBLAS/cuBLAS LT 11.3.0 but loaded cuBLAS/cuBLAS LT 11.2.1
[07/05/2021-16:10:21] [V] [TRT] Allocated persistent device memory of size 22041600
[07/05/2021-16:10:21] [V] [TRT] Allocated activation device memory of size 12861440
[07/05/2021-16:10:21] [V] [TRT] Assigning persistent memory blocks for various profiles
[07/05/2021-16:10:21] [I] Starting inference
[07/05/2021-16:10:24] [I] Warmup completed 1 queries over 200 ms
[07/05/2021-16:10:24] [I] Timing trace has 692 queries over 2.67593 s
[07/05/2021-16:10:24] [I] Trace averages of 10 runs:
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 4.47834 ms - Host latency: 4.65532 ms (end to end 4.66834 ms, enqueue 3.53961 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 4.46054 ms - Host latency: 4.63631 ms (end to end 4.64689 ms, enqueue 3.50911 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 4.34721 ms - Host latency: 4.52316 ms (end to end 4.53468 ms, enqueue 3.51492 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 4.29115 ms - Host latency: 4.46871 ms (end to end 4.48012 ms, enqueue 3.50678 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 4.27463 ms - Host latency: 4.44891 ms (end to end 4.45999 ms, enqueue 3.49173 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 4.24718 ms - Host latency: 4.42275 ms (end to end 4.4335 ms, enqueue 3.50724 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 4.1771 ms - Host latency: 4.35397 ms (end to end 4.36735 ms, enqueue 3.4988 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.63334 ms - Host latency: 3.80919 ms (end to end 3.82156 ms, enqueue 3.50828 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.64056 ms - Host latency: 3.81435 ms (end to end 3.82441 ms, enqueue 3.51563 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.62659 ms - Host latency: 3.80853 ms (end to end 3.81926 ms, enqueue 3.5986 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.64018 ms - Host latency: 3.8121 ms (end to end 3.82317 ms, enqueue 3.5411 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.64753 ms - Host latency: 3.82269 ms (end to end 3.83407 ms, enqueue 3.52636 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.62808 ms - Host latency: 3.80856 ms (end to end 3.81971 ms, enqueue 3.58896 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.63591 ms - Host latency: 3.8116 ms (end to end 3.82231 ms, enqueue 3.63445 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.64324 ms - Host latency: 3.81995 ms (end to end 3.8308 ms, enqueue 3.49835 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.64309 ms - Host latency: 3.81698 ms (end to end 3.82705 ms, enqueue 3.53059 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.61602 ms - Host latency: 3.80594 ms (end to end 3.81816 ms, enqueue 3.52579 ms)
[07/05/2021-16:10:24] [I] Average on 10 runs - GPU latency: 3.61676 ms - Host latency: 3.7908 ms (end to end 3.80214 ms, enqueue 3.52205 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62858 ms - Host latency: 3.80416 ms (end to end 3.81516 ms, enqueue 3.49937 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.65522 ms - Host latency: 3.82915 ms (end to end 3.8406 ms, enqueue 3.6266 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.6408 ms - Host latency: 3.81987 ms (end to end 3.83143 ms, enqueue 3.54858 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.65204 ms - Host latency: 3.82781 ms (end to end 3.83912 ms, enqueue 3.51987 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.6458 ms - Host latency: 3.81986 ms (end to end 3.83136 ms, enqueue 3.54774 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63689 ms - Host latency: 3.8111 ms (end to end 3.82258 ms, enqueue 3.51028 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62875 ms - Host latency: 3.80417 ms (end to end 3.81539 ms, enqueue 3.51521 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63724 ms - Host latency: 3.81412 ms (end to end 3.82561 ms, enqueue 3.50889 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62866 ms - Host latency: 3.80507 ms (end to end 3.81771 ms, enqueue 3.50651 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.6327 ms - Host latency: 3.80632 ms (end to end 3.81772 ms, enqueue 3.50669 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62728 ms - Host latency: 3.80145 ms (end to end 3.81346 ms, enqueue 3.54459 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63237 ms - Host latency: 3.80636 ms (end to end 3.81697 ms, enqueue 3.49496 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63295 ms - Host latency: 3.80886 ms (end to end 3.81989 ms, enqueue 3.53853 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63535 ms - Host latency: 3.80947 ms (end to end 3.82128 ms, enqueue 3.5089 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63622 ms - Host latency: 3.81218 ms (end to end 3.82427 ms, enqueue 3.49529 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62695 ms - Host latency: 3.80164 ms (end to end 3.81267 ms, enqueue 3.52327 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63604 ms - Host latency: 3.8099 ms (end to end 3.82036 ms, enqueue 3.5209 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62144 ms - Host latency: 3.80687 ms (end to end 3.81854 ms, enqueue 3.52112 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63824 ms - Host latency: 3.81213 ms (end to end 3.82321 ms, enqueue 3.5219 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63219 ms - Host latency: 3.80695 ms (end to end 3.81779 ms, enqueue 3.61628 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.6297 ms - Host latency: 3.80494 ms (end to end 3.81666 ms, enqueue 3.57588 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.64023 ms - Host latency: 3.81289 ms (end to end 3.82388 ms, enqueue 3.53071 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63079 ms - Host latency: 3.80391 ms (end to end 3.8148 ms, enqueue 3.52686 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.6334 ms - Host latency: 3.80745 ms (end to end 3.81875 ms, enqueue 3.50173 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62969 ms - Host latency: 3.80315 ms (end to end 3.81406 ms, enqueue 3.52236 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.64097 ms - Host latency: 3.81418 ms (end to end 3.82424 ms, enqueue 3.52075 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62791 ms - Host latency: 3.80112 ms (end to end 3.81169 ms, enqueue 3.51719 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63215 ms - Host latency: 3.80503 ms (end to end 3.81545 ms, enqueue 3.50833 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62739 ms - Host latency: 3.80176 ms (end to end 3.81411 ms, enqueue 3.51423 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63521 ms - Host latency: 3.8115 ms (end to end 3.82346 ms, enqueue 3.51267 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63699 ms - Host latency: 3.81064 ms (end to end 3.82119 ms, enqueue 3.51682 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63081 ms - Host latency: 3.80879 ms (end to end 3.82004 ms, enqueue 3.51499 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63472 ms - Host latency: 3.80791 ms (end to end 3.81892 ms, enqueue 3.51201 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62651 ms - Host latency: 3.79915 ms (end to end 3.80896 ms, enqueue 3.48987 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63857 ms - Host latency: 3.81482 ms (end to end 3.82664 ms, enqueue 3.50044 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63191 ms - Host latency: 3.81443 ms (end to end 3.82666 ms, enqueue 3.5261 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63079 ms - Host latency: 3.80383 ms (end to end 3.81445 ms, enqueue 3.52966 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63081 ms - Host latency: 3.8052 ms (end to end 3.81621 ms, enqueue 3.50908 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62898 ms - Host latency: 3.80417 ms (end to end 3.81521 ms, enqueue 3.52354 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63877 ms - Host latency: 3.81355 ms (end to end 3.82478 ms, enqueue 3.49944 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62327 ms - Host latency: 3.8134 ms (end to end 3.82505 ms, enqueue 3.50242 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63423 ms - Host latency: 3.80815 ms (end to end 3.81916 ms, enqueue 3.52219 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63674 ms - Host latency: 3.81042 ms (end to end 3.82097 ms, enqueue 3.75645 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63191 ms - Host latency: 3.80645 ms (end to end 3.8178 ms, enqueue 3.46851 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63333 ms - Host latency: 3.8073 ms (end to end 3.81895 ms, enqueue 3.52285 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.6324 ms - Host latency: 3.80645 ms (end to end 3.81809 ms, enqueue 3.50906 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62847 ms - Host latency: 3.80525 ms (end to end 3.8178 ms, enqueue 3.56873 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63489 ms - Host latency: 3.80857 ms (end to end 3.81938 ms, enqueue 3.74004 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.62952 ms - Host latency: 3.80518 ms (end to end 3.8166 ms, enqueue 3.47241 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.61292 ms - Host latency: 3.78755 ms (end to end 3.80486 ms, enqueue 3.51299 ms)
[07/05/2021-16:10:25] [I] Average on 10 runs - GPU latency: 3.63286 ms - Host latency: 3.80718 ms (end to end 3.81912 ms, enqueue 3.53171 ms)
[07/05/2021-16:10:25] [I] Host Latency
[07/05/2021-16:10:25] [I] min: 3.7511 ms (end to end 3.76282 ms)
[07/05/2021-16:10:25] [I] max: 4.73224 ms (end to end 4.7486 ms)
[07/05/2021-16:10:25] [I] mean: 3.879 ms (end to end 3.89038 ms)
[07/05/2021-16:10:25] [I] median: 3.80347 ms (end to end 3.81451 ms)
[07/05/2021-16:10:25] [I] percentile: 4.66724 ms at 99% (end to end 4.67633 ms at 99%)
[07/05/2021-16:10:25] [I] throughput: 258.602 qps
[07/05/2021-16:10:25] [I] walltime: 2.67593 s
[07/05/2021-16:10:25] [I] Enqueue Time
[07/05/2021-16:10:25] [I] min: 3.39746 ms
[07/05/2021-16:10:25] [I] max: 4.59717 ms
[07/05/2021-16:10:25] [I] median: 3.5083 ms
[07/05/2021-16:10:25] [I] GPU Compute
[07/05/2021-16:10:25] [I] min: 3.58008 ms
[07/05/2021-16:10:25] [I] max: 4.54773 ms
[07/05/2021-16:10:25] [I] mean: 3.70334 ms
[07/05/2021-16:10:25] [I] median: 3.62756 ms
[07/05/2021-16:10:25] [I] percentile: 4.49329 ms at 99%
[07/05/2021-16:10:25] [I] total compute time: 2.56271 s
&&&& PASSED TensorRT.trtexec # trtexec.exe --loadEngine=Facevisa_DtyStainSidebottomcenterDetection.bin --verbose=true