&&&& RUNNING TensorRT.trtexec # ./trtexec --onnx=../samples/sampleSEGNET/APA_FS_Model.onnx --verbose --batch=32
[03/18/2021-10:21:22] [I] === Model Options ===
[03/18/2021-10:21:22] [I] Format: ONNX
[03/18/2021-10:21:22] [I] Model: ../samples/sampleSEGNET/APA_FS_Model.onnx
[03/18/2021-10:21:22] [I] Output:
[03/18/2021-10:21:22] [I] === Build Options ===
[03/18/2021-10:21:22] [I] Max batch: 32
[03/18/2021-10:21:22] [I] Workspace: 16 MB
[03/18/2021-10:21:22] [I] minTiming: 1
[03/18/2021-10:21:22] [I] avgTiming: 8
[03/18/2021-10:21:22] [I] Precision: FP32
[03/18/2021-10:21:22] [I] Calibration: 
[03/18/2021-10:21:22] [I] Safe mode: Disabled
[03/18/2021-10:21:22] [I] Save engine: 
[03/18/2021-10:21:22] [I] Load engine: 
[03/18/2021-10:21:22] [I] Builder Cache: Enabled
[03/18/2021-10:21:22] [I] NVTX verbosity: 0
[03/18/2021-10:21:22] [I] Inputs format: fp32:CHW
[03/18/2021-10:21:22] [I] Outputs format: fp32:CHW
[03/18/2021-10:21:22] [I] Input build shapes: model
[03/18/2021-10:21:22] [I] Input calibration shapes: model
[03/18/2021-10:21:22] [I] === System Options ===
[03/18/2021-10:21:22] [I] Device: 0
[03/18/2021-10:21:22] [I] DLACore: 
[03/18/2021-10:21:22] [I] Plugins:
[03/18/2021-10:21:22] [I] === Inference Options ===
[03/18/2021-10:21:22] [I] Batch: 32
[03/18/2021-10:21:22] [I] Input inference shapes: model
[03/18/2021-10:21:22] [I] Iterations: 10
[03/18/2021-10:21:22] [I] Duration: 3s (+ 200ms warm up)
[03/18/2021-10:21:22] [I] Sleep time: 0ms
[03/18/2021-10:21:22] [I] Streams: 1
[03/18/2021-10:21:22] [I] ExposeDMA: Disabled
[03/18/2021-10:21:22] [I] Spin-wait: Disabled
[03/18/2021-10:21:22] [I] Multithreading: Disabled
[03/18/2021-10:21:22] [I] CUDA Graph: Disabled
[03/18/2021-10:21:22] [I] Skip inference: Disabled
[03/18/2021-10:21:22] [I] Inputs:
[03/18/2021-10:21:22] [I] === Reporting Options ===
[03/18/2021-10:21:22] [I] Verbose: Enabled
[03/18/2021-10:21:22] [I] Averages: 10 inferences
[03/18/2021-10:21:22] [I] Percentile: 99
[03/18/2021-10:21:22] [I] Dump output: Disabled
[03/18/2021-10:21:22] [I] Profile: Disabled
[03/18/2021-10:21:22] [I] Export timing to JSON file: 
[03/18/2021-10:21:22] [I] Export output to JSON file: 
[03/18/2021-10:21:22] [I] Export profile to JSON file: 
[03/18/2021-10:21:22] [I] 
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::Proposal version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::Split version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[03/18/2021-10:21:22] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
----------------------------------------------------------------
Input filename:   ../samples/sampleSEGNET/APA_FS_Model.onnx
ONNX IR version:  0.0.6
Opset version:    9
Producer name:    pytorch
Producer version: 1.7
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::Proposal version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::Split version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:202: Adding network input: input with dtype: float32, dimensions: (1, 3, 192, 256)
[03/18/2021-10:21:23] [V] [TRT] ImporterContext.hpp:116: Registering tensor: input for ONNX tensor: input
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 228
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 229
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 231
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 232
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 234
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 235
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 237
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 238
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 240
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 241
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 243
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 244
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 246
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 247
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 249
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 250
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 252
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 253
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 255
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 256
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 258
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 259
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 261
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 262
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 264
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 265
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 267
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 268
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 270
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 271
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 273
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 274
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 276
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 277
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 279
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 280
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 282
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 283
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 285
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: 286
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: Panoptic3_up.weight
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: deconv1.weight
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: deconv2.weight
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: down2/bd.weight
[03/18/2021-10:21:23] [V] [TRT] ModelImporter.cpp:90: Importing initializer: down3/bd.weight

 ----- Parsing of ONNX model ../samples/sampleSEGNET/APA_FS_Model.onnx is Done ---- 

[03/18/2021-10:21:43] [I] Starting inference threads
[03/18/2021-10:21:46] [I] Warmup completed 1696 queries over 200 ms
[03/18/2021-10:21:46] [I] Timing trace has 25312 queries over 3.00624 s
[03/18/2021-10:21:46] [I] Trace averages of 10 runs:
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.75922 ms - Host latency: 3.78046 ms (end to end 3.79189 ms, enqueue 0.663293 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76406 ms - Host latency: 3.78565 ms (end to end 3.79602 ms, enqueue 0.617079 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76668 ms - Host latency: 3.78657 ms (end to end 3.79847 ms, enqueue 0.622818 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.75948 ms - Host latency: 3.78063 ms (end to end 3.7916 ms, enqueue 0.635294 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76077 ms - Host latency: 3.78339 ms (end to end 3.79384 ms, enqueue 0.621078 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76379 ms - Host latency: 3.78264 ms (end to end 3.79327 ms, enqueue 0.635245 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76252 ms - Host latency: 3.78528 ms (end to end 3.79321 ms, enqueue 0.625687 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76378 ms - Host latency: 3.78544 ms (end to end 3.79462 ms, enqueue 0.598599 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76323 ms - Host latency: 3.78471 ms (end to end 3.79488 ms, enqueue 0.611871 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76666 ms - Host latency: 3.78895 ms (end to end 3.80128 ms, enqueue 0.603107 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76234 ms - Host latency: 3.78376 ms (end to end 3.79395 ms, enqueue 0.620581 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76478 ms - Host latency: 3.78672 ms (end to end 3.79719 ms, enqueue 0.603223 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76288 ms - Host latency: 3.78495 ms (end to end 3.79374 ms, enqueue 0.616186 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76234 ms - Host latency: 3.78329 ms (end to end 3.7926 ms, enqueue 0.591516 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76185 ms - Host latency: 3.78455 ms (end to end 3.79212 ms, enqueue 0.639941 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76268 ms - Host latency: 3.78387 ms (end to end 3.79575 ms, enqueue 0.603186 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76423 ms - Host latency: 3.78772 ms (end to end 3.79879 ms, enqueue 0.62453 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.7635 ms - Host latency: 3.78611 ms (end to end 3.79567 ms, enqueue 0.598602 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76271 ms - Host latency: 3.78307 ms (end to end 3.79289 ms, enqueue 0.626678 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76505 ms - Host latency: 3.78823 ms (end to end 3.79926 ms, enqueue 0.595667 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76531 ms - Host latency: 3.78705 ms (end to end 3.79755 ms, enqueue 0.600238 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76563 ms - Host latency: 3.78602 ms (end to end 3.79522 ms, enqueue 0.596472 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76439 ms - Host latency: 3.78816 ms (end to end 3.79777 ms, enqueue 0.615186 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76541 ms - Host latency: 3.78564 ms (end to end 3.79528 ms, enqueue 0.587073 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76533 ms - Host latency: 3.78793 ms (end to end 3.79779 ms, enqueue 0.614624 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76284 ms - Host latency: 3.78468 ms (end to end 3.79403 ms, enqueue 0.594995 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76198 ms - Host latency: 3.78462 ms (end to end 3.79542 ms, enqueue 0.653735 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76504 ms - Host latency: 3.78768 ms (end to end 3.79623 ms, enqueue 0.602124 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76586 ms - Host latency: 3.78619 ms (end to end 3.79662 ms, enqueue 0.612793 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76359 ms - Host latency: 3.78614 ms (end to end 3.79807 ms, enqueue 0.594983 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.75981 ms - Host latency: 3.78053 ms (end to end 3.79004 ms, enqueue 0.632227 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76318 ms - Host latency: 3.78247 ms (end to end 3.79343 ms, enqueue 0.595911 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76238 ms - Host latency: 3.78563 ms (end to end 3.79596 ms, enqueue 0.609705 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76309 ms - Host latency: 3.78483 ms (end to end 3.79537 ms, enqueue 0.615674 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76079 ms - Host latency: 3.78224 ms (end to end 3.79211 ms, enqueue 0.628992 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.7632 ms - Host latency: 3.78575 ms (end to end 3.79589 ms, enqueue 0.601501 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76328 ms - Host latency: 3.78433 ms (end to end 3.79412 ms, enqueue 0.620898 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76273 ms - Host latency: 3.78522 ms (end to end 3.7941 ms, enqueue 0.620532 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76389 ms - Host latency: 3.78407 ms (end to end 3.79463 ms, enqueue 0.609436 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76415 ms - Host latency: 3.78701 ms (end to end 3.79683 ms, enqueue 0.601489 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.7641 ms - Host latency: 3.7847 ms (end to end 3.79637 ms, enqueue 0.617932 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76157 ms - Host latency: 3.78171 ms (end to end 3.79227 ms, enqueue 0.606433 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76372 ms - Host latency: 3.78667 ms (end to end 3.79666 ms, enqueue 0.61648 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76876 ms - Host latency: 3.79684 ms (end to end 3.80811 ms, enqueue 0.608203 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77255 ms - Host latency: 3.80337 ms (end to end 3.81268 ms, enqueue 0.614685 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77086 ms - Host latency: 3.80437 ms (end to end 3.81235 ms, enqueue 0.599658 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76859 ms - Host latency: 3.80027 ms (end to end 3.80873 ms, enqueue 0.631201 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77 ms - Host latency: 3.80255 ms (end to end 3.8141 ms, enqueue 0.600623 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76776 ms - Host latency: 3.79617 ms (end to end 3.8069 ms, enqueue 0.646045 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77358 ms - Host latency: 3.80459 ms (end to end 3.81533 ms, enqueue 0.584326 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77461 ms - Host latency: 3.80713 ms (end to end 3.81553 ms, enqueue 0.607227 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77024 ms - Host latency: 3.79807 ms (end to end 3.80798 ms, enqueue 0.586987 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77112 ms - Host latency: 3.80239 ms (end to end 3.81196 ms, enqueue 0.630396 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77253 ms - Host latency: 3.80359 ms (end to end 3.81218 ms, enqueue 0.616235 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77034 ms - Host latency: 3.8001 ms (end to end 3.81099 ms, enqueue 0.607959 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77095 ms - Host latency: 3.80171 ms (end to end 3.81169 ms, enqueue 0.610278 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77087 ms - Host latency: 3.79966 ms (end to end 3.81069 ms, enqueue 0.65022 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77241 ms - Host latency: 3.80615 ms (end to end 3.81462 ms, enqueue 0.588086 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77061 ms - Host latency: 3.80176 ms (end to end 3.8116 ms, enqueue 0.603296 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77363 ms - Host latency: 3.80515 ms (end to end 3.81375 ms, enqueue 0.58479 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76877 ms - Host latency: 3.79944 ms (end to end 3.80981 ms, enqueue 0.604297 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77634 ms - Host latency: 3.80571 ms (end to end 3.81438 ms, enqueue 0.601172 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76953 ms - Host latency: 3.80044 ms (end to end 3.81323 ms, enqueue 0.602466 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77161 ms - Host latency: 3.80188 ms (end to end 3.81145 ms, enqueue 0.62085 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77256 ms - Host latency: 3.8033 ms (end to end 3.81428 ms, enqueue 0.585815 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.77061 ms - Host latency: 3.80566 ms (end to end 3.81477 ms, enqueue 0.61604 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76868 ms - Host latency: 3.79475 ms (end to end 3.80513 ms, enqueue 0.592627 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76685 ms - Host latency: 3.78765 ms (end to end 3.79731 ms, enqueue 0.601245 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76672 ms - Host latency: 3.78918 ms (end to end 3.79832 ms, enqueue 0.59729 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76592 ms - Host latency: 3.78733 ms (end to end 3.79487 ms, enqueue 0.615479 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76904 ms - Host latency: 3.79214 ms (end to end 3.80312 ms, enqueue 0.584302 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76526 ms - Host latency: 3.78652 ms (end to end 3.79565 ms, enqueue 0.60918 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76526 ms - Host latency: 3.78645 ms (end to end 3.79583 ms, enqueue 0.595044 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76426 ms - Host latency: 3.78779 ms (end to end 3.79805 ms, enqueue 0.606812 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76633 ms - Host latency: 3.78872 ms (end to end 3.79968 ms, enqueue 0.580811 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76975 ms - Host latency: 3.79456 ms (end to end 3.80681 ms, enqueue 0.593823 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76748 ms - Host latency: 3.7886 ms (end to end 3.79688 ms, enqueue 0.58042 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76497 ms - Host latency: 3.78796 ms (end to end 3.79631 ms, enqueue 0.604199 ms)
[03/18/2021-10:21:46] [I] Average on 10 runs - GPU latency: 1.76699 ms - Host latency: 3.78884 ms (end to end 3.79939 ms, enqueue 0.587793 ms)
[03/18/2021-10:21:46] [I] Host Latency
[03/18/2021-10:21:46] [I] min: 3.61304 ms (end to end 3.62695 ms)
[03/18/2021-10:21:46] [I] max: 3.82715 ms (end to end 3.83472 ms)
[03/18/2021-10:21:46] [I] mean: 3.79049 ms (end to end 3.8005 ms)
[03/18/2021-10:21:46] [I] median: 3.78918 ms (end to end 3.79956 ms)
[03/18/2021-10:21:46] [I] percentile: 3.81616 ms at 99% (end to end 3.82666 ms at 99%)
[03/18/2021-10:21:46] [I] throughput: 8419.82 qps
[03/18/2021-10:21:46] [I] walltime: 3.00624 s
[03/18/2021-10:21:46] [I] Enqueue Time
[03/18/2021-10:21:46] [I] min: 0.442322 ms
[03/18/2021-10:21:46] [I] max: 0.866211 ms
[03/18/2021-10:21:46] [I] median: 0.601685 ms
[03/18/2021-10:21:46] [I] GPU Compute
[03/18/2021-10:21:46] [I] min: 1.75018 ms
[03/18/2021-10:21:46] [I] max: 1.78931 ms
[03/18/2021-10:21:46] [I] mean: 1.7662 ms
[03/18/2021-10:21:46] [I] median: 1.76572 ms
[03/18/2021-10:21:46] [I] percentile: 1.78296 ms at 99%
[03/18/2021-10:21:46] [I] total compute time: 1.39706 s
&&&& PASSED TensorRT.trtexec # ./trtexec --onnx=../samples/sampleSEGNET/APA_FS_Model.onnx --verbose --batch=32