I don’t undersand how it is possible that on my PC, T2000 GPU, the build process works, but on your AGX it fails.
In addition, I don’t understand the limit issue you mentioned. If cuda really have this limit, so how should we use 8GB< ram GPUs?
And even though, so again, if it exceeds this limit, so why does it work on my T2000 GPU?
I used trtexec executable which contained in tensorrt 8.4 GA release, with the same command which you used, on my PC with 4GB ram GPU(T2000)-
[07/31/2022-10:07:36] [W] --workspace flag has been deprecated by --memPoolSize flag.
[07/31/2022-10:07:36] [I] === Model Options ===
[07/31/2022-10:07:36] [I] Format: ONNX
[07/31/2022-10:07:36] [I] Model: C:/projects/Dleware/Testers/Playground/Models/ASLtorch/Delivery/model2_folded.onnx
[07/31/2022-10:07:36] [I] Output:
[07/31/2022-10:07:36] [I] === Build Options ===
[07/31/2022-10:07:36] [I] Max batch: explicit batch
[07/31/2022-10:07:36] [I] Memory Pools: workspace: 30000 MiB, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[07/31/2022-10:07:36] [I] minTiming: 1
[07/31/2022-10:07:36] [I] avgTiming: 8
[07/31/2022-10:07:36] [I] Precision: FP32
[07/31/2022-10:07:36] [I] LayerPrecisions:
[07/31/2022-10:07:36] [I] Calibration:
[07/31/2022-10:07:36] [I] Refit: Disabled
[07/31/2022-10:07:36] [I] Sparsity: Disabled
[07/31/2022-10:07:36] [I] Safe mode: Disabled
[07/31/2022-10:07:36] [I] DirectIO mode: Disabled
[07/31/2022-10:07:36] [I] Restricted mode: Disabled
[07/31/2022-10:07:36] [I] Build only: Disabled
[07/31/2022-10:07:36] [I] Save engine: model2_folded.engine
[07/31/2022-10:07:36] [I] Load engine:
[07/31/2022-10:07:36] [I] Profiling verbosity: 0
[07/31/2022-10:07:36] [I] Tactic sources: Using default tactic sources
[07/31/2022-10:07:36] [I] timingCacheMode: local
[07/31/2022-10:07:36] [I] timingCacheFile:
[07/31/2022-10:07:36] [I] Input(s)s format: fp32:CHW
[07/31/2022-10:07:36] [I] Output(s)s format: fp32:CHW
[07/31/2022-10:07:36] [I] Input build shape: keypoints=1x1x2+1x2565x2+1x8000x2
[07/31/2022-10:07:36] [I] Input build shape: scores.1=1x1+1x2565+1x8000
[07/31/2022-10:07:36] [I] Input build shape: score_map=1x1x320x256+1x1x320x256+1x1x320x256
[07/31/2022-10:07:36] [I] Input build shape: dense_feat_map=1x128x80x64+1x128x80x64+1x128x190x173
[07/31/2022-10:07:36] [I] Input calibration shapes: model
[07/31/2022-10:07:36] [I] === System Options ===
[07/31/2022-10:07:36] [I] Device: 0
[07/31/2022-10:07:36] [I] DLACore:
[07/31/2022-10:07:36] [I] Plugins:
[07/31/2022-10:07:36] [I] === Inference Options ===
[07/31/2022-10:07:36] [I] Batch: Explicit
[07/31/2022-10:07:36] [I] Input inference shape: keypoints=1x2565x2
[07/31/2022-10:07:36] [I] Input inference shape: scores.1=1x2565
[07/31/2022-10:07:36] [I] Input inference shape: score_map=1x1x320x256
[07/31/2022-10:07:36] [I] Input inference shape: dense_feat_map=1x128x80x64
[07/31/2022-10:07:36] [I] Iterations: 10
[07/31/2022-10:07:36] [I] Duration: 3s (+ 200ms warm up)
[07/31/2022-10:07:36] [I] Sleep time: 0ms
[07/31/2022-10:07:36] [I] Idle time: 0ms
[07/31/2022-10:07:36] [I] Streams: 1
[07/31/2022-10:07:36] [I] ExposeDMA: Disabled
[07/31/2022-10:07:36] [I] Data transfers: Enabled
[07/31/2022-10:07:36] [I] Spin-wait: Disabled
[07/31/2022-10:07:36] [I] Multithreading: Disabled
[07/31/2022-10:07:36] [I] CUDA Graph: Disabled
[07/31/2022-10:07:36] [I] Separate profiling: Disabled
[07/31/2022-10:07:36] [I] Time Deserialize: Disabled
[07/31/2022-10:07:36] [I] Time Refit: Disabled
[07/31/2022-10:07:36] [I] Inputs:
[07/31/2022-10:07:36] [I] === Reporting Options ===
[07/31/2022-10:07:36] [I] Verbose: Disabled
[07/31/2022-10:07:36] [I] Averages: 10 inferences
[07/31/2022-10:07:36] [I] Percentile: 99
[07/31/2022-10:07:36] [I] Dump refittable layers:Disabled
[07/31/2022-10:07:36] [I] Dump output: Disabled
[07/31/2022-10:07:36] [I] Profile: Disabled
[07/31/2022-10:07:36] [I] Export timing to JSON file:
[07/31/2022-10:07:36] [I] Export output to JSON file:
[07/31/2022-10:07:36] [I] Export profile to JSON file:
[07/31/2022-10:07:36] [I]
[07/31/2022-10:07:36] [I] === Device Information ===
[07/31/2022-10:07:36] [I] Selected Device: Quadro T2000
[07/31/2022-10:07:36] [I] Compute Capability: 7.5
[07/31/2022-10:07:36] [I] SMs: 16
[07/31/2022-10:07:36] [I] Compute Clock Rate: 1.785 GHz
[07/31/2022-10:07:36] [I] Device Global Memory: 4096 MiB
[07/31/2022-10:07:36] [I] Shared Memory per SM: 64 KiB
[07/31/2022-10:07:36] [I] Memory Bus Width: 128 bits (ECC disabled)
[07/31/2022-10:07:36] [I] Memory Clock Rate: 4.001 GHz
[07/31/2022-10:07:36] [I]
[07/31/2022-10:07:36] [I] TensorRT version: 8.4.2
[07/31/2022-10:07:37] [I] [TRT] [MemUsageChange] Init CUDA: CPU +412, GPU +0, now: CPU 8675, GPU 901 (MiB)
[07/31/2022-10:07:38] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +215, GPU +68, now: CPU 9078, GPU 969 (MiB)
[07/31/2022-10:07:38] [I] Start parsing network model
[07/31/2022-10:07:38] [I] [TRT] ----------------------------------------------------------------
[07/31/2022-10:07:38] [I] [TRT] Input filename: C:/projects/Dleware/Testers/Playground/Models/ASLtorch/Delivery/model2_folded.onnx
[07/31/2022-10:07:38] [I] [TRT] ONNX IR version: 0.0.7
[07/31/2022-10:07:38] [I] [TRT] Opset version: 13
[07/31/2022-10:07:38] [I] [TRT] Producer name:
[07/31/2022-10:07:38] [I] [TRT] Producer version:
[07/31/2022-10:07:38] [I] [TRT] Domain:
[07/31/2022-10:07:38] [I] [TRT] Model version: 0
[07/31/2022-10:07:38] [I] [TRT] Doc string:
[07/31/2022-10:07:38] [I] [TRT] ----------------------------------------------------------------
[07/31/2022-10:07:38] [W] [TRT] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/31/2022-10:07:38] [I] Finish parsing network model
[07/31/2022-10:07:38] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +539, GPU +206, now: CPU 9479, GPU 1175 (MiB)
[07/31/2022-10:07:39] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +407, GPU +146, now: CPU 9886, GPU 1321 (MiB)
[07/31/2022-10:07:39] [W] [TRT] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.0.4
[07/31/2022-10:07:39] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[07/31/2022-10:07:50] [W] [TRT] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[07/31/2022-10:07:50] [W] [TRT] (# 1 (SHAPE keypoints))
[07/31/2022-10:07:50] [W] [TRT] (# 2 (SHAPE dense_feat_map))
[07/31/2022-10:07:50] [W] [TRT] (# 3 (SHAPE dense_feat_map))
[07/31/2022-10:07:52] [I] [TRT] Detected 4 inputs and 3 output network tensors.
[07/31/2022-10:07:52] [W] [TRT] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[07/31/2022-10:07:52] [W] [TRT] (# 1 (SHAPE keypoints))
[07/31/2022-10:07:52] [W] [TRT] (# 2 (SHAPE dense_feat_map))
[07/31/2022-10:07:52] [W] [TRT] (# 3 (SHAPE dense_feat_map))
[07/31/2022-10:07:52] [I] [TRT] Total Host Persistent Memory: 8832
[07/31/2022-10:07:52] [I] [TRT] Total Device Persistent Memory: 2460160
[07/31/2022-10:07:52] [I] [TRT] Total Scratch Memory: 21085440
[07/31/2022-10:07:52] [I] [TRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 375 MiB
[07/31/2022-10:07:52] [I] [TRT] [BlockAssignment] Algorithm ShiftNTopDown took 0.972ms to assign 13 blocks to 22 nodes requiring 27150848 bytes.
[07/31/2022-10:07:52] [I] [TRT] Total Activation Memory: 27150848
[07/31/2022-10:07:52] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +0, GPU +6, now: CPU 0, GPU 6 (MiB)
[07/31/2022-10:07:52] [W] [TRT] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[07/31/2022-10:07:52] [W] [TRT] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[07/31/2022-10:07:52] [I] Engine built in 15.3986 sec.
[07/31/2022-10:07:52] [I] [TRT] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 10203, GPU 1403 (MiB)
[07/31/2022-10:07:52] [I] [TRT] Loaded engine size: 0 MiB
[07/31/2022-10:07:52] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +2, now: CPU 0, GPU 2 (MiB)
[07/31/2022-10:07:52] [I] Engine deserialized in 0.0029188 sec.
[07/31/2022-10:07:52] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +28, now: CPU 0, GPU 30 (MiB)
[07/31/2022-10:07:52] [I] Using random values for input keypoints
[07/31/2022-10:07:52] [I] Created input binding for keypoints with dimensions 1x2565x2
[07/31/2022-10:07:52] [I] Using random values for input scores.1
[07/31/2022-10:07:52] [I] Created input binding for scores.1 with dimensions 1x2565
[07/31/2022-10:07:52] [I] Using random values for input score_map
[07/31/2022-10:07:52] [I] Created input binding for score_map with dimensions 1x1x320x256
[07/31/2022-10:07:52] [I] Using random values for input dense_feat_map
[07/31/2022-10:07:52] [I] Created input binding for dense_feat_map with dimensions 1x128x80x64
[07/31/2022-10:07:52] [I] Using random values for output scores
[07/31/2022-10:07:52] [I] Created output binding for scores with dimensions 2565x1
[07/31/2022-10:07:52] [I] Using random values for output descs
[07/31/2022-10:07:52] [I] Created output binding for descs with dimensions 1x2565x128
[07/31/2022-10:07:52] [I] Using random values for output kpts
[07/31/2022-10:07:52] [I] Created output binding for kpts with dimensions 1x2565x2
[07/31/2022-10:07:52] [I] Starting inference
[07/31/2022-10:07:55] [I] Warmup completed 210 queries over 200 ms
[07/31/2022-10:07:55] [I] Timing trace has 3524 queries over 3.00251 s
[07/31/2022-10:07:55] [I]
[07/31/2022-10:07:55] [I] === Trace details ===
[07/31/2022-10:07:55] [I] Trace averages of 10 runs:
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.859322 ms - Host latency: 1.24322 ms (enqueue 0.0853851 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.886624 ms - Host latency: 1.27203 ms (enqueue 0.0851501 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.887576 ms - Host latency: 1.27273 ms (enqueue 0.0850586 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.906545 ms - Host latency: 1.31155 ms (enqueue 0.0981308 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.851265 ms - Host latency: 1.24614 ms (enqueue 0.0927414 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833945 ms - Host latency: 1.21683 ms (enqueue 0.0826996 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843884 ms - Host latency: 1.23357 ms (enqueue 0.0887588 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836957 ms - Host latency: 1.23304 ms (enqueue 0.103876 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.848743 ms - Host latency: 1.25468 ms (enqueue 0.103326 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838507 ms - Host latency: 1.23635 ms (enqueue 0.110602 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.856219 ms - Host latency: 1.25658 ms (enqueue 0.100003 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837473 ms - Host latency: 1.22844 ms (enqueue 0.0931549 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.844672 ms - Host latency: 1.24457 ms (enqueue 0.0841492 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832016 ms - Host latency: 1.21906 ms (enqueue 0.0833954 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.856549 ms - Host latency: 1.26299 ms (enqueue 0.0895905 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840356 ms - Host latency: 1.23782 ms (enqueue 0.0828003 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835391 ms - Host latency: 1.21867 ms (enqueue 0.085907 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838684 ms - Host latency: 1.22591 ms (enqueue 0.0845184 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835736 ms - Host latency: 1.21844 ms (enqueue 0.084845 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.824088 ms - Host latency: 1.20951 ms (enqueue 0.0883972 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830377 ms - Host latency: 1.21364 ms (enqueue 0.0848358 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.825198 ms - Host latency: 1.21041 ms (enqueue 0.0879944 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.826975 ms - Host latency: 1.21212 ms (enqueue 0.0891357 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.825629 ms - Host latency: 1.2089 ms (enqueue 0.084552 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842841 ms - Host latency: 1.22822 ms (enqueue 0.0841461 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837549 ms - Host latency: 1.23329 ms (enqueue 0.0915497 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838528 ms - Host latency: 1.22437 ms (enqueue 0.085199 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830536 ms - Host latency: 1.21369 ms (enqueue 0.0863037 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832925 ms - Host latency: 1.22036 ms (enqueue 0.0900391 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834775 ms - Host latency: 1.21823 ms (enqueue 0.0831207 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828998 ms - Host latency: 1.21858 ms (enqueue 0.0882782 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840463 ms - Host latency: 1.23916 ms (enqueue 0.101453 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.845233 ms - Host latency: 1.25261 ms (enqueue 0.0965973 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838043 ms - Host latency: 1.22797 ms (enqueue 0.084845 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.86246 ms - Host latency: 1.28496 ms (enqueue 0.112051 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836343 ms - Host latency: 1.22968 ms (enqueue 0.0884583 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837421 ms - Host latency: 1.23841 ms (enqueue 0.0906982 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827765 ms - Host latency: 1.2217 ms (enqueue 0.0824036 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.847058 ms - Host latency: 1.23698 ms (enqueue 0.0850952 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836847 ms - Host latency: 1.22327 ms (enqueue 0.0854126 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835754 ms - Host latency: 1.2186 ms (enqueue 0.08479 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.853723 ms - Host latency: 1.26522 ms (enqueue 0.0907776 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83313 ms - Host latency: 1.22538 ms (enqueue 0.0969055 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830518 ms - Host latency: 1.21332 ms (enqueue 0.0853516 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.8349 ms - Host latency: 1.21843 ms (enqueue 0.0847107 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.852478 ms - Host latency: 1.24062 ms (enqueue 0.0872131 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829071 ms - Host latency: 1.21252 ms (enqueue 0.0853027 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835474 ms - Host latency: 1.22286 ms (enqueue 0.0851624 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831311 ms - Host latency: 1.21435 ms (enqueue 0.0846191 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842072 ms - Host latency: 1.23209 ms (enqueue 0.0908997 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838806 ms - Host latency: 1.22181 ms (enqueue 0.0851318 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.847821 ms - Host latency: 1.25037 ms (enqueue 0.105359 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834015 ms - Host latency: 1.22733 ms (enqueue 0.083136 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.844849 ms - Host latency: 1.23366 ms (enqueue 0.0888733 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.849872 ms - Host latency: 1.23564 ms (enqueue 0.0852966 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833441 ms - Host latency: 1.2165 ms (enqueue 0.0830872 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.924591 ms - Host latency: 1.34191 ms (enqueue 0.103687 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.823163 ms - Host latency: 1.20475 ms (enqueue 0.0827515 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829382 ms - Host latency: 1.21262 ms (enqueue 0.0825256 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834332 ms - Host latency: 1.21812 ms (enqueue 0.0824036 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833228 ms - Host latency: 1.21859 ms (enqueue 0.0854614 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836633 ms - Host latency: 1.22004 ms (enqueue 0.0821838 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.82868 ms - Host latency: 1.21219 ms (enqueue 0.0826416 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829584 ms - Host latency: 1.21762 ms (enqueue 0.0818848 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83006 ms - Host latency: 1.21465 ms (enqueue 0.0850342 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838202 ms - Host latency: 1.23391 ms (enqueue 0.0910584 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828882 ms - Host latency: 1.21259 ms (enqueue 0.0840637 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834589 ms - Host latency: 1.22388 ms (enqueue 0.0918884 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827295 ms - Host latency: 1.21053 ms (enqueue 0.0827881 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.845959 ms - Host latency: 1.23838 ms (enqueue 0.0887329 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830682 ms - Host latency: 1.21265 ms (enqueue 0.0824219 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833289 ms - Host latency: 1.23255 ms (enqueue 0.0954773 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.826154 ms - Host latency: 1.20916 ms (enqueue 0.0828064 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836102 ms - Host latency: 1.22457 ms (enqueue 0.0873901 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829858 ms - Host latency: 1.2126 ms (enqueue 0.0825317 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834741 ms - Host latency: 1.22208 ms (enqueue 0.0898682 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840527 ms - Host latency: 1.23145 ms (enqueue 0.0823486 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843439 ms - Host latency: 1.2346 ms (enqueue 0.0867188 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832532 ms - Host latency: 1.22645 ms (enqueue 0.0942383 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832129 ms - Host latency: 1.21588 ms (enqueue 0.083667 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831354 ms - Host latency: 1.2136 ms (enqueue 0.0828125 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832275 ms - Host latency: 1.21531 ms (enqueue 0.082721 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83382 ms - Host latency: 1.219 ms (enqueue 0.0854553 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835278 ms - Host latency: 1.21858 ms (enqueue 0.0821716 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.853497 ms - Host latency: 1.27411 ms (enqueue 0.105786 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830609 ms - Host latency: 1.21467 ms (enqueue 0.0827515 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.846484 ms - Host latency: 1.23685 ms (enqueue 0.0847046 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.8297 ms - Host latency: 1.21359 ms (enqueue 0.0842102 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.839795 ms - Host latency: 1.22993 ms (enqueue 0.0882019 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840765 ms - Host latency: 1.23369 ms (enqueue 0.0849793 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84856 ms - Host latency: 1.26513 ms (enqueue 0.110126 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.876874 ms - Host latency: 1.31198 ms (enqueue 0.105914 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.844049 ms - Host latency: 1.25071 ms (enqueue 0.0927063 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.901624 ms - Host latency: 1.29758 ms (enqueue 0.0908752 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83172 ms - Host latency: 1.21497 ms (enqueue 0.0856445 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834332 ms - Host latency: 1.22054 ms (enqueue 0.0878113 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83125 ms - Host latency: 1.21384 ms (enqueue 0.0851257 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.863159 ms - Host latency: 1.25837 ms (enqueue 0.0903931 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829565 ms - Host latency: 1.21222 ms (enqueue 0.0845459 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.879639 ms - Host latency: 1.26652 ms (enqueue 0.0863159 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840601 ms - Host latency: 1.23518 ms (enqueue 0.0894775 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830005 ms - Host latency: 1.21495 ms (enqueue 0.0847656 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.855969 ms - Host latency: 1.33939 ms (enqueue 0.0988647 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835327 ms - Host latency: 1.22085 ms (enqueue 0.0910889 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827295 ms - Host latency: 1.21193 ms (enqueue 0.0858887 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.82666 ms - Host latency: 1.21025 ms (enqueue 0.0844605 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83783 ms - Host latency: 1.22771 ms (enqueue 0.0938232 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827844 ms - Host latency: 1.21084 ms (enqueue 0.0840576 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.841052 ms - Host latency: 1.22742 ms (enqueue 0.0859619 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829785 ms - Host latency: 1.21326 ms (enqueue 0.0840454 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836353 ms - Host latency: 1.23527 ms (enqueue 0.100244 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838611 ms - Host latency: 1.23417 ms (enqueue 0.0828979 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.848975 ms - Host latency: 1.25283 ms (enqueue 0.103711 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833533 ms - Host latency: 1.2226 ms (enqueue 0.0829956 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829346 ms - Host latency: 1.21124 ms (enqueue 0.0821533 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835974 ms - Host latency: 1.22661 ms (enqueue 0.0830322 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830969 ms - Host latency: 1.21372 ms (enqueue 0.0828491 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.857263 ms - Host latency: 1.28945 ms (enqueue 0.0991089 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842749 ms - Host latency: 1.23348 ms (enqueue 0.0822754 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.853088 ms - Host latency: 1.27131 ms (enqueue 0.0949341 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833948 ms - Host latency: 1.22043 ms (enqueue 0.0870605 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833875 ms - Host latency: 1.22531 ms (enqueue 0.0857422 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.825049 ms - Host latency: 1.20707 ms (enqueue 0.0846069 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833215 ms - Host latency: 1.21714 ms (enqueue 0.0849365 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83678 ms - Host latency: 1.22214 ms (enqueue 0.0841919 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828503 ms - Host latency: 1.21194 ms (enqueue 0.0845825 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827576 ms - Host latency: 1.21047 ms (enqueue 0.0846191 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831665 ms - Host latency: 1.21471 ms (enqueue 0.0841309 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831457 ms - Host latency: 1.21527 ms (enqueue 0.0852661 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832117 ms - Host latency: 1.21449 ms (enqueue 0.0840088 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837939 ms - Host latency: 1.22415 ms (enqueue 0.0847778 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830176 ms - Host latency: 1.2139 ms (enqueue 0.0838257 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831213 ms - Host latency: 1.2146 ms (enqueue 0.0860962 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.822961 ms - Host latency: 1.20868 ms (enqueue 0.0834595 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830054 ms - Host latency: 1.21355 ms (enqueue 0.0828735 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830432 ms - Host latency: 1.21323 ms (enqueue 0.0816284 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832568 ms - Host latency: 1.2194 ms (enqueue 0.0872559 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835754 ms - Host latency: 1.22704 ms (enqueue 0.08396 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831616 ms - Host latency: 1.21447 ms (enqueue 0.0838013 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834436 ms - Host latency: 1.22274 ms (enqueue 0.0897705 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828882 ms - Host latency: 1.21113 ms (enqueue 0.0824707 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838953 ms - Host latency: 1.22628 ms (enqueue 0.0826782 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828052 ms - Host latency: 1.21205 ms (enqueue 0.0820801 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829907 ms - Host latency: 1.21862 ms (enqueue 0.0878052 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835913 ms - Host latency: 1.21791 ms (enqueue 0.0829346 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832397 ms - Host latency: 1.21688 ms (enqueue 0.082251 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.823804 ms - Host latency: 1.21432 ms (enqueue 0.0803467 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.863525 ms - Host latency: 1.2825 ms (enqueue 0.0903198 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.849744 ms - Host latency: 1.26742 ms (enqueue 0.094812 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.824084 ms - Host latency: 1.20725 ms (enqueue 0.0803833 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.82655 ms - Host latency: 1.20924 ms (enqueue 0.0800903 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829138 ms - Host latency: 1.21207 ms (enqueue 0.0799561 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843408 ms - Host latency: 1.2635 ms (enqueue 0.097998 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.82804 ms - Host latency: 1.21045 ms (enqueue 0.0807617 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828015 ms - Host latency: 1.21122 ms (enqueue 0.0806763 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828418 ms - Host latency: 1.21178 ms (enqueue 0.0808594 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.845337 ms - Host latency: 1.26279 ms (enqueue 0.0958618 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831995 ms - Host latency: 1.21536 ms (enqueue 0.0819824 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827942 ms - Host latency: 1.21389 ms (enqueue 0.0829956 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.839441 ms - Host latency: 1.24434 ms (enqueue 0.0817383 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.846704 ms - Host latency: 1.26348 ms (enqueue 0.109119 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830005 ms - Host latency: 1.2189 ms (enqueue 0.0919189 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.825879 ms - Host latency: 1.20835 ms (enqueue 0.0826538 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84104 ms - Host latency: 1.2292 ms (enqueue 0.0850342 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830029 ms - Host latency: 1.21318 ms (enqueue 0.0826172 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84021 ms - Host latency: 1.22241 ms (enqueue 0.0847046 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.825671 ms - Host latency: 1.20869 ms (enqueue 0.0822144 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834583 ms - Host latency: 1.21918 ms (enqueue 0.0835083 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830225 ms - Host latency: 1.21354 ms (enqueue 0.0820679 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.854334 ms - Host latency: 1.27435 ms (enqueue 0.10719 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.826697 ms - Host latency: 1.20953 ms (enqueue 0.0827881 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831457 ms - Host latency: 1.21637 ms (enqueue 0.0831787 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840698 ms - Host latency: 1.242 ms (enqueue 0.091272 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843127 ms - Host latency: 1.23904 ms (enqueue 0.0945801 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837549 ms - Host latency: 1.23268 ms (enqueue 0.0886963 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.824243 ms - Host latency: 1.20665 ms (enqueue 0.0825195 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84281 ms - Host latency: 1.24524 ms (enqueue 0.0898438 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827795 ms - Host latency: 1.20991 ms (enqueue 0.0820679 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.82627 ms - Host latency: 1.21797 ms (enqueue 0.0864502 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84646 ms - Host latency: 1.23036 ms (enqueue 0.0817139 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.822717 ms - Host latency: 1.20627 ms (enqueue 0.0834106 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.860059 ms - Host latency: 1.24708 ms (enqueue 0.0837891 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83938 ms - Host latency: 1.22191 ms (enqueue 0.0858765 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840784 ms - Host latency: 1.23271 ms (enqueue 0.0846069 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832605 ms - Host latency: 1.21559 ms (enqueue 0.085083 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.824805 ms - Host latency: 1.20996 ms (enqueue 0.0847412 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828223 ms - Host latency: 1.21141 ms (enqueue 0.0846069 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84939 ms - Host latency: 1.24041 ms (enqueue 0.0853516 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832678 ms - Host latency: 1.21617 ms (enqueue 0.0845337 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831787 ms - Host latency: 1.22168 ms (enqueue 0.0962524 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832239 ms - Host latency: 1.21715 ms (enqueue 0.0844238 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836414 ms - Host latency: 1.21998 ms (enqueue 0.084375 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829456 ms - Host latency: 1.21276 ms (enqueue 0.0842773 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.839575 ms - Host latency: 1.24779 ms (enqueue 0.108923 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843457 ms - Host latency: 1.23511 ms (enqueue 0.0834717 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831482 ms - Host latency: 1.21654 ms (enqueue 0.0855713 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.841089 ms - Host latency: 1.24626 ms (enqueue 0.0929321 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842285 ms - Host latency: 1.23074 ms (enqueue 0.0886597 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834741 ms - Host latency: 1.22548 ms (enqueue 0.0908936 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.82688 ms - Host latency: 1.21039 ms (enqueue 0.083374 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829871 ms - Host latency: 1.21349 ms (enqueue 0.0830688 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832446 ms - Host latency: 1.21677 ms (enqueue 0.0822388 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832727 ms - Host latency: 1.223 ms (enqueue 0.0895142 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827905 ms - Host latency: 1.2104 ms (enqueue 0.0820923 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838623 ms - Host latency: 1.22957 ms (enqueue 0.0838013 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833032 ms - Host latency: 1.21613 ms (enqueue 0.0821045 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83147 ms - Host latency: 1.22078 ms (enqueue 0.090918 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84906 ms - Host latency: 1.2694 ms (enqueue 0.0916382 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836707 ms - Host latency: 1.22567 ms (enqueue 0.0859131 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835107 ms - Host latency: 1.22338 ms (enqueue 0.0846313 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834888 ms - Host latency: 1.21793 ms (enqueue 0.0832397 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.851978 ms - Host latency: 1.28446 ms (enqueue 0.0956665 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833508 ms - Host latency: 1.2301 ms (enqueue 0.0862305 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829077 ms - Host latency: 1.21282 ms (enqueue 0.0830688 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.826477 ms - Host latency: 1.21061 ms (enqueue 0.0822998 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830774 ms - Host latency: 1.22504 ms (enqueue 0.0905273 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830835 ms - Host latency: 1.21519 ms (enqueue 0.082019 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831982 ms - Host latency: 1.21604 ms (enqueue 0.082666 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.839136 ms - Host latency: 1.23425 ms (enqueue 0.0901855 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827686 ms - Host latency: 1.21062 ms (enqueue 0.082373 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.860083 ms - Host latency: 1.28042 ms (enqueue 0.0949951 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.8323 ms - Host latency: 1.21609 ms (enqueue 0.0834717 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.844287 ms - Host latency: 1.23547 ms (enqueue 0.0877686 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831152 ms - Host latency: 1.21455 ms (enqueue 0.0828125 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83562 ms - Host latency: 1.22092 ms (enqueue 0.0844971 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829712 ms - Host latency: 1.21309 ms (enqueue 0.0822266 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.85166 ms - Host latency: 1.24553 ms (enqueue 0.0846436 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835083 ms - Host latency: 1.22151 ms (enqueue 0.0829102 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837891 ms - Host latency: 1.22278 ms (enqueue 0.0889648 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843823 ms - Host latency: 1.23188 ms (enqueue 0.0839355 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.839087 ms - Host latency: 1.23643 ms (enqueue 0.0982422 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828711 ms - Host latency: 1.21157 ms (enqueue 0.0838379 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837036 ms - Host latency: 1.22014 ms (enqueue 0.0825684 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.846606 ms - Host latency: 1.2481 ms (enqueue 0.0932861 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833008 ms - Host latency: 1.22271 ms (enqueue 0.087207 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842822 ms - Host latency: 1.24768 ms (enqueue 0.0946533 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827319 ms - Host latency: 1.21104 ms (enqueue 0.0823975 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.887769 ms - Host latency: 1.36157 ms (enqueue 0.103345 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.862524 ms - Host latency: 1.27756 ms (enqueue 0.0913574 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834155 ms - Host latency: 1.23677 ms (enqueue 0.090332 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836133 ms - Host latency: 1.22549 ms (enqueue 0.0843994 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833081 ms - Host latency: 1.2166 ms (enqueue 0.0843262 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840698 ms - Host latency: 1.23267 ms (enqueue 0.0935059 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834692 ms - Host latency: 1.2189 ms (enqueue 0.0838867 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.86814 ms - Host latency: 1.25515 ms (enqueue 0.0849854 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.8354 ms - Host latency: 1.21902 ms (enqueue 0.0838623 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833423 ms - Host latency: 1.22114 ms (enqueue 0.0847656 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835596 ms - Host latency: 1.22009 ms (enqueue 0.0844238 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835547 ms - Host latency: 1.22056 ms (enqueue 0.0854492 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837036 ms - Host latency: 1.21936 ms (enqueue 0.0842773 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833472 ms - Host latency: 1.22209 ms (enqueue 0.0941162 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.852734 ms - Host latency: 1.25601 ms (enqueue 0.0841797 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831982 ms - Host latency: 1.21462 ms (enqueue 0.0817139 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.823267 ms - Host latency: 1.20984 ms (enqueue 0.0813477 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828247 ms - Host latency: 1.21101 ms (enqueue 0.080542 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.846338 ms - Host latency: 1.24124 ms (enqueue 0.0855469 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834302 ms - Host latency: 1.22117 ms (enqueue 0.0799316 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832105 ms - Host latency: 1.22332 ms (enqueue 0.0886475 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827441 ms - Host latency: 1.21021 ms (enqueue 0.0802734 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.853833 ms - Host latency: 1.25996 ms (enqueue 0.101245 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840527 ms - Host latency: 1.2418 ms (enqueue 0.102295 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.846533 ms - Host latency: 1.25808 ms (enqueue 0.104297 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843579 ms - Host latency: 1.24536 ms (enqueue 0.102295 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837549 ms - Host latency: 1.24902 ms (enqueue 0.11499 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.850586 ms - Host latency: 1.25159 ms (enqueue 0.10437 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.859717 ms - Host latency: 1.28196 ms (enqueue 0.12063 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842871 ms - Host latency: 1.25647 ms (enqueue 0.116211 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836279 ms - Host latency: 1.23013 ms (enqueue 0.103198 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83877 ms - Host latency: 1.23752 ms (enqueue 0.103418 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.847217 ms - Host latency: 1.27427 ms (enqueue 0.12207 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837036 ms - Host latency: 1.24502 ms (enqueue 0.117725 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834644 ms - Host latency: 1.23037 ms (enqueue 0.105566 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837769 ms - Host latency: 1.24167 ms (enqueue 0.111035 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.849097 ms - Host latency: 1.25066 ms (enqueue 0.103589 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837012 ms - Host latency: 1.22368 ms (enqueue 0.0866699 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.839429 ms - Host latency: 1.22771 ms (enqueue 0.0900147 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830762 ms - Host latency: 1.21409 ms (enqueue 0.084375 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.858447 ms - Host latency: 1.29075 ms (enqueue 0.0987549 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.844263 ms - Host latency: 1.24761 ms (enqueue 0.10625 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.856323 ms - Host latency: 1.27632 ms (enqueue 0.10105 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840283 ms - Host latency: 1.23445 ms (enqueue 0.086377 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.839526 ms - Host latency: 1.23699 ms (enqueue 0.0917969 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834082 ms - Host latency: 1.21821 ms (enqueue 0.0840332 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832935 ms - Host latency: 1.21506 ms (enqueue 0.0844482 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831714 ms - Host latency: 1.2146 ms (enqueue 0.0837158 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.844043 ms - Host latency: 1.23508 ms (enqueue 0.0981201 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84353 ms - Host latency: 1.23154 ms (enqueue 0.083252 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843994 ms - Host latency: 1.25059 ms (enqueue 0.100586 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836694 ms - Host latency: 1.21963 ms (enqueue 0.0837646 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830542 ms - Host latency: 1.21228 ms (enqueue 0.0830322 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838672 ms - Host latency: 1.22827 ms (enqueue 0.0833252 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835303 ms - Host latency: 1.22607 ms (enqueue 0.0852051 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830957 ms - Host latency: 1.22275 ms (enqueue 0.0892822 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828638 ms - Host latency: 1.21067 ms (enqueue 0.0824219 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83584 ms - Host latency: 1.22908 ms (enqueue 0.0960449 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831714 ms - Host latency: 1.21516 ms (enqueue 0.0822022 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836694 ms - Host latency: 1.21973 ms (enqueue 0.0834961 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832959 ms - Host latency: 1.22087 ms (enqueue 0.0822266 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833423 ms - Host latency: 1.22505 ms (enqueue 0.0906494 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.841968 ms - Host latency: 1.25115 ms (enqueue 0.0937012 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.851416 ms - Host latency: 1.26257 ms (enqueue 0.110278 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842554 ms - Host latency: 1.24771 ms (enqueue 0.0902344 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834863 ms - Host latency: 1.21853 ms (enqueue 0.0855713 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834644 ms - Host latency: 1.21873 ms (enqueue 0.0829346 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831226 ms - Host latency: 1.21499 ms (enqueue 0.0823486 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.901685 ms - Host latency: 1.28892 ms (enqueue 0.0876221 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832471 ms - Host latency: 1.21553 ms (enqueue 0.0826172 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829224 ms - Host latency: 1.21357 ms (enqueue 0.0890137 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827173 ms - Host latency: 1.21021 ms (enqueue 0.0822754 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830151 ms - Host latency: 1.21355 ms (enqueue 0.0831055 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830884 ms - Host latency: 1.21467 ms (enqueue 0.0822022 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.854419 ms - Host latency: 1.28079 ms (enqueue 0.119238 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831274 ms - Host latency: 1.21978 ms (enqueue 0.0849365 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.832373 ms - Host latency: 1.2167 ms (enqueue 0.083374 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.840161 ms - Host latency: 1.22998 ms (enqueue 0.0827148 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831934 ms - Host latency: 1.21558 ms (enqueue 0.0825195 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838452 ms - Host latency: 1.23484 ms (enqueue 0.0985107 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831519 ms - Host latency: 1.21516 ms (enqueue 0.0822754 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.84165 ms - Host latency: 1.2355 ms (enqueue 0.0889404 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831055 ms - Host latency: 1.21545 ms (enqueue 0.0821045 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.831885 ms - Host latency: 1.21841 ms (enqueue 0.0866699 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.826978 ms - Host latency: 1.21055 ms (enqueue 0.0817871 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838721 ms - Host latency: 1.22283 ms (enqueue 0.0838867 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836816 ms - Host latency: 1.23162 ms (enqueue 0.0831543 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842847 ms - Host latency: 1.24116 ms (enqueue 0.0947021 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.833057 ms - Host latency: 1.21687 ms (enqueue 0.0821777 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829883 ms - Host latency: 1.21526 ms (enqueue 0.0871094 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835864 ms - Host latency: 1.23181 ms (enqueue 0.0931152 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828931 ms - Host latency: 1.21106 ms (enqueue 0.0821045 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.849536 ms - Host latency: 1.25818 ms (enqueue 0.0858398 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.838379 ms - Host latency: 1.23892 ms (enqueue 0.0863525 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.843872 ms - Host latency: 1.24458 ms (enqueue 0.0919922 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828955 ms - Host latency: 1.21326 ms (enqueue 0.0821777 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830762 ms - Host latency: 1.21721 ms (enqueue 0.0829346 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.823364 ms - Host latency: 1.20754 ms (enqueue 0.0821777 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835498 ms - Host latency: 1.22427 ms (enqueue 0.0867432 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830493 ms - Host latency: 1.21829 ms (enqueue 0.0818848 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.835034 ms - Host latency: 1.22585 ms (enqueue 0.086084 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.849927 ms - Host latency: 1.29221 ms (enqueue 0.0899414 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.834277 ms - Host latency: 1.2312 ms (enqueue 0.0897461 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.836182 ms - Host latency: 1.23203 ms (enqueue 0.0893799 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.83457 ms - Host latency: 1.21765 ms (enqueue 0.0828125 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829785 ms - Host latency: 1.2179 ms (enqueue 0.0825928 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830591 ms - Host latency: 1.21311 ms (enqueue 0.0823486 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.837012 ms - Host latency: 1.23003 ms (enqueue 0.090332 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.828931 ms - Host latency: 1.21121 ms (enqueue 0.0816895 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.842188 ms - Host latency: 1.25808 ms (enqueue 0.109253 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.830713 ms - Host latency: 1.21748 ms (enqueue 0.0827393 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.841187 ms - Host latency: 1.24834 ms (enqueue 0.106104 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.827417 ms - Host latency: 1.21809 ms (enqueue 0.0899902 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.825659 ms - Host latency: 1.20979 ms (enqueue 0.0828857 ms)
[07/31/2022-10:07:55] [I] Average on 10 runs - GPU latency: 0.829321 ms - Host latency: 1.21692 ms (enqueue 0.0875488 ms)
[07/31/2022-10:07:55] [I]
[07/31/2022-10:07:55] [I] === Performance summary ===
[07/31/2022-10:07:55] [I] Throughput: 1173.68 qps
[07/31/2022-10:07:55] [I] Latency: min = 1.15747 ms, max = 2.0061 ms, mean = 1.2299 ms, median = 1.21655 ms, percentile(99%) = 1.42554 ms
[07/31/2022-10:07:55] [I] Enqueue Time: min = 0.0787354 ms, max = 0.267578 ms, mean = 0.0883669 ms, median = 0.0834961 ms, percentile(99%) = 0.17807 ms
[07/31/2022-10:07:55] [I] H2D Latency: min = 0.245361 ms, max = 0.545654 ms, mean = 0.264068 ms, median = 0.259033 ms, percentile(99%) = 0.372803 ms
[07/31/2022-10:07:55] [I] GPU Compute Time: min = 0.776611 ms, max = 1.62408 ms, mean = 0.838034 ms, median = 0.831894 ms, percentile(99%) = 0.944931 ms
[07/31/2022-10:07:55] [I] D2H Latency: min = 0.115479 ms, max = 0.33667 ms, mean = 0.127797 ms, median = 0.124512 ms, percentile(99%) = 0.208435 ms
[07/31/2022-10:07:55] [I] Total Host Walltime: 3.00251 s
[07/31/2022-10:07:55] [I] Total GPU Compute Time: 2.95323 s
[07/31/2022-10:07:55] [W] * GPU compute time is unstable, with coefficient of variance = 4.18864%.
[07/31/2022-10:07:55] [W] If not already in use, locking GPU clock frequency or adding --useSpinWait may improve the stability.
[07/31/2022-10:07:55] [I] Explanations of the performance metrics are printed in the verbose logs.
[07/31/2022-10:07:55] [I]
&&&& PASSED TensorRT.trtexec [TensorRT v8402] # trtexec.exe --onnx=C:/projects/Dleware/Testers/Playground/Models/ASLtorch/Delivery/model2_folded.onnx --saveEngine=model2_folded.engine --minShapes=keypoints:1x1x2,scores.1:1x1,score_map:1x1x320x256,dense_feat_map:1x128x80x64 --optShapes=keypoints:1x2565x2,scores.1:1x2565,score_map:1x1x320x256,dense_feat_map:1x128x80x64 --maxShapes=keypoints:1x8000x2,scores.1:1x8000,score_map:1x1x320x256,dense_feat_map:1x128x190x173 --workspace=30000