$ ./bin/trtexec --onnx=./data/mnist/mnist.onnx --dumpProfile
&&&& RUNNING TensorRT.trtexec # ./bin/trtexec --onnx=./data/mnist/mnist.onnx --dumpProfile
[03/22/2021-06:47:46] [I] === Model Options ===
[03/22/2021-06:47:46] [I] Format: ONNX
[03/22/2021-06:47:46] [I] Model: ./data/mnist/mnist.onnx
[03/22/2021-06:47:46] [I] Output:
[03/22/2021-06:47:46] [I] === Build Options ===
[03/22/2021-06:47:46] [I] Max batch: 1
[03/22/2021-06:47:46] [I] Workspace: 16 MB
[03/22/2021-06:47:46] [I] minTiming: 1
[03/22/2021-06:47:46] [I] avgTiming: 8
[03/22/2021-06:47:46] [I] Precision: FP32
[03/22/2021-06:47:46] [I] Calibration: 
[03/22/2021-06:47:46] [I] Safe mode: Disabled
[03/22/2021-06:47:46] [I] Save engine: 
[03/22/2021-06:47:46] [I] Load engine: 
[03/22/2021-06:47:46] [I] Inputs format: fp32:CHW
[03/22/2021-06:47:46] [I] Outputs format: fp32:CHW
[03/22/2021-06:47:46] [I] Input build shapes: model
[03/22/2021-06:47:46] [I] === System Options ===
[03/22/2021-06:47:46] [I] Device: 0
[03/22/2021-06:47:46] [I] DLACore: 
[03/22/2021-06:47:46] [I] Plugins:
[03/22/2021-06:47:46] [I] === Inference Options ===
[03/22/2021-06:47:46] [I] Batch: 1
[03/22/2021-06:47:46] [I] Iterations: 10 (200 ms warm up)
[03/22/2021-06:47:46] [I] Duration: 10s
[03/22/2021-06:47:46] [I] Sleep time: 0ms
[03/22/2021-06:47:46] [I] Streams: 1
[03/22/2021-06:47:46] [I] Spin-wait: Disabled
[03/22/2021-06:47:46] [I] Multithreading: Enabled
[03/22/2021-06:47:46] [I] CUDA Graph: Disabled
[03/22/2021-06:47:46] [I] Skip inference: Disabled
[03/22/2021-06:47:46] [I] Input inference shapes: model
[03/22/2021-06:47:46] [I] === Reporting Options ===
[03/22/2021-06:47:46] [I] Verbose: Disabled
[03/22/2021-06:47:46] [I] Averages: 10 inferences
[03/22/2021-06:47:46] [I] Percentile: 99
[03/22/2021-06:47:46] [I] Dump output: Disabled
[03/22/2021-06:47:46] [I] Profile: Enabled
[03/22/2021-06:47:46] [I] Export timing to JSON file: 
[03/22/2021-06:47:46] [I] Export profile to JSON file: 
[03/22/2021-06:47:46] [I] 
----------------------------------------------------------------
Input filename:   ./data/mnist/mnist.onnx
ONNX IR version:  0.0.3
Opset version:    8
Producer name:    CNTK
Producer version: 2.5.1
Domain:           ai.cntk
Model version:    1
Doc string:       
----------------------------------------------------------------
[03/22/2021-06:47:48] [I] [TRT] Detected 1 inputs and 1 output network tensors.
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0659616 ms (host walltime is 0.153658 ms, 99% percentile time is 0.125632).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0587616 ms (host walltime is 0.0862769 ms, 99% percentile time is 0.062528).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0596704 ms (host walltime is 0.0905902 ms, 99% percentile time is 0.093792).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0500096 ms (host walltime is 0.0712584 ms, 99% percentile time is 0.054944).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.049408 ms (host walltime is 0.0621439 ms, 99% percentile time is 0.054656).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0519808 ms (host walltime is 0.06239 ms, 99% percentile time is 0.062656).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0499744 ms (host walltime is 0.0604112 ms, 99% percentile time is 0.054176).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0537184 ms (host walltime is 0.0673972 ms, 99% percentile time is 0.07648).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0496512 ms (host walltime is 0.0626041 ms, 99% percentile time is 0.057376).
[03/22/2021-06:47:48] [I] Average over 10 runs is 0.0526688 ms (host walltime is 0.0689098 ms, 99% percentile time is 0.060896).
[03/22/2021-06:47:48] [I] ========== Layer time profile ==========
[03/22/2021-06:47:48] [I]                                                    TensorRT layer name    Runtime, %  Invocations  Runtime, ms
[03/22/2021-06:47:48] [I]     (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation]         17.5%          100         0.64
[03/22/2021-06:47:48] [I]                                           (Unnamed Layer* 3) [Pooling]         12.6%          100         0.46
[03/22/2021-06:47:48] [I]     (Unnamed Layer* 4) [Convolution] + (Unnamed Layer* 6) [Activation]         17.4%          100         0.64
[03/22/2021-06:47:48] [I]                                           (Unnamed Layer* 7) [Pooling]         11.5%          100         0.42
[03/22/2021-06:47:48] [I]                                          (Unnamed Layer* 9) [Constant]          4.6%          100         0.17
[03/22/2021-06:47:48] [I]                                  (Unnamed Layer* 10) [Matrix Multiply]         14.2%          100         0.52
[03/22/2021-06:47:48] [I]                                            (Unnamed Layer* 12) [Scale]         11.4%          100         0.42
[03/22/2021-06:47:48] [I]                                          (Unnamed Layer* 13) [Shuffle]         10.8%          100         0.40
[03/22/2021-06:47:48] [I] ========== Layer time total runtime = 3.67619 ms ==========
&&&& PASSED TensorRT.trtexec # ./bin/trtexec --onnx=./data/mnist/mnist.onnx --dumpProfile