Description
I have a neural network that originally wrote in Pytorch
I have converted it to Onnx, and then to TensorRT
I’ve tried to use onnx checker,and it passes
then I use
“trtexec --onnx=E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx --workspace=9000”,the output is right.
While when I tried to save the engine using saveEngine like this:
trtexec --onnx=E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx --saveEngine=E:\RSVP\model\hdr\hdr.engine --workspace=9000
it returns nothing,and the engine file is Null.
Environment
TensorRT Version: 7.0.0.11.Windows10.x86_64.cuda-10.0.cudnn7.6
GPU Type: GeForce 1080ti
Nvidia Driver Version: 26.21.14.3200
CUDA Version: 10.0
CUDNN Version: 7.6.3
Operating System + Version: Windows10
Python Version (if applicable): 3.6.9
PyTorch Version (if applicable): 1.3
Relevant Files
The model convert with opset10 and pytorch1.3 is below:
hdr.zip (2.0 MB)
Steps To Reproduce
the output of" trtexec --onnx=E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx --workspace=1600"
&&&& RUNNING TensorRT.trtexec # trtexec --onnx=E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx --workspace=1600
[10/09/2020-16:12:05] [I] === Model Options ===
[10/09/2020-16:12:05] [I] Format: ONNX
[10/09/2020-16:12:05] [I] Model: E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx
[10/09/2020-16:12:05] [I] Output:
[10/09/2020-16:12:05] [I] === Build Options ===
[10/09/2020-16:12:05] [I] Max batch: 1
[10/09/2020-16:12:05] [I] Workspace: 1600 MB
[10/09/2020-16:12:05] [I] minTiming: 1
[10/09/2020-16:12:05] [I] avgTiming: 8
[10/09/2020-16:12:05] [I] Precision: FP32
[10/09/2020-16:12:05] [I] Calibration:
[10/09/2020-16:12:05] [I] Safe mode: Disabled
[10/09/2020-16:12:05] [I] Save engine:
[10/09/2020-16:12:05] [I] Load engine:
[10/09/2020-16:12:05] [I] Inputs format: fp32:CHW
[10/09/2020-16:12:05] [I] Outputs format: fp32:CHW
[10/09/2020-16:12:05] [I] Input build shapes: model
[10/09/2020-16:12:05] [I] === System Options ===
[10/09/2020-16:12:05] [I] Device: 0
[10/09/2020-16:12:05] [I] DLACore:
[10/09/2020-16:12:05] [I] Plugins:
[10/09/2020-16:12:05] [I] === Inference Options ===
[10/09/2020-16:12:05] [I] Batch: 1
[10/09/2020-16:12:05] [I] Iterations: 10
[10/09/2020-16:12:05] [I] Duration: 3s (+ 200ms warm up)
[10/09/2020-16:12:05] [I] Sleep time: 0ms
[10/09/2020-16:12:05] [I] Streams: 1
[10/09/2020-16:12:05] [I] ExposeDMA: Disabled
[10/09/2020-16:12:05] [I] Spin-wait: Disabled
[10/09/2020-16:12:05] [I] Multithreading: Disabled
[10/09/2020-16:12:05] [I] CUDA Graph: Disabled
[10/09/2020-16:12:05] [I] Skip inference: Disabled
[10/09/2020-16:12:05] [I] Input inference shapes: model
[10/09/2020-16:12:05] [I] Inputs:
[10/09/2020-16:12:05] [I] === Reporting Options ===
[10/09/2020-16:12:05] [I] Verbose: Disabled
[10/09/2020-16:12:05] [I] Averages: 10 inferences
[10/09/2020-16:12:05] [I] Percentile: 99
[10/09/2020-16:12:05] [I] Dump output: Disabled
[10/09/2020-16:12:05] [I] Profile: Disabled
[10/09/2020-16:12:05] [I] Export timing to JSON file:
[10/09/2020-16:12:05] [I] Export output to JSON file:
[10/09/2020-16:12:05] [I] Export profile to JSON file:
[10/09/2020-16:12:05] [I]
Input filename: E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx
ONNX IR version: 0.0.6
Opset version: 10
Producer name: pytorch
Producer version: 1.5
Domain:
Model version: 0
Doc string:
[10/09/2020-16:15:26] [I] [TRT] Detected 2 inputs and 1 output network tensors.
[10/09/2020-16:15:26] [W] [TRT] Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
[10/09/2020-16:15:29] [I] Warmup completed 22 queries over 200 ms
[10/09/2020-16:15:29] [I] Timing trace has 330 queries over 3.02023 s
[10/09/2020-16:15:29] [I] Trace averages of 10 runs:
…
[10/09/2020-16:15:30] [I] Average on 10 runs - GPU latency: 9.18462 ms - Host latency: 9.53843 ms (end to end 12.7199 ms)
[10/09/2020-16:15:30] [I] Host latency
[10/09/2020-16:15:30] [I] min: 8.35626 ms (end to end 11.4463 ms)
[10/09/2020-16:15:30] [I] max: 13.0778 ms (end to end 15.7726 ms)
[10/09/2020-16:15:30] [I] mean: 9.49547 ms (end to end 12.6458 ms)
[10/09/2020-16:15:30] [I] median: 9.40466 ms (end to end 12.5342 ms)
[10/09/2020-16:15:30] [I] percentile: 11.2314 ms at 99% (end to end 14.2551 ms at 99%)
[10/09/2020-16:15:30] [I] throughput: 109.263 qps
[10/09/2020-16:15:30] [I] walltime: 3.02023 s
[10/09/2020-16:15:30] [I] GPU Compute
[10/09/2020-16:15:30] [I] min: 8.00256 ms
[10/09/2020-16:15:30] [I] max: 12.6981 ms
[10/09/2020-16:15:30] [I] mean: 9.14056 ms
[10/09/2020-16:15:30] [I] median: 9.05035 ms
[10/09/2020-16:15:30] [I] percentile: 10.8799 ms at 99%
[10/09/2020-16:15:30] [I] total compute time: 3.01639 s
&&&& PASSED TensorRT.trtexec # trtexec --onnx=E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx --workspace=1600
the output of"trtexec --onnx=E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx --saveEngine=E:\RSVP\model\hdr\hdr.engine --workspace=9000"
Input filename: E:\ToLMD\HDR-Neuro-0929-1080ti\hdr.onnx
ONNX IR version: 0.0.6
Opset version: 10
Producer name: pytorch
Producer version: 1.5
Domain:
Model version: 0
Doc string:
[10/09/2020-16:29:27] [I] [TRT] Detected 2 inputs and 1 output network tensors.
I don’t know how to solve it.Thanks a lot.