Description
Our model is not able to run in TensorRT, I have tried both running in ORT with TensorrtExecutionProvider and using trtexec to convert it to a plan.
For compliance reason, I cannot share my full model, but I can build a mini one to repro my issue.
please check the model binary directly:
tiny_model.onnx (15.9 KB)
OR build from the script:
tiny_model_builder.py (1.1 KB)
-
Using ORT + TensorrtExecutionProvider reports error:
Error Code 4: Internal Error ((Unnamed Layer* 23) [LoopOutput]: an ILoopOutputLayer cannot be used to compute a shape tensor)
ORT_repro.py (690 Bytes) -
Using
trtexec --onnx=tiny_model.onnx --saveEngine=tiny_model.trt --memPoolSize=workspace:10000 --minShapes=input:1x1 --maxShapes=input:1x512 --optShapes=input:1x64 --device=0 --verbosewill show below error:
[02/21/2023-14:36:04] [E] Error[2]: [topSort.cpp::trivialChoice::314] Error Code 2: Internal Error (Assertion c != kNoColor failed. )
[02/21/2023-14:36:04] [E] Error[2]: [builder.cpp::buildSerializedNetwork::738] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
Environment
TensorRT Version: 8.5.0
GPU Type: T4
Nvidia Driver Version: 470.161.03
CUDA Version: 11.8
CUDNN Version:
Operating System + Version: Ubuntu20.04
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
Relevant Files
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
Steps To Reproduce
Please include:
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered