Description
I’m using python polygraphy API to convert an ONNX model exported from PyTorch to a TensorRT Engine. The code consists of the two following lines of code:
build_engine = EngineFromNetwork(NetworkFromOnnxPath(onnx_file))
engine = build_engine()
The parsing is successful but building the engine causes lots of warnings
[02/02/2022-18:28:37] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[02/02/2022-18:28:37] [TRT] [W] Tensor DataType is determined at build time for tensors not marked as input or output.
[02/02/2022-18:30:36] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[02/02/2022-18:30:48] [TRT] [W] Output type must be INT32 for shape outputs
[02/02/2022-18:30:48] [TRT] [W] Output type must be INT32 for shape outputs
[02/02/2022-18:30:48] [TRT] [W] Output type must be INT32 for shape outputs
[02/02/2022-18:30:48] [TRT] [W] Output type must be INT32 for shape outputs
[02/02/2022-18:30:48] [TRT] [W] Output type must be INT32 for shape outputs
(Tensor DataType is determined at build time for tensors not marked as input or output.
appears about 600 times) before eventually failing with the following error:
[I] Building engine with configuration:
Workspace | 16777216 bytes (16.00 MiB)
Precision | TF32: False, FP16: False, INT8: False, Obey Precision Constraints: False, Strict Types: False
Tactic Sources | ['CUBLAS', 'CUBLAS_LT', 'CUDNN']
Safety Restricted | False
Profiles | 1 profile(s)
[02/02/2022-18:39:05] [TRT] [E] 2: [ltWrapper.cpp::nvinfer1::rt::CublasLtWrapper::setupHeuristic::327] Error Code 2: Internal Error (Assertion cublasStatus == CUBLAS_STATUS_SUCCESS failed. )
[02/02/2022-18:39:05] [TRT] [E] 2: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
I have two questions:
-
Are all those warnings an actual problem potentially resulting in a slow-down of the inference? In that case, how can I get rid of them?
-
How can solve the error? I read it could be caused by not having CUDA10.2 patches installed but installing them didn’t solve the problem.
Environment
TensorRT Version: 8.2.3.0
NVIDIA GPU: GeForce GTX 1650 Ti
NVIDIA Driver Version: 30.0.14.9649
CUDA Version: 10.2
CUDNN Version: 8.2.1.32
Operating System: Windows 10
Python Version (if applicable): 3.7
PyTorch Version (if applicable): 1.10.1
Baremetal or Container (if so, version): Baremetal
Relevant Files
pcr.onnx (45.8 MB)