TRT conversion of T5-base model

I am facing issue while converting T5 base model using the steps in the blog Optimizing T5 and GPT-2 for Real-Time Inference with NVIDIA TensorRT | NVIDIA Technical Blog. I was able to convert the T5 -small model to TRT using the above blog and the associated notebook.

Below is the issue I am facing when converting T5-base to TRT:
PolygraphyException Traceback (most recent call last)
in ()
1 t5_trt_encoder_engine = T5EncoderONNXFile(
2 os.path.join(onnx_model_path, encoder_onnx_model_fpath), metadata
----> 3 ).as_trt_engine(os.path.join(tensorrt_model_path, encoder_onnx_model_fpath) + “.engine”)
4
5 t5_trt_decoder_engine = T5DecoderONNXFile(

8 frames
in func_impl(network, config, save_timing_cache)

in func_impl(serialized_engine)

/usr/local/lib/python3.7/dist-packages/polygraphy/logger/logger.py in critical(self, message)
347 from polygraphy.exception import PolygraphyException
348
→ 349 raise PolygraphyException(message) from None
350
351 def internal_error(self, message):

PolygraphyException: Invalid Engine. Please ensure the engine was built correctly

I have also tried increasing precision to FP 32. But still getting the same issue.

Edit: The problem was coming because of dependencies issue. Using the docker container method in the article we can convert T5-base model as well.

Hi @user158228 , I have the same problem. I tried with my own trained t5-base. How did you solve the problem?

Moved to the TensorRT forum

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

have you solved the problem?I met the same issue

Try using TensorRT docker container or following official installation instruction from NVidia site which involves installing TensorRT through tar. This error pops up if installation is not proper.