Builder.build_cuda_engine fails to build engine but prints no errors

tomendarias · September 29, 2021, 2:36pm

Hello!

I am trying to convert an ONNX version of a ReIdentification vehicle network to TensorRT. I am running this script for the conversion from ONNX to TensorRT. However, the engine is never created. I have found this error in other posts, but the solution suggested is to use trtexec…however, I was wondering how to solve it with Python.

My ONNX model is here.

I am running this on a Jetson Xavier NX:
TensorRT = 7.1.3.0
PyCUDA = built from this wheel
Pytorch = 1.6.0

I appreciate your help =)

NVES · September 29, 2021, 3:07pm

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

tomendarias · September 30, 2021, 8:07am

@NVES Thanks for the quick reply. As you can see in my original post, I shared my ONNX file since the beginning.

Today I was able to successfully convert the model using trtexec. However, If I try to load it into pytorch (torch.load) it gives me the following error:

“UnpicklingError: unpickling stack underflow”

Could you please tell me the easiest way to load my trained trt model into pytorch?

Thanks! :)

spolisetty · September 30, 2021, 10:24am

Hi,

Hope following sample will help you, which tells how to use trt engine for inference.

github.com

NVIDIA/TensorRT/blob/master/samples/python/network_api_pytorch_mnist/sample.py#L112


      
              engine = build_engine(weights)
          
          
    # Build an engine, allocate buffers and create a stream.
              # For more information on buffer allocation, refer to the introductory samples.
              inputs, outputs, bindings, stream = common.allocate_buffers(engine)
              context = engine.create_execution_context()
          
          
    case_num = load_random_test_case(mnist_model, pagelocked_buffer=inputs[0].host)
              # For more information on performing inference, refer to the introductory samples.
              # The common.do_inference function will return a list of outputs - we only have one in this case.
              [output] = common.do_inference_v2(context, bindings=bindings, inputs=inputs, outputs=outputs, stream=stream)
              pred = np.argmax(output)
              print("Test Case: " + str(case_num))
              print("Prediction: " + str(pred))
          
          
if __name__ == '__main__':
              main()

Thank you.

Topic		Replies	Views
FAILED TensorRT.trtexec TensorRT	1	2423	October 4, 2021
PyTorch FCN-ResNet50 --> ONNX --> TensorRT TensorRT	3	946	February 17, 2022
Conversion from onnx to TensorRT engine TensorRT tensorrt , cuda	1	476	July 24, 2023
Build engine from onnx failed TensorRT	2	959	December 14, 2021
Runtime.deserialize_cuda_engine return a NoneType, how to fix ti? TensorRT tensorrt	10	2299	July 15, 2022
Convet onnx to trt engine got error TensorRT	3	1188	January 7, 2022
Simple ResNet model from PyTorch - "nan" Output TensorRT	1	1553	April 9, 2021
Trouble deserialising a trt engine file TensorRT	1	1480	September 5, 2021
Builder.build_cuda_engine(network) silently returns None TensorRT	4	2731	January 10, 2024
How to serialize a model and do inference in Python TensorRT	5	2258	July 8, 2020

Builder.build_cuda_engine fails to build engine but prints no errors

check_model.py

Related topics