Use tensorrt to inference met error

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

[TensorRT] ERROR: INVALID_ARGUMENT: Cannot deserialize with an empty memory buffer.
[TensorRT] ERROR: INVALID_CONFIG: Deserialize the cuda engine failed.
Traceback (most recent call last):
File “inference.py”, line 90, in
trt_infer()
File “inference.py”, line 82, in trt_infer
context = engine.create_execution_context()
AttributeError: ‘NoneType’ object has no attribute ‘create_execution_context’

3

Hi,

The error indicates your buffer is empty.
In general, we deserialize TensorRT engine as following:

with open("xxx.plan", 'rb') as f:
    buf = f.read()
    engine = runtime.deserialize_cuda_engine(buf)

Could you double check if your inference.py do read the input data correctly first?
Thanks.

4

thank you ,i have solved the problem,my tensortrt is 7.1.3.0, i try to use onnx-smiplifier to smiplify onnx ,but i met the following error:
[E] [TRT] Network must have at least one output
[05/12/2021-14:11:34] [E] [TRT] Network validation failed.
[05/12/2021-14:11:34] [E] Engine creation failed
[05/12/2021-14:11:34] [E] Engine set up failed
but if i am not smiplify the onnx ,i could use convert onnx to tensorrt,

5

Hi,

The error indicates TensorRT cannot find a valid output on your ONNX model.

Does the model follow the standard ONNX format after onnx-smiplifier?
And does it work by running TensorRT with the original ONNX format?

If yes, please do so.
TensorRT will merge and combine all the possible operations, so you can an optimized output directly.

Thanks.