Encountered some difficulties on the tensorrt serialization engine

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
,
1

Hello, I want to generate the tensorrt engine file on the x86_64 machine, and then put it on jetson for reasoning, but I can’t find a suitable version on the x86_64 machine.
This is the result I run on jetson

orin@ubuntu:~/workspace/code/Pytorch2TensorRT$ python3 Runengine_real_ESRGAN.py --engine engine_file/22.engine --images_path test_img
/home/orin/.local/lib/python3.8/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
[04/22/2023-23:13:08] [TRT] [E] 6: The engine plan file is not compatible with this version of TensorRT, expecting library version 8.4.0.11 got 8.4.0.6, please rebuild.
[04/22/2023-23:13:08] [TRT] [E] 4: [runtime.cpp::deserializeCudaEngine::49] Error Code 4: Internal Error (Engine deserialization failed.)
Traceback (most recent call last):
  File "Runengine_real_ESRGAN.py", line 126, in <module>
    main(args)
  File "Runengine_real_ESRGAN.py", line 96, in main
    predict(parser_data.engine,parser_data.images_path,
  File "Runengine_real_ESRGAN.py", line 44, in predict
    context = engine.create_execution_context()
AttributeError: 'NoneType' object has no attribute 'create_execution_context'
4

Dear @Ambition ,
I want to generate the tensorrt engine file on the x86_64 machine, and then put it on jetson for reasoning

FYI, The TRT engine file generated on x86 can not be used on target. You need to generate TRT engine file on target.

I can’t find a suitable version on the x86_64 machine

You mean you want TRT version same as on Jetson?

May I know if you can use below workflow ?
On Host : PyTorch → ONNX → Copy ONNX to target
On target : Use trtexec or TRT APIs to convert ONNX-> TRT model?

5

Thank you for answering my question, my original workflow is like you said, but I recently encountered a problem of insufficient device memory when serializing a super-resolution engine, so I want to complete it on a host with enough memory Serialization.

6

Dear @Ambition,
But Serialized TRT engine on host does not work in target. You need to serialize the engine on target when you want to use the model on target

7

Is this theoretically possible if I use cross-compiled docker on another machine？

8

Dear @Ambition,

Is this theoretically possible if I use cross-compiled docker on another machine？

I did not get this query in the current context. Could you provide more details on what you want to do

9

Sorry, my previous understanding was wrong, what I want to do is, I serialize the engine on the x86_64 device, and then deploy it on jetson

10

Dear @Ambition,
what I want to do is, I serialize the engine on the x86_64 device, and then deploy it on jetson

It does not work.

11

Oh, by setting the computing power of the gpu, then the version is the same?

12

In engine building phase of TensorRT model, it runs different set of kernels for each layer on GPU and picks the best set of kernels. So if you want to use TRT model on target, you need to build it on target.

14

Thank you for your answer, my doubts have been resolved

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.