Guidance for PyTorch to TensorRT Deployment

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Hi,

Currently I am working in integrating a PyTorch developed model and I wanted to see if I could have some guidance in main things to consider for this. I would be following this guide: Developer Guide :: NVIDIA Deep Learning TensorRT Documentation for the process and my main concerns are in the gaps when converting from PyTorch to ONNX and then from ONNX creating a runtmie engine. Should I only be worried about the PyTorch to ONNX conversion and after having that step done its almost reassured that the next steps wont have larger roadblocks?

Thanks,

Mark

2

Hi @Mark.RiveraMelendez,

You are referring to correct document. It would be easy to build trt engine from ONNX.
Please refer below TensorRT samples doc for more examples.

You can also use GitHub - onnx/onnx-tensorrt: ONNX-TensorRT: TensorRT backend for ONNX for onnx to tensorrt.

Hope following doc about exporting pytorch model to ONNX will be helpful to you.
https://pytorch.org/tutorials/advanced/super_resolution_with_onnxruntime.html

If you face any issues feel free to reachout us.

Thank you.

1 Like