Guidance for PyTorch to TensorRT Deployment



Currently I am working in integrating a PyTorch developed model and I wanted to see if I could have some guidance in main things to consider for this. I would be following this guide: Developer Guide :: NVIDIA Deep Learning TensorRT Documentation for the process and my main concerns are in the gaps when converting from PyTorch to ONNX and then from ONNX creating a runtmie engine. Should I only be worried about the PyTorch to ONNX conversion and after having that step done its almost reassured that the next steps wont have larger roadblocks?



Hi @Mark.RiveraMelendez,

You are referring to correct document. It would be easy to build trt engine from ONNX.
Please refer below TensorRT samples doc for more examples.

You can also use GitHub - onnx/onnx-tensorrt: ONNX-TensorRT: TensorRT backend for ONNX for onnx to tensorrt.

Hope following doc about exporting pytorch model to ONNX will be helpful to you.

If you face any issues feel free to reachout us.

Thank you.

1 Like