Best practice for running models like Yolo on Orin AGX?

[ian86]

I’m developing an application to run on the Orin AGX Development kit that needs to run some neural networks including a YoloV5 object detector. Is the standard way to do this, convert the YoloV5 model from Pytorch format (pt) to the TensorRT format (engine) and then write some python to call the engine and do the inferencing?

[dusty_nv]

Hi @ian86, yes, by converting/running the model with TensorRT, you will see increased inferencing performance. Here are two examples on GitHub that I found for YOLOv5 + TensorRT inference:

• https://github.com/BlueMirrors/Yolov5-TensorRT
• https://github.com/SeanAvery/yolov5-tensorrt

[ian86]

Thanks.
Looks like these examples have a dependencies on python modules: tensorrt and pycuda.
Does Nvidia supply these modules for the Orin? The reason I ask, is that I haven’t been able to find them.

[dusty_nv]

@ian86

Looks like these examples have a dependencies on python modules: tensorrt and pycuda.

@ian86 the tensorrt Python module should already be installed along with JetPack, and you can install PyCUDA from pip (or you can run it inside the l4t-pytorch or l4t-ml containers which already have it installed). You can see the steps I follow for building PyCUDA in the containers here:

https://github.com/dusty-nv/jetson-containers/blob/bc8d0264ef25aa0d1d25a54e4658f491d2fa130f/Dockerfile.pytorch#L167

[ian86]

Thanks Dusty, its going now.