I have a serialized TensorRT engine (*.plan) file that I’ve created from a pre-trained PyTorch/Retinanet model that I’ve further trained (fine-tuned) using a custom dataset as input. The model and code I’ve based this upon is provided by NVIDIA here.
This NVIDIA RetinaNet model is intended to be run within the NVIDIA PyTorch Docker container. However, this isn’t usable on a Jetson Nano since nvidia-docker is not supported yet on ARM64. So in order to use this model on Jetson Nano I need to perform inference using the TensorRT engine outside of the context of the Docker container. I’ve not yet found documentation that clearly explains how I would do this. (For example this guide is about as clear as mud to a rookie like me.)
My goal is to read image frames from a video stream and use the model to perform inference on each frame for object detection. I have this working as planned on a laptop using the fine-tuned PyTorch-RetinaNet (*.pth) model, and the TensorRT on Jetson Nano is my next frontier.
Thanks in advance for any comments or suggestions.