I have a general question about the expectation of TensorRT engine files.
I used Transfer Learning Toolkit/TAO Toolkit on a dGPU to train a custom YOLOv3 model. I iterated through several pruning and retraining steps until I had a model that worked really well on my training set and test sets. I then created a .etlt file and finally a TensorRT .engine file and ran inference on a test set on the dGPU and was happy with the results. Then, I moved the model to the Jetson Nano by creating a .engine file on the Nano, using the same tlt-converter command line as I did with the dGPU. However, the inference results - as measured by eye (bounding boxes in Deepstream 5.1) were much worse than on the dGPU. I don’t know of an easy way to just do inference on the Nano using a .engine file to test the results quantitatively. But my question is this: Should TensorRT engines produce the same results on different platforms? If so, then I will continue to try to figure out how to quantitatively measure inference on the TLT YOLOv3 model outside of Deepstream 5.1. If not, then how does one select the proper model on the dGPU to deploy to edge devices?
TensorRT Version: 7.2.1
GPU Type: Tesla V100
Nvidia Driver Version: 460.32.03
Operating System + Version: CentOS7
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered