Importing TensorRT model to TLT

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hey everyone,
I am working on Pose Estimation and I was thinking about trying to prune and quantize the 2 models I am using thanks to TLT.
So here’s the question: is there a way to import a tensorrt model to TLT as a pretrained model? Is there a way yo convert a model to the format .tlt?

Thanks in advance!

3

Could you please give an example of “a tensorrt model”?

4

I’ll be more specific, I am sorry if the question was malformed:
My objective is to prune the 2 models you can find over here: GitHub - NVIDIA-AI-IOT/trt_pose: Real-time pose estimation accelerated with NVIDIA TensorRT
A densenet and a resnet. In that same link a utility called touch2trt manages the conversion from the format pytorch (.pth) to tensorflow.
I am trying to convert these models to then give them (one at a time) as input to tlt, specifying the path in a spec file. While trying to solve this I found out that all the pretrained models downloadable directly from tlt are in the hdf5 format.
Do I need to convert the 2 models mentioned above in said format to be able to read them inside tlt environment?

Thank you for the help!

5

Unfortunately these custom pre-trained models are not supported with TLT. With TLT you will have to use pre-trained weights from one of our models in ngc.

7

Hi Morganh, thank you for the answer!
I’m having trouble understanding your statement. So if I want to prune those 2 models using tlt-prune (because of its ease of use) there is NO WAY for me to import the models? What if I convert the pytorch files to hdf5?
Is the toolkit built to accept only what comes from NGC (maybe by doing a check or some cryptography) or is it possible for me to find a workaround by doing the right conversion??
I really need to shrink those models, pruning with TLT would be fantastic.

Thanks for your time

8

Hi nicola,
From https://docs.nvidia.com/metropolis/TLT/FAQ/index.html
Those two models are not compatible with TLT process. In TLT training spec, user can set ngc models or tlt models as pretrained model.

Instead of NVIDIA provided pre-trained models, can I use TLT with my own or any open source pre-trained models?
No third party pre-trained models are supported by TLT. Only NVIDIA pre-trained models from NGC are currently supported which can be retrained with your custom data

For pruning, only tlt format model trained via TLT is accepted for the tool tlt-prune.