Can etlt models be used with TensorRT C++ API?

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

I would like to be able to leverage some of the pretrained models that NVidia offers (PeopleNet, VehicleTypeNet, etc), all of which are offered in the etlt format.
If I understand things correctly, in order to create a device-specific TRT engine, the trt-converter is used to convert the etlt to an engine file that can be used for inference.
Similar to how I convert uff and onnx files at runtime, is there some way to use the TensorRT C++ API to create an engine file from the etlt file? Or is there always going to be a manual step of using the trt-converter to create the engine before I can deploy to a specific system?
Thanks!!

3

For trt engine generation, tlt user guide provides two ways.

  1. Using the tool tlt-converter.
  2. Config the etlt model and API_key in config file of Deepstream. Run deepstream’s applications.
4

Thank you for the quick response. I had already read about these two options in the docs. I was just curious if this could be done through the TensorRT C++ API, and the answer appears to be “no”. I do lots of work with Deepstream, and have run the etlt models through it, but in some of my work, it is better (more control and flexibility) to run models in native TensorRT. In these cases, it looks like in order to use a etlt model, I’ll have to have the stand-along tlt-converter tool and deploy it with my apps for users to be able to build the engines for their hardware. For me, this really makes the etlt format far less desirable. Regardless, thank you for your great tools.

1 Like