I have trained SSD ResNet-18 model using TLT 2.0 (transfer learning toolkit)
docker pull nvcr.io/nvidia/tlt-streamanalytics:v2.0_dp_py2
I have 10 checkpoints for the same model e.g.
ssd_resnet_epoch1.tlt ssd_resnet_epoch2.tlt ssd_resnet_epoch3.tlt . . ssd_resnet_epoch10.tlt
What I want to do is load all these models on TRTIS (triton inference server) at once and perform inference.
- What TRTIS version/image tag should I use to support the models trained on TLT 2.0
- Can I load
.etltmodel directly on TRTIS
- According the documentation we need:
Is it necessary to have github repository / client container ?
Is there any basic script which can help to directly communicate with the inference server via HTTP ?
How to quickly load/infer/unload TLT models on TRTIS using single container and HTTP API ?
What versions of TRTIS are compatible with TLT v2.0_dp_py2 models ?