Custom Models for Deepstream (TRT)

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi,

We are having some trouble using custom models in Deepstream. We tried multiple models (onnx, caffe, uff) however the models seem to be TRT (TensorRT) incompatible. We would like to know can we make a model TRT compatible, is there some documentation for that? Also are there readily availabe models that are TRT compatible which would help in easy prototyping?

Some other things that we tried:

  1. We tried using the TensorFlow -TRT converter, but when we supply the engine file in Deep Stream (via the config file), it doesn’t build
  2. We tried using TLT as well, but that is throwing an error while creating TF-Records.

Specifically:

  1. Could you tell us how do we use custom models in Deepstream?
  2. How do we use TensorFlow -TRT converter with Deepstream
  3. A way to use TLT effectively (some blog post, some documentation) because we followed the current documentation and it still threw an error.

Specs:
• Hardware Platform (Jetson / GPU): Jetson TX2
• DeepStream Version: 5.1
• JetPack Version (valid for Jetson only): 4.5.1

4

Hi @lambda1,

We recommend you to post your query on Deepstream forum to get better help.

Thank you.