I have been trying to get faster (real-time) inference for my Mask-RCNN custom model on the Jetson Xavier. Right now, the setup takes around 15s and then each inference takes 10s per frame, which is very slow. I am informed that Deepstream can be helpful to increase the data pipeline speed and TensorRT can speed up the inference through model optimisation.
1). What is the entire procedure for getting a faster inference ?
2) What all options do I have ?
3) Regarding TensorRT, I am aware that it provides TRT-inference-engine, ONNX-parsers, UFF-parsers, etc. Is there a way to select what works for me ?