Mask RCNN TensorRT in Triton

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I have used TensorRT’s sample uff Mask RCNN example to convert our model from h5 and create an engine file. I have loaded this engine onto the Triton server and used the --strict-model-config=false flag to generate the config.pbtxt. The server is started successfully on the HTTP and gRPC ports. What I need help with is using the C++ API to write the inference code. Not sure which client examples to follow. Any lead or direction of how to go about this or which particular example to follow would really help! Thank you.

Environment

TensorRT Version: 7.0.0.11
GPU Type: Tesla M60
Nvidia Driver Version: NVidia 540 Linux Driver
CUDA Version: 10.0
CUDNN Version: 7.6.3.30
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): Python 3.6
TensorFlow Version (if applicable): TensorFlow 1.13.1

2

Hi @megha1,
Please check the below link. This might be useful for you to proceed.

Thanks!

3

Hi! Thanks for this. However I already have been able to perform inference directly using the TensorRT C++ API. What I need help with is to use the Triton inference server C++ API. Could you assist with that?

4

Hi @megha1,
Please refer to the links below :

For more details, kindly post it in Triton forum.
Thanks!