Running a TensorRT model on Xavier

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hi all,
I have a TensorRT model (converted to FP16), I’m currently running it using the Triton inference server on my main Ubuntu machine.
How do I deploy it on the Xavier? I’ve tried running Triton docker on the AGX but it’s not supported on aarch64… (wtf? - surely the agx is an ideal candidate…)

Any tips / guides how I should run this model? Any help would be greatly appreciated!

3

Hi,

TensorRT library is pre-installed when setting-up the device with SDK manger.

What kind of model do you want to use?
You can find some TensorRT sample in this folder:

/usr/src/tensorrt/samples

Thanks.

4

it’s a retinanet model

5

Hi,

You can find an example for RetinaNet to TensorRT here:

Thanks.