Triton Installation on Jetson Nano


I am trying to install Triton inference server from source file (according to published latest release: Releases · triton-inference-server/server · GitHub)

Below are my system information:
Nvidia Jetson Nano,
Ubuntu 18.04 LTS
Jetpack 4.6

I have download the file called tritonserver2.12.0-jetpack4.6.tgz file but, I am not be able to compile/install Triton Server.

When I try to execute following code, it says ‘bash: tritonserver: command not found’
tritonserver --model-repository=/path/to/model_repo --backend-directory=/path/to/tritonserver/backends \ --backend-config=tensorflow,version=2

How can I compile/install and run triton inference server?

What should I do with source code file (tar.gz file at the bottom)?

I have attached the image of tritonserver2.12.0-jetpack4.6.tgz file folder.



Hi, This might be better posted in the Jetson Nano category. I will move it over for you.

If you’ve followed the install instructions from your link, installed all dependencies, you wouldn’t need the sources tarball until you want to customize and rebuild yourself.

You may just set up the environment for the path where you’ve installed the triton server binaries:

TRITON_SERVER_DIR= <where_ever_you_installed>

Note there will be a webinar for more details about triton with Jetsons.