Running TensorRT Inference Server without Docker Command

Hi all,

I am currently trying to run tensorrt inference server and I followed instructions listed here:

I have successfully built the server from source with correcting a few C++ codes. However, there is literally no instruction about running the server without Docker command.

It doesn’t make sense if you put some instructions about building a software while not giving any info when it comes to run it.

Hi doruk898,

I reached out to the TRTIS team about updating the docs for this case (running TRT server on host after building from source), and they created a task to do so.

In the meantime, I believe you can navigate into the trtis/install directory created during the build process as mentioned here:

And then you can run the executable in there:

# Inside trtis/install directory after building...
./trtserver --model-store=<path_to_models>

This is essentially the same way that you would run it if executing in a docker container:

nvidia-docker run <container_config_flags> ... trtserver --model-store=<path_to_models>

NVIDIA Enterprise Support