Set the model repository to S3


I’m learning how to use the Triton Inference Server. I want to set the model repository to S3 bucket.

I created a “triton-testmodels file” in S3 bucket and placed the example model.

Amazon S3

So I ran the following command.

sudo docker run --gpus=1 --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v $PWD/docs/examples/model_repository:/models tritonserver --model-repository=/models --model-repository=s3://triton-testmodels --model-control-mode=explicit

But it fails to start. Error message
I1126 02: 26: 44.921849 1 249] No server context available. Exiting immediately.
error: creating server: Internal --Could not get MetaData for bucket with name triton-testmodels due to exception:, error message: No response body.

Since I already have the AWS CLI installed, I can use S3 outside the docker container. But since I haven’t attached a volume, I think this container cannot use S3.

If I want to set the model repository to S3, what kind of settings do we need? Is there a way to set the S3 credentials inside the container?

Best regards,


Triton Inference Server


This looks like Triton related. We recommend you to please post your concern here to get better help.

Thank you.