Set the model repository to S3

Description

I’m learning how to use the Triton Inference Server. I want to set the model repository to S3 bucket.
(nvcr.io/nvidia/tritonserver:21.10-py3)

I created a “triton-testmodels file” in S3 bucket and placed the example model.

Amazon S3
tirton-testmodels
|—inception_graphdef_aws/
|—config.pbtxt
|—inception_labels.txt
|—1/
|----model.graphdef

So I ran the following command.

sudo docker run --gpus=1 --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v $PWD/docs/examples/model_repository:/models nvcr.io/nvidia/tritonserver:21.10-py3 tritonserver --model-repository=/models --model-repository=s3://triton-testmodels --model-control-mode=explicit

But it fails to start. Error message
I1126 02: 26: 44.921849 1 server.cc: 249] No server context available. Exiting immediately.
error: creating server: Internal --Could not get MetaData for bucket with name triton-testmodels due to exception:, error message: No response body.

Since I already have the AWS CLI installed, I can use S3 outside the docker container. But since I haven’t attached a volume, I think this container cannot use S3.

If I want to set the model repository to S3, what kind of settings do we need? Is there a way to set the S3 credentials inside the container?

Best regards,
Rei

Environment

Triton Inference Server
nvcr.io/nvidia/tritonserver:21.10-py3

Hi,

This looks like Triton related. We recommend you to please post your concern here to get better help.

Thank you.