Description
I’m learning how to use the Triton Inference Server. I want to set the model repository to S3 bucket.
(nvcr.io/nvidia/tritonserver:21.10-py3)
I created a “triton-testmodels file” in S3 bucket and placed the example model.
Amazon S3
tirton-testmodels
|—inception_graphdef_aws/
|—config.pbtxt
|—inception_labels.txt
|—1/
|----model.graphdef
So I ran the following command.
sudo docker run --gpus=1 --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v $PWD/docs/examples/model_repository:/models nvcr.io/nvidia/tritonserver:21.10-py3 tritonserver --model-repository=/models --model-repository=s3://triton-testmodels --model-control-mode=explicit
But it fails to start. Error message
I1126 02: 26: 44.921849 1 server.cc: 249] No server context available. Exiting immediately.
error: creating server: Internal --Could not get MetaData for bucket with name triton-testmodels due to exception:, error message: No response body.
Since I already have the AWS CLI installed, I can use S3 outside the docker container. But since I haven’t attached a volume, I think this container cannot use S3.
If I want to set the model repository to S3, what kind of settings do we need? Is there a way to set the S3 credentials inside the container?
Best regards,
Rei
Environment
Triton Inference Server
nvcr.io/nvidia/tritonserver:21.10-py3