Hello,
I deployed to docker Tensorflow pretrained model using Nvidia Triton , model is loaded successfully, but on the very last step, when I am calling model to get predictions I get error which I was not able to fix: Error while reading resource variable stack_4/block_3/expand_conv2d/kernel from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (screenshot attached). Any ideas what it means and what should I do?
What I did:
- Downloaded pretrained model from tfhub.dev/tensorflow/efficientnet/b7/classification/1
- Deployed model on GPU using Nvidia Triton
nvidia-docker run --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 \ -v/home/renata/ktu/nvidia/model_repository:/models nvcr.io/nvidia/tritonserver:22.03-py3 tritonserver \ –model-repository=/models --strict-model-config=true
- Used this file from triton inference client on github (cannot post full link as I am new member and it is not allowed to me) /triton-inference-server/client/blob/main/src/python/examples/image_client.py to access my model and get predictions
python3 image_client.py -m efficientnetB7 -s INCEPTION -u 0.0.0.0:8001 -i gRPC dog.jpg
-
Get the error:
inference failed: [StatusCode.INTERNAL] 2 root error(s) found.
(0) Failed precondition: Error while reading resource variable stack_4/block_3/depthwise_bn/gamma from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/stack_4/block_3/depthwise_bn/gamma)
[[{{node model_14/stack_4/block_3/depthwise_bn/ReadVariableOp}}]]
[[StatefulPartitionedCall/_7]]
(1) Failed precondition: Error while reading resource variable stack_4/block_3/depthwise_bn/gamma from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/stack_4/block_3/depthwise_bn/gamma)
[[{{node model_14/stack_4/block_3/depthwise_bn/ReadVariableOp}}]]
0 successful operations.
0 derived errors ignored. -
As I was able to deploy same model using Tensorflow Serving without any development, I believe that model file is correct. But it looks like Triton requires something else, what is not defined anywhere.