Hi i want to know if i am doing this correctly…
I am using Inference server container 19.05. I want to test inference performance for resnet50 int8.
- I have saved a new resnet50 int8 model.plan using the trtexec resnet50.caffemodel with --int8 flag.
- git cloned the infernece_server repo
- I have placed the resnet50 int8 model.plan in the inferenceserver/docs/examples/modelrepository/resnet50_netdef/1/
- edited the config file in input and output as TYPE_INT8.
- When i create the server - I am getting an error that the the input and the output expect fp32 and its set to int8.
How to perform int8 inference in the inference server??