I face an issue with the max_batch_size when trying to host pretrained resnet18 model using triton server. Setting max_batch_size larger than 0 causes triton server failed to launch.
For better clarity, kindly refer below for the steps that I took:
- Do onnx to trt conversion using nvcr.io/nvidia/tensorrt:22.09-py3
- Launch triton from the NGC Triton container (nvcr.io/nvidia/tritonserver:22.09-py3)
Snippets below show the error msg