Description
I am trying to host a custom model on Triton Inference server and I am trying to enable dynamic batching for the model.
I am converting a pytorch model to onnx and enabling dynamic batching on the input and output nodes.
I attach a copy of the script i use to convert from pytorch to onnx in the attachments and a copy of my config.pbtxt file.
Relevant Files
export_onnx.py (683 Bytes)
config.pbtxt (270 Bytes)
Error Encountered
E0522 09:23:57.598550 82 model_repository_manager.cc:1215] failed to load ‘par’ version 1: Invalid argument: model ‘par’, tensor ‘512’: for the model to support batching the shape should have at least 1 dimension and the first dimension must be -1; but shape expected by the model is [1,22]