Trtexec : Static model does not take explicit shapes since the shape of inference tensors will be determined by the model itself


I have ERROR when running ONNX model using trtexec CLI when adding the shapes options as done here.



TensorRT Version: trtexec command line interface
Nvidia Driver Version:
CUDA Version: 11.4
CUDNN Version:
Operating System + Version: ubuntu 20.04
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag): baremetal

Relevant Files

resnet18 ONNX model
resnet18-v1-7.onnx (44.7 MB)

Steps To Reproduce

$ trtexec --onnx=resnet18-v1-7.onnx --explicitBatch --int8 --allowGPUFallback --useDLACore=1 --workspace=2048 --shapes=data:1x3x224x224


this error appears when adding the –shapes option, I am using this option to change the batch size in this command below I put 1x3x224x224 so batch=1 I know by default it is equal to one but my goal is to change it to like 8 or 16 but I put 1 just to show you an example.
If you have other solutions that let me change the batch size using ONNX pretrained model feel free to suggest them to me.

Thank you in advance.

Best regard,

Hi, Please refer to the below links to perform inference in INT8


Hello @NVES
Thank you for your reply, I will look at them. But I need to change the batch size first even in FP32 I can’t change the batch size because apparently I have a static ONNX model. How I can change my ONNX static model into a dynamic ONNX model using trtexec so I can change my batch size value.

Thank you!


From which framework model are you converting to onnx, Hope the following may help you to modify the onnx model dims.

Thank you.

Hello @spolisetty ,

Thank you very much for your suggestion. Now for my case what should I do to get for example a batch=4, I did something (see below) but I have errors.

from typing import Any, List, Dict, Set
from onnx import ModelProto, ValueInfoProto

import onnx.checker
batch = 4
layer = 3
W = 224
H = 224
input_dims = {"data": [batch, layer, W, H]}
output_dims = {"data": [batch, layer, W, H]}
model = onnx.load('resnet18/resnet18-v1-7.onnx')
updated_model = update_inputs_outputs_dims(model, input_dims, output_dims), 'model.onnx')

def update_inputs_outputs_dims(model: ModelProto, input_dims: Dict[str, List[Any]], output_dims: Dict[str, List[Any]]) -> ModelProto:
        This function updates the dimension sizes of the model's inputs and outputs to the values
        provided in input_dims and output_dims. if the dim value provided is negative, a unique dim_param
        will be set for that dimension.
        Example. if we have the following shape for inputs and outputs:
                shape(input_1) = ('b', 3, 'w', 'h')
                shape(input_2) = ('b', 4)
                and shape(output)  = ('b', 'd', 5)
            The parameters can be provided as:
                input_dims = {
                    "input_1": ['b', 3, 'w', 'h'],
                    "input_2": ['b', 4],
                output_dims = {
                    "output": ['b', -1, 5]
            Putting it together:
                model = onnx.load('model.onnx')
                updated_model = update_inputs_outputs_dims(model, input_dims, output_dims)
      , 'model.onnx')
    dim_param_set: Set[str] = set()

    def init_dim_param_set(dim_param_set: Set[str], value_infos: List[ValueInfoProto]) -> None:
        for info in value_infos:
            shape = info.type.tensor_type.shape
            for dim in shape.dim:
                if dim.HasField('dim_param'):
                    dim_param_set.add(dim.dim_param)  # type: ignore

    init_dim_param_set(dim_param_set, model.graph.input)  # type: ignore
    init_dim_param_set(dim_param_set, model.graph.output)  # type: ignore
    init_dim_param_set(dim_param_set, model.graph.value_info)  # type: ignore

    def update_dim(tensor: ValueInfoProto, dim: Any, j: int, name: str) -> None:
        dim_proto = tensor.type.tensor_type.shape.dim[j]
        if isinstance(dim, int):
            if dim >= 0:
                if dim_proto.HasField('dim_value') and dim_proto.dim_value != dim:
                    raise ValueError('Unable to set dimension value to {} for axis {} of {}. Contradicts existing dimension value {}.'
                        .format(dim, j, name, dim_proto.dim_value))
                dim_proto.dim_value = dim
                generated_dim_param = name + '_' + str(j)
                if generated_dim_param in dim_param_set:
                    raise ValueError('Unable to generate unique dim_param for axis {} of {}. Please manually provide a dim_param value.'
                        .format(j, name))
                dim_proto.dim_param = generated_dim_param
        elif isinstance(dim, str):
            dim_proto.dim_param = dim
            raise ValueError(f'Only int or str is accepted as dimension value, incorrect type: {type(dim)}')

    for input in model.graph.input:
        input_name =
        input_dim_arr = input_dims[input_name]
        for j, dim in enumerate(input_dim_arr):
            update_dim(input, dim, j, input_name)

    for output in model.graph.output:
        output_name =
        output_dim_arr = output_dims[output_name]
        for j, dim in enumerate(output_dim_arr):
            update_dim(output, dim, j, output_name)

    return model



Hello again @spolisetty @NVES ,

I found that you can turn your ONNX model from static into a dynamic shapes by replacing the value in the input to whatever character, see here. It is pretty much the same as @spolisetty suggested to me. So for me I replaced my default input shape 1x3x224x224 into bx3x224x224 and now when running trtexec I can add the --shape option without any error.

for example, now I can put:

--shapes=data:1x3x224x224 for batch=1
--shapes=data:32x3x224x224 for batch=32

Thank you very much for your suggestions. 🙂

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.