Question about maxbatchsize in TRT7

Hi, I feel confused for the maxbatchsize in TRT7.

After using the TRT7 with explicitBatch, I set the OptimizationProfile when the ONNX has a dynamic batchsize. But if the batchsize of the ONNX is fixed, did the onnx-parser set the maxbatchsize by dim[0] automatically?

There are some interfaces in TRT6 and TRT5 doc, but almost disappeared in TRT7 doc. What do they do?


Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging

Hi, I will combine with reality.

For example, There is only one input with dim[3000, 32, 32] in the ONNX, and I think batchsize is dim[0] : 3000. Should I set the maxbatchsize by builder->setMaxBatchSize(batchsize)? I found that if I don’t set it up, it runs normally.


ONNX parser only supports networks with an explicit batch dimension. So using explicitBatch we need not to call setMaxBatchSize.
For your reference sample

1 Like