Question about maxbatchsize in TRT7

Hi, I feel confused for the maxbatchsize in TRT7.

After using the TRT7 with explicitBatch, I set the OptimizationProfile when the ONNX has a dynamic batchsize. But if the batchsize of the ONNX is fixed, did the onnx-parser set the maxbatchsize by dim[0] automatically?

There are some interfaces in TRT6 and TRT5 doc, but almost disappeared in TRT7 doc. What do they do?

builder->setMaxBatchSize(batchsize)
engine->getMaxBatchSize()

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

Hi, I will combine with reality.

For example, There is only one input with dim[3000, 32, 32] in the ONNX, and I think batchsize is dim[0] : 3000. Should I set the maxbatchsize by builder->setMaxBatchSize(batchsize)? I found that if I don’t set it up, it runs normally.
:D

@opluss,

ONNX parser only supports networks with an explicit batch dimension. So using explicitBatch we need not to call setMaxBatchSize.
For your reference sample

1 Like