After using the TRT7 with explicitBatch, I set the OptimizationProfile when the ONNX has a dynamic batchsize. But if the batchsize of the ONNX is fixed, did the onnx-parser set the maxbatchsize by dim[0] automatically?
There are some interfaces in TRT6 and TRT5 doc, but almost disappeared in TRT7 doc. What do they do?
Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:
validating your model with the below snippet
check_model.py
import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command. https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!
For example, There is only one input with dim[3000, 32, 32] in the ONNX, and I think batchsize is dim[0] : 3000. Should I set the maxbatchsize by builder->setMaxBatchSize(batchsize)? I found that if I don’t set it up, it runs normally.
:D
ONNX parser only supports networks with an explicit batch dimension. So using explicitBatch we need not to call setMaxBatchSize.
For your reference sample