I’ve seen in the TensorRT developer guide document that there is a:
‣ maxBatchSize is the size for which the engine will be tuned. At execution time, smaller batches may be used, but not larger.
But I am not quite clear of this parameter, could anyone help me to clarify this parameter?
What does BatchSize here means? Is it the same meaning of batch size in deep neural work training? Or other meanings?