TensorRT 5 builder when set max_batch_size to 8 the output shape?

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,
I would like to know ，When I set the maxBatchSize to 8, what is the corresponding output?
for input （3，224，224），maxBatchSize set to 1, output is (1,101)
for input （3，224，224），maxBatchSize set to 8, my output is still (1,101),but why not (8,101)?

Thanks.

2

Hi,

The real inference tensor number is indicated by batchSize value.

context->enqueue(batchSize, &buffers[0], stream, nullptr);

maxBatchSize is used for allocating memory when generating TensorRT.
It indicates the engine capacity and may be different from the real inference batch number.

By the way, for batch=8:
Input should be ( 8, 3, 224, 224) and output is ( 8, 1, 101).

Thanks.

3

thanks