Whats the differnece between implicit and explicit batch in tensor rt

new to tensor rt and am confused between impleict and explict batch sizes in trt. whats is the difference between them? Does it mean that if explicit batch flag is set, the network can only be inferred with the batch size specified during building process? and what about implicit batch

would be really helpful if someone could explain these two.
thanks.

Hi @gjraza
Explicit batch is required when you are dealing with Dynamic shapes, otherwise network will be created using implicit batch dimension.
The link below will be helpful to understand

Thanks!

1 Like