The input of the CV network are (N, C, H, W) format. Tensor-RT works well when batch size in 0th dimention,
But in same NLP networks, the input format are (seqence, batch, embedding).
So my question:
- Does Tensor-RT support format which the batch size at 1st dimension?
- It would be better to provide an example network about definition and execution