Tao converter Optimization profiles

[david221]

When converting model to trt there is a parameter

-p input_1,1x1x320x320,4x1x320x320,16x1x320x320 \

I could not understand the explanation here

Optimization profiles for .etlt models with dynamic shape. Comma separated list of optimization profile shapes in the format <input_name>,<min_shape>,<opt_shape>,<max_shape> , where each shape has the format: <n>x<c>x<h>x<w> . Can be specified multiple times if there are multiple input tensors for the model. This is only useful for new models introduced in TAO Toolkit 3.21.08. This parameter is not required for models that are already existed in TAO Toolkit 2.0.

whats the meaning of n and c ? I’m guessing c is channels …??

Thanks!

[Morganh]

Yes, channels.

[david221]

And what’s n? 1, 4, 16 in the example?

[Morganh]

N means batch N.

[david221]

Meaning inference batch sizes of 1, 4 or 16 images?

Like if I have 4 video feeds I can give the model a batch of 4 frames at a time? How are the resulting frames returned? Is there an example of this?

Many thanks Morganh!

[Morganh]

See dynamic shapes.

TensorRT Developer Guide :: NVIDIA Deep Learning SDK Documentation

Some topics, for example,

trtexec dynamic batch size · Issue #976 · NVIDIA/TensorRT · GitHub
Trtexec and dynamic batch size - #5 by NVES