What are the available algorithms of TensorRT?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Upon reading about TensorRT, I found a lot of places that mentioned TensorRT algorithms. While the algorithm selector could be found in here, I almost found no references to specific algorithms available.

The only time one, called MinimumWorkspaceAlgorithmSelector was mentioned is here, back in version 7.2.

So my questions are:

  • Is MinimumWorkspaceAlgorithmSelector the default algorithm selector? If that is the case, then setting the max_workspace_size to be smaller than all available memory makes no difference, as the algorithm chosen is already the smallest in terms of workspace size.
  • Are there any other default algorithm choices? If there are, could you provide some documents for them?

Thanks a lot.

3

Hi,

We recommend that you please refer to the latest official TensorRT documents. And we hope the following sample may help you:

Thank you.

4

Thank you. This definitely clears things up a bit.
However, it is mentioned in the docs that “The default behavior of TensorRT’s optimizer is to choose the algorithms that globally minimize the execution time of the engine.” Are there other options like choosing the algorithm requiring the least memory?

And in the MNIST example you provided, What doesMinimumWorkspaceAlgorithmSelector mean?
It is said to “build a GPU inference engine for MNIST using Algorithms with minimum workspace requirements.” What are the requirements? RAM and VRAM?

Thank you.

7

@spolisetty Hi. Could you help me answer the above questions? Thanks.

8

Currently, not available.

The sample is meant to illustrate the IAlgorithmSelector API. This example shows how users can select tactics with the smallest WS size for all layers for illustration.