How to get the input dimensions in INetworkDefinition？

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi, I am new to TensorRT.

If I have only the serialized TensorRT engine file with explicit batch, how can I get the input dimensions when I define the network (i.e. the input dimensions in network->addInput(…))?

For example, considering two cases:

  • Case 1: network->addInput(“x”, …, Dims2{-1, -1}), optimization profile dims: min_dims = (1, 10), opt_dims = (2, 10), max_dims = (2, 10).
  • Case 2: network->addInput(“x”, …, Dims2{-1, 10}). The optimization profile dims are the same with Case 1.

When I call ICudaEngine::getBindingDimensions, these 2 cases return the same result. I cannot know the input dimension definition rightly.

5

Hi @sneaxiy,

We can query the optimization profiles that are stored in the engine to get dimension.

Thank you.

6

Thank you. However, the dimensions stored in the optimization profiles are not the dimensions in INetworkDefinition. Considering the example above, the dimensions stored in the optimization profiles are the same in these 2 cases, but the dimensions defined in INetworkDefinition are different.

7

@sneaxiy,

The optimization profiles specify what are the valid ranges for each dimension. Whether it is dynamic or static at network build time is really irrelevant, the engine only works for the dimension in the optimization profile range.
If we truly want the original network definition, need to get them from the network definition author as that information is not useful when executing the network.

8

Thank you for your reply.

1 Like