Tool to view Optimized TRT networks

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello,

I am wondering if a TensorRT tool exists to show the post-optimized structure of a TensorRT network.

After networks go through the building/optimizing process, it’s a bit of a mystery what actually comes out the other end. Does a tool exist that can read built models and display some helpful information?

For example, some data that would be helpful to know is (for each layer):

  • Name of layer
  • Type of layer
  • Input and output Sizes
  • Input and output data types
2

Unfortunately, we don’t have a visual tool (similar to tensorboard/netron) yet. We are always taking customer feedback and integrating them into future product releases.

3

Thanks for the response NVES.

Although there is no visual tool, is there a tool to get this information at all? I.e. via printf() in a C++ program?

I did a little digging and found that you can query the contents of networks an INetworkDefinition object using functions like:

nvinfer1::INetworkDefinition::getLayer(int index) const

(https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/c_api/classnvinfer1_1_1_i_network_definition.html#a4a81749aaa08e93ca4ae1dbb1739c7bd)

However, when the INetworkDefinition object is passed to the buildCudaEngine function (below), all of the optimizations and changes to the network are not visible to the INetworkDefinition object.

nvinfer1::IBuilder::buildCudaEngine(nvinfer1::INetworkDefinition & network)

Is there a way to retrieve an INetworkDefinition object after the build process so it can be analyzed?