If I create an model.engine, how can I know the precision, int8, float16?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

A clear and concise description of the bug or issue.

Environment

TensorRT Version:
GPU Type:
Nvidia Driver Version:
CUDA Version:
CUDNN Version:
Operating System + Version:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered

how can I check the precision of a model.engine with python? I want to check the precision(int8, float16 or float32) of the created model.engine to confirm that the precision is what I need.

5

You can check the precision TRT uses in verbose logs by searching for “Engine Layer Information”.

6

thanks for your reply.
could you please illustrate more concrete?
Such as the attachment, what’s the precision of the end2end.engine model? And how do you get the precision?
If there is code that you check the model precision provided, it would be excellent.

end2end.engine (69.5 MB)

7

We can check each layer’s precision using the IEngineInspector APIs:

https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Core/EngineInspector.html
But we need to first enable kDETAILED ProfilingVerbosity when building the engine.