INT8 cache portable across TRT versions?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi,

I see the following statement in the FAQ:

Applications must build new engines and INT8 calibration tables when using a new version of TensorRT.

Is it still accurate? Why aren’t INT8 calibration tables portable across TRT versions? They describe properties of the input model and the calibration data (scale factors). Why would this be dependent on the TRT version?

2

Hi, Please refer to the below links to perform inference in INT8

Thanks!

4

Hi,

As mentioned in the docs, Internal implementations and formats are continually changed and optimized between versions. Currently, we do not support INT8 calibration tables portable across the versions.

Thank you.

5

Thanks for the answer!

Is the INT8 calibration table really “internal implementation” though? The following sample describes the format of the calibration table, so it seems rather “public” to me:

Where:

<TRT-xxxx>-<xxxxxxx> The TensorRT version followed by the calibration algorithm, for example, EntropyCalibration2.
<layer name> : value corresponds to the floating point activation scales determined during calibration for each tensor in the network.

Otherwise, why should a document publicly describe to such level of detail the structure and the format of the calibration table, if it’s an “internal implementation detail”?

1 Like
6

Hi,

We are checking with our team internally on this. Will get back to you.

Thank you.

7

Hi,

Some calibrators run after graph optimization, which is release-specific. Our team is working on this, in future releases, we may have the flexibility to port cache across versions.

Thank you.