How to use INT8 LSTM with TensorRT 7.0.0 ?

I want to use LSTM with INT8 data type.

However, the orgin IRNNv2 Layer only support for FP32 and FP16, not INT8.So I can’t use the origin API for INT8 LSTM.

TensorRT 7.0.0 released a new plugin,but it also only support for FP16.
The following section describes the new Persistent LSTM plugin. The Persistent LSTM plugin supports half-precision persistent LSTM.

The IPluginV2 supports INT8 and I may use it to implement a custom INT8 LSTM Layer.It seems that this is the only way to use INT8 LSTM.

I would like to know is it possible for me to implement a custom INT8 LSTM Layer with IPluginV2 Layer. And is there a better way for me to do that?


You can implement any custom plugin you want for your application/use-case.
For newer TRT version, IPluginV2ExtDynamic might be a better idea for custom plugin.


In the new version of trt8, is int8 quantization of LSTM already supported? Is there any plan for LSTM int8? Thank you for your reply.