How to use INT8 LSTM with TensorRT 7.0.0 ?

I want to use LSTM with INT8 data type.

However, the orgin IRNNv2 Layer only support for FP32 and FP16, not INT8.So I can’t use the origin API for INT8 LSTM.

TensorRT 7.0.0 released a new plugin,but it also only support for FP16.
The following section describes the new Persistent LSTM plugin. The Persistent LSTM plugin supports half-precision persistent LSTM.

The IPluginV2 supports INT8 and I may use it to implement a custom INT8 LSTM Layer.It seems that this is the only way to use INT8 LSTM.

I would like to know is it possible for me to implement a custom INT8 LSTM Layer with IPluginV2 Layer. And is there a better way for me to do that?

Hi,

You can implement any custom plugin you want for your application/use-case.
For newer TRT version, IPluginV2ExtDynamic might be a better idea for custom plugin.

Thanks

In the new version of trt8, is int8 quantization of LSTM already supported? Is there any plan for LSTM int8? Thank you for your reply.