Yes, you choose to set both INT8 and FP16 mode if the platform supports it. TensorRT will choose the most performance optimal kernel to perform inference. For more information, please reference:
Hi cjluo, Have you figured out whether tensorrt runs plugin in FP16 mode? I am trying to run my custom plugin in FP16 mode, but seems there’s some technical issues with plugin implementation. I am wondering if we need to manually to do data type conversion.