Is there any accuracy difference between fp32 and fp16?

We know TensorRT has fp32 and fp16 inference mode,so is there any inferecne accuracy difference between them? whether fp16 has lower accuracy than fp32?


Yes there will be minor accuracy loss between fp32 and fp16. It can vary based on the model you are using.
But with minor impact on accuracy fp16 will increase the inference speed of the optimized model.