I want to know about the peak performance of Mixed precision GEMM (Tensor Cores operate on FP16 input data with FP32 accumulation) for Ampere and Volta architecture. Do we have any refrence of is it poosible to predeict it without performing an experiment?
This is published data.
V100 “Tensor Performance”: https://images.nvidia.com/content/technologies/volta/pdf/volta-v100-datasheet-update-us-1165301-r5.pdf and NVIDIA V100 | NVIDIA
A100 “FP16 Tensor Core” : NVIDIA A100 | NVIDIA and https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-us-nvidia-1758950-r4-web.pdf