Good Afternoon,
the nVidia A100 has a huge performance advantage in computing Tensorfloat32(TF32) numbers. The TF32 has just 19 Bits (source: What is the TensorFloat-32 Precision Format? | NVIDIA Blog). Is there an possibility to fully customize the 19Bits of TF32 numbers?
Kind regardes,
Daniel Wulf.
edit:
at the link above it is described that the input numbers and the output numbers are FT32’s but internal calculations are made by map the FT32 x FT32 in dot products with less need of bits. So there is no need for me to customize the TF32 Format for me.
Thanks.