I need to know if the TOPS are in int8s or floating-point? anyone knows?
TOPs indicate INT8 performance. TFLOPs is used for the FP32 performance score.
For example, in NVIDIA Jetson AGX Orin Series Technical Brief:
Jetson AGX Orin 64GB … up to 170 Sparse TOPs of INT8 Tensor compute, and up to 5.3 FP32 TFLOPs of CUDA compute.
So the sparse word involves something about the nvidia tensor cores i’m thinking correct? so i’m thinking it would make it 275 int8 and 52==10(fp16) and 102(fp8) which aren’t the tensor cores which involve the “sparse” ints… correct? so i can still get 276 int8 for calculations per second… correct?
5 multiply 2 == 10 (FP16)
10 MULTIPLY 2 == 20(fp8)
No, it means sparse operation.
The performance is achieved with tensor cores + cuda cores.