How many 16FP Tops for agx-orin 32G

[zhl]

It’s described in the datasheet,but there’s no answer I need.
and also see,

DLA-v2 is slower than DLA-v1 - #8 by AastaLLL
“Orin’s DLA has more int8 dense TOPs but fewer fp16 TOPs.”

I want to know what the actual data of FP16 TOPs should be，
Thank you for your answer.

AI Performance
JAO 64GB: Up to 275 Sparse TOPS (INT8)
JAO 32GB: Up to 200 Sparse TOPs (INT8)

Ampere GPU
JAO 64GB: 2 GPC | 8 TPC | 2048 NVIDIA® CUDA® cores | 64 Tensor cores
Ray-Tracing cores | 170 Sparse TOPS | Maximum Operating Frequency:1.3 GHz

JAO 32GB: 2 GPC | 7 TPC | | 1792 NVIDIA® CUDA® cores | 56 Tensor
cores Ray-Tracing cores | 108 Sparse TOPS Maximum Operating Frequency: 939 MHz

JAO: End-to-end lossless compression | Tiled Caching | OpenGL® 4.6+ |
OpenGL ES 3.2 | Vulkan™ 1.2+◊ | CUDA 10.2+ | Maximum Operating Frequency: 1.3 GHz

Arm Cortex-A78AE CPU
Arm v8.2 (64-bit) heterogeneous multi-processing (HMP) CPU
architecture |
JAO 64GB: 12x cores | 3 CPU clusters (4 cores/cluster) | 259
SPECint_rate2006
JAO 32GB: 8x cores | 2 CPU clusters (4 cores/cluster) | 177
SPECint_rate2006
JAO: L1 Cache: 64 KB L1 instruction cache (I-cache) + 64 KB L1 data
cache (D-cache) per CPU core | L2 Cache: 256 KB per CPU core | L3
Cache: 2MB per CPU cluster | Maximum Operating Frequency: 2.2 GHz

DL Accelerator
JAO: 2x NVDLA 2.0 Engines |
JAO 64GB: Maximum Operating Frequency: 1.6 GHz | 52.5 TOPS
each(Sparse INT8)
JAO 32GB: Maximum Operating Frequency: 1.4 GHz | 46 TOPs each
(Sparse INT8)

[zhl]

I also saw this picture, but I don’t know how to add it up to get the correct answer

[AastaLLL]

Hi,

We are checking the details with our internal team.
Will share more information with you later.

Thanks.

[AastaLLL]

Hi,

Orin’s TOPS is as low as 50% of FP16 TOPS in Xavier.
Thanks.

[zhl]

@zhl

64GB:

Thanks，and it‘s means about 5.5TOPs of FP16 for Orin ,
and does Orin 32G ram and Orin 64G ram as same TOPs when using FP16?

[AastaLLL]

Hi,

Sorry that we are not able to share the exact number here.
Since TOPs are only related to ops and clock, it’s expected the TOPs between Orin 32GB and 64GB are similar.

Thanks.