Hello, This html page announces AI performance of Orin 32GB as 200 TOPS. I’d like to know how to measure this AI performance. To get some sort of certification, we need to prepare method to measure some performances. We are interested in TOPS and AI performance. How can we measure AI performance…

[image] jhjo: We still need the method to measure the TOPS value [image] Robert_Crovella: Choose a properly constructed CUBLAS INT8 GEMM operation Here is a possible recipe. That would get you int8 non-sparsity TOPS. To go to int4, I would suggest CUTLASS. I don’t have a specific e…

You might be able to get a better response asking this question on a orin forum .

I’m not 100% certain, (the folks on the Orin forum would probably know) but I believe the “AI performance” referred to on that page would be the throughput of the tensor core (TC) units. The ampere white paper indicates that an Ampere Tensor Core unit delivers 256 FP16 FMA ops/clk. If we do the ma…

Hi @Robert_Crovella , Thank you for the kind and professional answer! I firstly saw the calculation equation of the TOPS! There are arricles I noticed . From your explanation with those articles above, I think TOPS is such a thing that people like me cannot measure of. It’s like cpu clock frequen…

Hello @Robert_Crovella , I unchecked your answer as the solution of this question. We still need the method to measure the TOPS value for orin device. So, later sometime if anyone can provide this information, we would welcome and appreciate it. Thank you very much!

How to measure 200 TOPS of AI Performance of Orin 32GB?

Accelerated Computing CUDA CUDA Programming and Performance

jhjo July 11, 2023, 11:55pm 8

Hi @Robert_Crovella ,

Thanks!
I will share your answer with our company people with this one also.
Thank you!

Topic		Replies	Views
How to measure 200 TOPS of AI Performance of Orin 32GB? Jetson AGX Orin jetson-inference	2	957	July 12, 2023
How to verify Orin the TOPS performance Jetson Orin NX cuda	9	2396	September 10, 2024
How to test GPU performance Jetson AGX Orin gpu	1	484	January 10, 2025
Jetson series TOPS mean in FLOPS or INTS? Jetson AGX Orin performance	4	7679	November 20, 2023
AI Performance and CPU & GPU Loading Jetson AGX Orin gpu-computing	1	225	September 16, 2024
About Orin SoC Performance DRIVE AGX Orin General drive-docs	6	1500	November 21, 2022
The performance of the Jetson Orin Nano module does not match the data provided on the official website Jetson AGX Orin cuda , performance	14	3086	September 28, 2023
Question about AI performance Jetson Orin Nano cuda	1	254	December 20, 2024
The tensor core performance detail of Jetson AGX Orin 32GB Jetson AGX Orin	13	1621	June 13, 2023
Jetson AGX Orin TOPs / CUDA Cores Explained Jetson AGX Orin jetson-inference	7	7966	May 11, 2023

How to measure 200 TOPS of AI Performance of Orin 32GB?

Related topics