Continuing the discussion from The performance of the Jetson Orin Nano module does not match the data provided on the official website : Hi I read the topic The performance of the Jetson Orin Nano module does not match the data provided on the official website But still confused on the last reply …

Hi, Since you have less memory, maybe use a smaller matrix size, ex. m=256, n=256, will help. DLA doesn’t support CUTLASS. Please use TensorRT API instead. Below is some DLA sample for your reference: [image] GitHub - NVIDIA/Deep-Learning-Accelerator-SW: NVIDIA DLA-SW, the recipes …

Hi, We recommended trying our CUTLASS library to benchmark peak performance. You can find more details in the below topic: [image] Discrepancy Between Claimed and Actual Sparse INT8 Performance of Tensor Cores on Jetson AGX Orin Jetson AGX Orin Hi, Thanks for your pati…

Hi Aastalll, Thanks for your help. After read the topic, I ran the CULTASS as below $ git clone https://github.com/NVIDIA/cutlass.git $ cd cutlass/ $ mkdir build && cd build $ cmake .. -DCUTLASS_NVCC_ARCHS=87 -DCUTLASS_LIBRARY_KERNELS=i16864spgemm $ make cutlass_profiler -j12 The result is bleow …

Hi, Just want to double-confirm, have you maximized the device performance first? $ sudo nvpmodel -m 0 $ sudo jetson_clocks Since Orin NX has less memory, could you try Identity4, Identity2, and Identity1 in the below line to see if it helps? NVIDIA/cutlass/blob/main/python/cutlass_librar…

Hi Aastalll, Sorry I forget to to sudo jetson_clocks. After sudo jetson_clocks, the max tops can be higher. to 20456.4 GFLOP/s. I also tried Identity8 Identity4 Identity2 Identity1. After changed, also make cutlass_profiler -j12. Now 21060.5 GFLOP/s is what I can get maximum.

Hi, Thanks for the info. We will give it a check and provide more info to you later. Thanks.

Hi We test this with Orin NX 16GB, which expects to get 60TOPs on GPU. (Total 100 TOPs and 2x DLA can reach 40TOPs ) Then the peak sparse INT8 performance we can get is 34.122 TOPs, around 56 %SOL. m=512, n=512, k=8192 with Identity2 ============================= Problem ID: 1 Provi…

Thanks AastaLLL. I test it on Orin NX 8GB, which expects to get 50TOPS. m=512, n=512, k=8192 with Identity2 result is 28.5T is ~57%. Problem ID: 1 Provider: CUTLASS OperationKind: spgemm Operation: cutlass_tensorop_s8_i16864spgemm_s8_256x128_128x3_tn_align16 Status: S…

Thanks so much! On my board, m=256,n=256 didn’t help. ============================= Problem ID: 1 Provider: CUTLASS OperationKind: spgemm Operation: cutlass_tensorop_s8_i16864spgemm_s8_256x128_128x3_tn_align16 Status: Success Verification: ON Disposition: …

How to verify Orin the TOPS performance

Robotics & Edge Computing Jetson Systems Jetson Orin NX

AastaLLL September 2, 2024, 4:54am 3

Hi,

We recommended trying our CUTLASS library to benchmark peak performance.
You can find more details in the below topic:

Thanks.

Topic		Replies	Views
Verifying TOPS with Jetson Orin Nano Jetson Orin NX cudnn	1	611	December 30, 2024
The performance of the Jetson Orin Nano module does not match the data provided on the official website Jetson AGX Orin cuda , performance	14	3086	September 28, 2023
GPU stress test for Orin NX Jetson Orin NX tensorrt	8	1400	June 4, 2025
How to test GPU performance Jetson AGX Orin gpu	1	484	January 10, 2025
Verify ai performance by cutlass_profiler,but it was too slow,why? Jetson Orin Nano cudnn , cublas	1	51	March 4, 2026
How to measure 200 TOPS of AI Performance of Orin 32GB? CUDA Programming and Performance jetson	6	3626	July 11, 2023
Discrepancy Between Claimed and Actual Sparse INT8 Performance of Tensor Cores on Jetson AGX Orin Jetson AGX Orin tensorrt , performance	14	982	September 11, 2024
Jetson AGX Orin TOPs / CUDA Cores Explained Jetson AGX Orin jetson-inference	7	7964	May 11, 2023
Why I get much higher TFLOPS in Orin AGX than what claimed in the document IGX Orin Support kernel , jetson-inference , documentation	6	642	November 4, 2024
How to measure 200 TOPS of AI Performance of Orin 32GB? Jetson AGX Orin jetson-inference	2	957	July 12, 2023

How to verify Orin the TOPS performance

Related topics