Hi admin, The jetson orin nano is the same architecture as the 3060Ti.Whether it’s the number of CUDAs or the clock, the nano is not as good as the 3060Ti, and the FP16’s power is about 17 TFLops compared to the 3060Ti. My actual testing comparing the nano running yolov8s is not as good as the 306…

Hi, Could you share how you run the benchmark? It’s expected to test it with TensorRT or CUTLASS library. Please note that you can maximize the Orin Nano’s performance via following command (super mode): $ sudo nvpmodel -m 2 $ sudo jetson_clocks Thanks.

$ sudo nvpmodel -m 2 $ sudo jetson_clocks alreay set thses two. and inference used TensorRT c++.

Hi, Do you use trtexec to benchmark the inference part only? Could you share your benchmark code with us or try it with trtexec? Thanks.

this benchmark with trtexec at jatson orin nano 8G: nano int8: 157qps [image] nano FP16 103qps [image] 3060Ti FP16 464qps [3060ti-1] Looking at the test data, the nano doesn’t perform at such a high arithmetic as the material states? [image]

Hi, How do you convert the qps into FP16 TFLOPS? Since TensorRT has optimized the model and changed its architecture, it is hard to know the exact operations used for inference. But our CUTLASS can output TFLOPS directly. It’s recommended to give it a try. Thanks.

I think for the same model, The same model, is the QPS proportionally related to the GPU’s TFLOPs? GPU fp16/int8 qps TLOPS TFPLOS Nano 8G FP16 103 17 ( ref ) 3060Ti FP16 464 16.2 ( ref ) I’ll try CUTLASS late and let you know the results .

Jetson orin nano fp16/int8 performance

Robotics & Edge Computing Jetson Systems Jetson Orin Nano

AastaLLL March 18, 2025, 3:04am 9

Hi,

Usually, we recommended to test this with GEMM directly (CUTLASS).

TFLOPS is the GPU computational performance.
A model usually contains layers that might be affected by memory read/write, memory bandwidth, or CPU performance.

Below is some discussion for CUTLASS for your reference:

Thanks.

Topic		Replies	Views
Jetson Orin AI Performance Jetson Orin Nano documentation	6	1776	February 1, 2023
FP16 does not decrease inference time on Jetson Nano Jetson Nano tensorrt	6	1431	August 23, 2022
TFlops seem too high Jetson Orin Nano jetson-inference	1	1049	June 16, 2023
Performance discrepancy: TensorRT achieves ~10 TFLOPS vs. 17 TFLOPS spec on Orin Nano (Super mode) Jetson Orin Nano tensorrt , cuda , kernel	6	273	February 25, 2026
Why jetson nano fp16 is slower than fp32 Jetson Nano jetson-inference	1	693	May 24, 2021
INT8 for jetson nano TAO Toolkit	3	649	May 30, 2020
Could you share me with some intuitive examples as below which tell the use cases for different TOPS value?(in the range of 0.5-10 TOPS) Jetson Orin Nano performance	3	230	July 10, 2024
Question about AI performance Jetson Orin Nano cuda	1	264	December 20, 2024
No performance improvement on Jetson Nano FP16 vs FP32 TensorRT	6	2909	February 22, 2021
TF/Keras inference 4 times faster with FP32 precision than with FP16 Jetson Nano	7	2858	September 2, 2019

Jetson orin nano fp16/int8 performance

Related topics