Tensorflow running very slow on Nvidia Jetson AGX Orin

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Description

TensorFlow models are running very slow on Nvidia Jetson Orin AGX 64GB module. FaceNet model for face recognition is running at 2-3 FPS whereas the same model is showing around 10FPS in windows CPU laptop. I want to increase the speed in terms of FPS on Nvidia controller. Similar issue (i.e. running very slow) is seen in PyTorch also.

Environment

TensorRT Version: 10.3
GPU Type:
Nvidia Driver Version: 540.4.0
CUDA Version: 12.6
CUDNN Version: 9.3
Operating System + Version: Ubuntu 22.04.5 LTS
Python Version (if applicable): 3.10
TensorFlow Version (if applicable): tensorflow-2.16.1+nv24.8.dist-info
PyTorch Version (if applicable): torch-2.5.0a0+872d972e41.nv24.8.dist-info
Baremetal or Container (if container which image + tag):

Steps To Reproduce

Running any TensorFlow model is so slow taking similar or more execution time than CPU devices.

Please support me in improving the speed of model inferences. I could not find TensorRT examples to optimize TensorFlow model. Also, please share the relevant examples information.

3

Hi,
Here are some suggestions for the common issues:

1. Performance

Please run the below command before benchmarking deep learning use case:

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

2. Installation

Installation guide of deep learning frameworks on Jetson:

3. Tutorial

Startup deep learning tutorial:

4. Report issue

If these suggestions don’t help and you want to report an issue to us, please attach the model, command/step, and the customized app (if any) with us to reproduce locally.

Thanks!

5

Hi,

As mentioned above, please maximize the device performance first:

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

More, could you monitor your device with tegrastats concurrently to check if the model is running on the GPU?

$ sudo tegrastats

Thanks.