Tensorflow not finding any GPUs

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

I am running a Jeston Xaiver NX on a freash install of Jectpack 5.01. I used the SDKmanger tool to flash jetpack and the developer tools to the NVMe ssd.

I followed the instructions to install tensorflow here Installing TensorFlow for Jetson Platform :: NVIDIA Deep Learning Frameworks Documentation

I am running movenet tflite model downloaded from here TensorFlow Hub

Tensorflow will only run on the CPU and I get this error for the GPUs

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
2022-06-21 19:47:57.196589: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:57.326235: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:57.326799: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:57.330058: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:57.330522: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:57.330910: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:58.964216: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:58.964669: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:58.964869: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1609] Could not identify NUMA node of platform GPU id 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-06-21 19:47:58.965176: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:938] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-21 19:47:58.965446: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 7123 MB memory: → device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2

3

Hi,

Usually, NUMA is a harmless warning.

Based on your log below, TensorFlow detects the GPU correctly.
Is there any issue when running the model in GPU mode?

2022-06-21 19:47:58.965446: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 7123 MB memory: → device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2

Thanks.

4

It seems this model (tflite movenet) runs poorly on Jetson. I can get 70 FPS running on my laptop only using CPU but on Jetson it only gets 13 FPS and also a big delay between what is happening in front of camera and when the stream comes through. I have changed to trt_pose model although it is a much heavier model it runs much faster as it is optimized for Jetson. It runs faster than the camera and I get real time between what is happening in front of the camera and the output.

5

Hi,

Have you maximized the device performance first?

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

Thanks.

6

In the GUI I set the highest performance 20w 6 cores

7

Hi,

Could you try the jetson_clocks script?

It will fix the frequency to the maximum.
The default mode is the dynamic frequency that affects the performance.

Thanks.

8

I will not this for future. I have removed all the Movenet model and am only using the trt_pose model now.

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.