Tensorflow and PyTorch do not use GPU on Jetson Xavier AGX

[heerah1493]

I’m trying to run some python application on Jetson Xavier AGX, however tensorflow is running only on cpu. I am using Python 3.8.

Installed tensorflow through this link Installing TensorFlow for Jetson Platform - NVIDIA Docs
Index of /compute/redist/jp/v502 tensorflow==2.10.0+nv22.11

My Jetson info:

NVIDIA Jetson-AGX
JetPack 5.0.2
Ubuntu 20.04 LTS
CUDA 11.4.0
CUDNN: 8.4
TensorRT: 8.4

I also tried installing Cuda version 11.8, but still running in CPU and not GPU.

Installed cuda through this link CUDA Toolkit 11.8 Downloads | NVIDIA Developer

Installed CuDNN 8.6.0 thourgh this link Installation Guide - NVIDIA Docs
and while verifying the installation through this link Installation Guide - NVIDIA Docs
I am getting error as follows

I also tried installing Pytorch with cuda 11.8 version with torch 1.12.0, but it’s running on CPU.

Could you please help me with this issue.

[dusty_nv]

@heerah1493 you do not need to install CUDA 11.8 to enable GPU acceleration in TensorFlow/PyTorch, and in fact at first you should probably stick with the version of CUDA that came with JetPack because that is what those pip wheels were built against.

Seeing as you now are also getting cuDNN failures, and are also on one of the earlier versions of JetPack 5, I would recommend re-flashing your board with the latest JetPack (at this time, that’s JetPack 5.1.1) and starting with a fresh environment. Then install PyTorch like in this thread:

PyTorch for Jetson

Below are pre-built PyTorch pip wheel installers for Jetson Nano, TX1/TX2, Xavier, and Orin with JetPack 4.2 and newer. Download one of the PyTorch binaries from below for your version of JetPack, and see the installation instructions to run on your Jetson. These pip wheels are built for ARM aarch64 architecture, so run these commands on your Jetson (not on a host PC). You can also use the pre-built l4t-pytorch and l4t-ml container images and Dockerfiles. PyTorch pip wheels JetPack 5

PyTo…

You can also try using the l4t-pytorch, l4t-tensorflow, or l4t-ml containers which already come with PyTorch/TensorFlow pre-installed. You should be able to do this without reflashing, as CUDA/cuDNN/ect are installed inside those containers.

[heerah1493]

Hi

The above issue seems to be resolved, but now the problem I am facing is w.r.t running Yolov5 model on Nvidia Xavier Jetson AGX device. I am getting below error while running Yolov5 model

RuntimeError: Unable to find a valid cuDNN algorithm to run convolution

[dusty_nv]

Hi @heerah1493, I would recommend posting a new topic about this specific issue along with references to the code that you are running, or to file an issue with the upstream maintainers of that YOLOv5 project.