Slow YOLOv8 inference speed in Jetson Orin Nano (had trouble with pytorch and torchvision installation)

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
,
1

hello and greetings,

Right now I am working on yolov8 project for object detection, during inference I realize that instead of using GPU it was using the CPU instead, therefore the slow inference speed which is around ~300ms

setup :

  • Hardware: NVIDIA Orin Nano 8GB
  • Object Detection Model: YOLOv8
  • Jetpack: 5.1.1

I was following this tutorial Explains how to make YOLOv8 work with Jetson Nano | i7y blog and https://docs.ultralytics.com/

then i proceed with installation of pytorch and torchvision which then lead to error during torchvision installation which said :
" error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not on PYTHONPATH and which Python does not read “.pth” files from. The installation directory you specified (via --install-dir, --prefix, or the disutils default setting) was :

/home/bizbot/.local/lib/python3.8/site-packages/

and your PYTHONPATH environment variable currently contains:
‘’ "

3

when i check my pytorch version using torch.version, it came out with ">> torch>version’2.0.0+nv23.05’ " and when im checking my torchvision using import torchvision, it shows error “/home/bizbot/ultralytics/venv/lib/python3.8/site-packages/torchvision/io/images.py:13: Userwarning: Failed to load image python extension: libcudart.so.10.2: cannot open shared object file:No such file or directory
warn(f"Failed to load image Python extension: {e}”) "

i’ve tried reinstalling, reboot, create another environment using another version, but the error still persist

4

Hi,

The tutorial is for JetPack 4 which uses CUDA 10.2.
But Orin Nano use JetPack 5 and CUDA 11.4.

Would you mind trying the below container (l4t-pytorch:r35.2.1-pth2.0-py3) to see if it can work on GPU?

Thanks.

6

it still runs on CPU, I reinstall pytorch v2.0.0 and torchvision v0.15.1 and the error still persist during torchvision installation, the bad install directory error,

is there any specific installation method to specific path or something?

7

Hi,

Is the libcudart.so.10.2 error fixed?
The “bad install directory” should be related to the way you install the package.

Thanks.

8

yes , I reinstall it back again but outside of the virtual environment and it seems it fixed it

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.