Cannot use GPUs

[mike99mac]

Hi, new to this forum and GPUs. Want to get whisper STT working locally in a venv. It seems the transription does not use the 1024 GPUs. Tried dozens of permutations, but torch.cuda.is_available() always returns False. Here are some settings in my Jetson Orin Nano environment:

(stt_venv) pi@papabear:~/$ pip list | grep torch
torch              2.5.1
(stt_venv) pi@papabear:~/$ /usr/local/cuda-12.6/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Aug_14_10:14:07_PDT_2024
Cuda compilation tools, release 12.6, V12.6.68
Build cuda_12.6.r12.6/compiler.34714021_0
(stt_venv) pi@papabear:/usr/local/sbin/$ cat testcuda.py
#!/usr/bin/env python3
import torch
print("CUDA available:", torch.cuda.is_available())
if torch.cuda.is_available():
    print("CUDA device:", torch.cuda.get_device_name(0))
$ python3 -V
Python 3.11.0rc1
(stt_venv) pi@papabear:/usr/local/sbin/$ head -1 /etc/os-release
PRETTY_NAME="Ubuntu 22.04.5 LTS"

[DavidDDD]

Hi,
could your use blow commands to share the result with us first.

# In Orin Nano
cat /etc/nv_tegra_release 

Thanks

[mike99mac]

# R36 (release), REVISION: 4.0, GCID: 37537400, BOARD: generic, EABI: aarch64, DATE: Fri Sep 13 04:36:44 UTC 2024
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

[DavidDDD]

Hi,

You can visit this site to download the Torch wheels compatible with CUDA 12.6.

Thanks

[mike99mac]

Thanks for the reply.

I go to the URL and download the torch, torchaudio and torchvision wheel files. Try to install but get an error:

(stt_venv) $ stt_venv/bin/pip install torchaudio-2.5.0-cp310-cp310-linux_aarch64.whl
ERROR: torchaudio-2.5.0-cp310-cp310-linux_aarch64.whl is not a supported wheel on this platform.

Hmmm, guess I will try to build Python 3.10 in the venv.

[mike99mac]

Finally got it working!

Wrote it up here: GitHub - mike99mac/minimy-mike99mac: simple nlp based voice assistant framework