Jetson device can't use cuda

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

I’m using Jetson Orin AGX device right now, my cuda version is 11.4, torch version is 2.4.1, I’m using conda virtual environment python==3.8, I’m trying to run the model of llama2-7b with this device, but found that he keeps running on the cpu, but I wanted to put it on the gpu, tested it and found torch.cuda.is_available() Returns false, so it can’t be put on the GPU, is there any way to fix it!
Any suggestions could be helpful!

4

Hi,

Please install the PyTorch package that has been built with CUDA support.
You can find the package in the topic below:

Thanks.

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.