Ollama is running slow on Jetson AGX Orin Dev-kit (32G)

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

I was trying to get Microsoft AutoGen Studio and using local LLAMA2 model via OLLAMA.

It is running, but very slow. Compared to my Laptop with an RTX2060 built-in GPU.

Any suggestion on what I need to change?

I got this message when I install the ollama

Adding ollama user to render group…
Adding current user to ollama group…
Creating ollama systemd service…
Enabling and starting ollama service…
Installing NVIDIA repository…
curl: (22) The requested URL returned error: 404

3

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi,

Could you check if the library runs on GPU?

You can find this information with tegrastats command:

$ sudo tegrastats

Thanks.