Thor ollama[7754]: CUDA error: an internal operation failed

Robotics & Edge Computing Jetson Systems Jetson Thor
,
1

$ sudo snap install ollama

$ ollama run --verbose deepseek-r1:8b

>>> hello

Error: model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details

$ journalctl -u ollama --no-pager | grep -i error

9月 02 14:03:21 thor ollama[7754]: CUDA error: an internal operation failed
9月 02 14:03:21 thor ollama[7754]: //ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:84: CUDA error
9月 02 14:03:21 thor ollama[112423]: #3 0x0000ffff007a4c70 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) () from /usr/local/lib/ollama/cuda_sbsa/libggml-cuda.so
9月 02 14:03:21 thor ollama[7754]: time=2025-09-02T14:03:21.966+08:00 level=ERROR source=server.go:1444 msg=“post predict” error=“Post “http://127.0.0.1:45381/completion\”: EOF”
9月 02 14:03:22 thor ollama[7754]: time=2025-09-02T14:03:22.078+08:00 level=ERROR source=server.go:414 msg=“llama runner terminated” error=“exit status 2”

then, what can I do for using ollama+deepseek on Thor natively?

3

Hi,

Suppose you can install it from the source with Thor’s GPU architecture (110).
Let us give it a try and update more info with you later.

Thanks.

4

Hi,

You can find our Ollama container for Thor in the link below:

Thanks.

5

Hi,

You can also run it locally.

Server

$ export PATH=/usr/local/cuda-13.0/bin:$PATH
$ export LD_LIBRARY_PATH=/usr/local/cuda-13.0/lib64:$LD_LIBRARY_PATH

$ cd /opt && wget https://go.dev/dl/go1.24.2.linux-arm64.tar.gz
$ rm -rf /usr/local/go && tar -xvf go1.24.2.linux-arm64.tar.gz && mv go /usr/local/
$ export PATH=$PATH:/usr/local/go/bin

$ cd /opt && git clone https://github.com/ollama/ollama
$ cd ollama && cmake -DCMAKE_CUDA_ARCHITECTURES=110 -B build && cmake --build build
$ cd /opt/ollama
$ go run . serve

Client

Open another console:

$ export PATH=$PATH:/usr/local/go/bin
$ go run . run --verbose deepseek-r1:8b
>>>

Thanks.