Jetson thor: run qwen2.5vl by ollama can't on GPU, only cpu

[957728106]

I’m trying VLM by ollama, base on the web" Tags · qwen2.5vl ", we can try ollama server + webUI + qwen2.5vl;

1、 Install the ollama server, the cmd is:
curl-fsSL https://ollama.com/install.sh / sh
sudo systemctl start ollama

2、 Install Open WebUI base by docker:
docker run -d
–name open-webui
-p 3000:8080
-e OLLAMA_API_BASE_URL=http://host.docker.internal:11434
-v /var/run/docker.sock:/var/run/docker.sock
–gpus all
ghcr.io/open-webui/open-webui:main

3、 start the VLM experiment
a、download the model: ollama run qwen2.5vl:3b
b、vim Modelfile:
FROM qwen2.5vl:3b
PARAMETER num_ctx 512
PARAMETER num_gpu 1
PARAMETER temperature 0.7
SYSTEM “你是一个支持中文的多模态 AI 助手”
c、run: ollama create qwen2.5vl:3b -f Modelfile
d、test：ollama run qwen2.5vl:3b “你好”

  but， the result is: 

if I modify the Modelfile as: PARAMETER num_gpu 1  ----> 0,  it's work,

but it’s not I wanted, it’s working on cpu, not GPU

  Please help me confirm what is wrong with my environment, thanks;

[jameskuo]

Hi:
You might need to use nvidia-smi to monitor GPU usage, as described in Thor GPU can not detected - #5 by AastaLLL

[957728106]

please see other my question: Jetson thor: nvidia-smi show Nvidia thor off

[jameskuo]

What about watch -n 0.1 nvidia-smi?
Run nvidia-smi will check the usage just once. Use watch to check it every 0.1 second.

[957728106]

[AastaLLL]

Hi,

Could you check if your Ollama has CUDA support? (libggml-cuda.so)
Or you can install it from the steps shared in the link below:

Jetson/L4T/Jetson AI Stack - eLinux.org

Thanks.

[957728106]

I follow the link step, only get libggml-cuda.so/libggml-cpu.so/libggml-base.so file , but no have the file “ollama”:

But, I pull docker image : “docker pull Package ollama · GitHub”
In the docker, I get the /opt/ollama files;

so In this way, I finally solved the problem that ollama compiles the source code without ollama executable program；

thanks for your answer， And share my solution with others；