No GPU usage

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
1

I have installed ollama and am running Ministral 3 3b model, but all the load is on the CPU, the GPU seems to have no usage at all.

nvidia-smi shows no process at all

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.78                 Driver Version: 595.78         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  Orin (nvgpu)                  N/A  |   N/A              N/A |                  N/A |
| N/A   N/A  N/A             N/A  /  N/A  | Not Supported          |     N/A          N/A |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

and nvtop says :

$ nvtop 
No GPU to monitor.

The Ministral model is very slow, and CPU usage is 100% on all cores. I did not expect this from Jetson nano.

4

Hi,

This is a known issue in the upstream Ollama source.
Please find below the workaround:

Thanks

5

That made no difference.

6

Hi,

The above mentioned issue is fixed in the recent release.

What kind of command do you test?
Please share the logs of the commands below.

$ ollama ps

$ journalctl -u ollama

Thanks.

7

Sorry, my bad, implementing your fix makes things better indeed, thanks !

8

Hi,

Thanks for the confirmation.