Ollama errors orin nano

Robotics & Edge Computing Jetson Systems Jetson Orin NX
,
25

Hi,

Thanks a lot for the information.

Our device is an original setup with SDKmanager.
Not sure if this is the reason that we cannot reproduce the issue locally.

Will try to set up the device with the image and provide more info to you later.

Thanks.

26

I am having the same problem ollama llama3.2.1b - and most models.

Error: 500 Internal Server Error: llama runner process has terminated: error loading model: unable to allocate CUDA0 buffer
llama_model_load_from_file_impl: failed to load model

I can only get one model to work:
gemma3:4b

I followed suggested Models ( only gemma3:4b worked ) :

x - ollama run llama3.2:3b
x - ollama run llama3.2:1b
=> Error: 500 Internal Server Error: llama runner process has terminated: error loading model: unable to allocate CUDA0 buffer

y - ollama run gemma3:4b

x - ollama run starcoder2:3b
=> Error: 500 Internal Server Error: llama runner process has terminated: error loading model: unable to allocate CUDA0 buffer

x - ollama run falcon3:3b
x - ollama run phi4-mini-reasoning:latest
=> Error: 500 Internal Server Error: llama runner process has terminated: cudaMalloc failed: out of memory

=> Error: 500 Internal Server Error: llama runner process has terminated: error loading model: unable to allocate CUDA0 buffer
llama_model_load_from_file_impl: failed to load model

27

I just did a clean reinstall from the ISO SD image.

Two different errors!!!

ollama run gemma:2b
Error: 500 Internal Server Error: llama runner process has terminated: error loading model: unable to allocate CUDA0 buffer
llama_model_load_from_file_impl: failed to load model

ollama run gemma:2b
Error: 500 Internal Server Error: llama runner process has terminated: cudaMalloc failed: out of memory

I did not use docker for this, just the native and updated environment

28

Hi, both

This issue is related to the r36.4.7 update and has a failure rate.
You might see either ‘cudaMalloc failed: out of memory’ or ’ unable to allocate CUDA0 buffer’.
The underlying error for both cases are (in Ollama log):

NvMapMemAllocInternalTagged: 1075072515 error 12

Our internal team is working on the issue.
Will update more information with you later.

Thanks.

29

Thank you I have the same problem do you have any update yet

30

no, the developers said they are working on it (see above)

31

Hi, both

Sorry that we are still working on this issue.
You can check the topic below for more information.

Thanks.

33

These units are still being sold and are unusable for their intended purpose for MONTHS now.

If it is really this difficult to fix, then at least release a WORKING IMAGE with utilities installed and auto-updates that break them DISABLED!!!

The ISO-image on your server is a good option, but it needs some updates to be able to run and install Ollama and by now has other updates in the ever increasing list that breaks proper functioning!

34

I agree. Waiting months for a fix to something most users buy this device for is poor support. We should have some kind of a resolution by now.

35

Can Nvidia provide a workaround or reaccommodation on what version of jetpack to roll back to? Also is Cuda 13.1 approved for the Nano now and should we move forward to that version to solve the problem?

36

Its a problem with the new version. Older boards and version I have works perfectly. I just bought 2 new ones and now I’m sunk. ;[

38

Hi,

The latest CUDA for Orin is 12.9.

The issue is fixed internally. Please try the fix shared in the link below:

Thanks.

39

Please provide a downloadable image we can put on the device if this fix works.

40

Hi,

We will release an official release for this issue.
Will keep you updated.

Thanks.

42

Hi,

We have released JetPack 6.2.2 with r36.5 now.
Please upgrade your environment to fix this issue.

Thanks.

43

I will try it out if I can find the correct ISO file

I did notice that the link from the orin nano site seems to point to Jetpack 7 which is NOT compatible…

44

the correct link seems to be:

Is it correct there is no direct 6.2.2 iso but we are supposed to follow the instructions to upgrade from 6.2.1?

SD Card Image Method for Jetson Orin Nano Developer Kit

NOTE: Use the SD Card image of JetPack 6.2.1/Jetson Linux 36.4.4 and APT upgrade to JetPack 6.2.2/Jetson Linux 36.5.

45

Hi,

Please follow the document below to upgrade:

Thanks.

46

that procedure seems to involve the method using another computer as a host. I was asking about am ISO file to be flashed to SD-card

47

Surely, it is beneath a big player like NVIDIA to only have a broken OS image that has to be updated before a core functionality of the sold product can be used.

I got the patch installed eventually and it is an improvement. However, with bigger LLMs *that DID WORK previously) I still get an error message, even after a fresh boot with no other programs running:

ollama run gemma:7b
Error: 500 Internal Server Error: llama runner process has terminated: cudaMalloc failed: out of memory