After Ubuntu update it said, “Error: 500 Internal Server Error: llama runner process has terminated: error loading model: unable to allocate CUDA0 buffer llama_model_load_from_file_impl: failed to load model” I typically would run ollama from jetson-containers, and up until the recent update it wo…

Hi, The fix is available in JetPack 6.2.2/r36.5: [image] JetPack 6.2.2/Jetson Linux 36.5 is now live Jetson AGX Orin We are pleased to announce the production release of JetPack 6.2.2 . JetPack 6.2.2 packages Jetson Linux 36.5 with Linux Kernel 5.15 and Ubuntu…

I notice it takes a LONG time to load models on the Orin Nano in Llama.cpp, even small ones. The ‘freeze’ may be just that; long model load time. On my Thor, I can load Llama 70b Q4, which is 42 Gigs, in under 30 seconds, on Orin Nano it takes tens of minutes to load Llama 3B. HTH.

Hi, Yes, there will be a public release for r35 as well. Thanks

Just wanted to say thank you. This got me working again and thanks to Ashwin for the original work, you guys are amazing

ggilman, the script worked flawlessly for me as well and resolved the CUDA allocation error/memory fragmentation issues. Thank you for sharing it.

Is there still no official guidance? I’m uncomfortable running community scripts that have such a deep effect.

They released 36.5.. supposed to be fixing this.. I am still trying it now but thought I update this thread Update: it worked! [image] JetPack 6.2.2/Jetson Linux 36.5 is now live Announcements We are pleased to announce the production release of JetPack 6.2.2 .…

Thanks for the info. I follow the instructions. Looks like I can run the bigger models again, yay! By bigger models I mean up to 8.0B. BTW I followed the instruction on the update link, but could not run: sudo apt install --fix-broken -o Dpkg: :=“–force-overwrite” Received the following error: E: …

try this: sudo apt install --fix-broken -o Dpkg::Options::=“–force-overwrite”

Some NVIDIA repository entries were duplicated or incomplete, so APT could not detect the newer version. I disabled the old and duplicate repository file and kept one clean repository configuration. I updated the repository entries so that all required NVIDIA repositories (jetson/common, jetson/t234…

Thanks @ttron , executed without error, didn’t fix anything so that’s kool and the gang.

"unable to allocate CUDA0 buffer" after Updating Ubuntu Packages

Robotics & Edge Computing Jetson Systems Jetson Orin Nano

a7med.hish February 6, 2026, 6:46pm 241

They released 36.5.. supposed to be fixing this.. I am still trying it now but thought I update this thread

Update: it worked!

3 Likes

Topic		Replies	Views
Updating Orin Nano breaks Ollama Jetson Orin Nano cuda , generative_ai	26	1468	December 11, 2025
Ollama errors orin nano Jetson Orin NX nvbugs , generative_ai	43	2504	March 23, 2026
Llama3.2:3b randomly outputting "GGGGGGGG" when running under ollama on Jetson Orin Nano Super (JP6.2) Jetson Orin Nano generative_ai	43	1158	February 25, 2026
CUDA0 Buffer Error Jetson Orin Nano cuda , jetson , llama	4	399	January 5, 2026
Cuda0 Buffer Error Jetson Orin Nano cuda	12	1621	November 5, 2025
How to control amount of shared memory available to LLM on Jetson Thor? Jetson Thor generative_ai	21	1186	November 10, 2025
Introducing Ollama Support for Jetson Devices Jetson Projects cuda , natural-language-processing-nlp , artificialintelligence , interactive , docker-machine-learning , generative_ai	29	14025	August 28, 2024
Ollama and Jetson issue Jetson Orin NX jetson-inference , generative_ai	12	6255	March 20, 2024
Run llm stuck while use jetson thor Jetson Thor cuda , generative_ai	7	467	September 25, 2025
@Dusty_nv has anyone managed to get Ollama running with llama3.2-vision yet? Jetson AGX Orin cuda , generative_ai , llama	7	687	December 28, 2024

"unable to allocate CUDA0 buffer" after Updating Ubuntu Packages

Related topics