I installed Open-WebUI using the playbook. For models that are downloaded from Ollama, it works well, especially for gpt-oss:120b. However, when I run a GGUF model downloaded locally using the “experimental” mode, I will get one response to a prompt and then never get another response to a subsequent prompt on the same thread. Any idea how to fix?
The other issue is I would like to have Ollama visible on port 11434 as well for other applications but it doesn’t appear. How to fix?
I think you are swimming upstream trying to run uploaded GGUF files in Ollama. a) It explicitly says experimental, so I wouldn’t expect it to work; b) pure llama.cpp is always ahead in features from the engine included in Ollama.
Recommendation:
Just use llama.cpp and pick a different port. Then in Open Webui “Admin Settings” you can add a connection for the new model by specifying the port #.
This playbook will work fine for almost any GGUF model with a little tweaking.
I did what you’ve suggested and have llama.cpp installed and serving up Nemotron on port 30000.
I also added “–network=host” to the open-webui docker script to fix the port 11434 problem, which now works so I can use Ollama in ComfyUI.
That’s great. Yeah, I didn’t read your full post. Now that I look, I’m assuming you are using the combined ollama/webui docker container in the playbook so I understand the network port issue. I’d also suggest grabbing the latest openwebui stand-alone docker because it’s had a number of improvements.
It’s all good now. Thanks!