open-webui VS llama.cpp

Luckily, there are some open-source projects like Open WebUI, which provide a web-based experience similar to ChatGPT, that you can also run locally and point to any model. To start the Open WebUI Docker container locally, run the command below in your Terminal (make sure, that ollama serve is still running).

I like the project.

What does this have that is better than https://github.com/open-webui/open-webui ?

Be careful with unstructured:

https://github.com/Unstructured-IO/unstructured/blob/d11c70c...

from: https://github.com/open-webui/open-webui/issues/687

For anyone else who missed the announcement a few hours ago, open-webui is the rebranding of the project formerly known as ollama-webui [0].

I can vouch for it as a solid frontend for Ollama. It works really well and has had an astounding pace of development. Every few weeks I pull the latest docker images and am always surprised by how much has improved.

[0] https://github.com/open-webui/open-webui/discussions/764

if you can compile stuff, then looking at llama.cpp (what ollama uses) is also interesting: https://github.com/ggerganov/llama.cpp

the server is here: https://github.com/ggerganov/llama.cpp/tree/master/examples/...

And you can search for any GGUF on huggingface

Yes, Metal seems to allow a maximum of 1/2 of the RAM for one process, and 3/4 of the RAM allocated to the GPU overall. There’s a kernel hack to fix it, but that comes with the usual system integrity caveats. https://github.com/ggerganov/llama.cpp/discussions/2182

For anyone interested in exploring this, llama.cpp has an example implementation here:

https://github.com/ggerganov/llama.cpp/tree/master/examples/...

Getting started with LLMs can be intimidating. In this tutorial we will show you how to fine-tune a large language model using LoRA, facilitated by tools like llama.cpp and KitOps.

well https://github.com/ggerganov/llama.cpp/issues/6849