Flowise VS llama.cpp

The GitHub repo is: https://github.com/FlowiseAI/Flowise

- https://github.com/FlowiseAI/Flowise

It's absolutely ok if the answer is "Yes", I think that in this hot market each product will find a place. And competition is also motivate :)

It would be also nice to add Rivet here:

If you're doing anything more custom or advanced than the regular ChatGPT type of interface can handle, you can use Flowise to build your own bot with any number of advanced plugins like internet search, calculators, recursion, file read/write access, long-term memory, other AI's...

Dove into Flowise Docs for a useful example.

For rapid prototyping try Flowise

Flowise as well, you are right!

Flowise

Drag & drop UI to build your customized LLM flow using LangchainJS (https://github.com/FlowiseAI/Flowise)

if you can compile stuff, then looking at llama.cpp (what ollama uses) is also interesting: https://github.com/ggerganov/llama.cpp

the server is here: https://github.com/ggerganov/llama.cpp/tree/master/examples/...

And you can search for any GGUF on huggingface

Yes, Metal seems to allow a maximum of 1/2 of the RAM for one process, and 3/4 of the RAM allocated to the GPU overall. There’s a kernel hack to fix it, but that comes with the usual system integrity caveats. https://github.com/ggerganov/llama.cpp/discussions/2182

For anyone interested in exploring this, llama.cpp has an example implementation here:

https://github.com/ggerganov/llama.cpp/tree/master/examples/...

Getting started with LLMs can be intimidating. In this tutorial we will show you how to fine-tune a large language model using LoRA, facilitated by tools like llama.cpp and KitOps.

well https://github.com/ggerganov/llama.cpp/issues/6849