localGPT VS koboldcpp

I think local LLMs are great for tinkerers, and with quantization can run on most modern PCs. I am not comfortable sending over my personal data over to OpenAI/Anthropic, so I've been playing around with https://github.com/PromtEngineer/localGPT/, GPT4All, etc. which keep the data all local.

Sliding window chunking, RAG, etc. seem more sophisticated than the other document LLM tools, so I would love to try this out if you ever add the ability to run LLMs locally!

localGPT can parse PDF into embeddings, see <https://github.com/PromtEngineer/localGPT>.

This is going so fast that it feels like a new thing pops up every day. LocalGPT seems to have gotten a lot of traction though: https://github.com/PromtEngineer/localGPT

i am currently trying to use localGPT (https://github.com/PromtEngineer/localGPT) for a project and i encountered a problem.

I tinker a lot with electronics. I have put datasheets for components, documentation for development boards, documentation for software libraries, etc into a database with localGPT.

There are a bunch of other methods to improve quality and performance like tree-of-thought-llm, connecting a LLM to a database or have it review its own output.

Check out local gpt on git hub. I tried but it had slow response for me. Other developers are fine. https://github.com/PromtEngineer/localGPT

FYI this is the easiest way to host on the horde: https://github.com/LostRuins/koboldcpp

For more information of Koboldcpp look this guide: https://github.com/LostRuins/koboldcpp/wiki

Out of curiosity, is there a specific reason for this? The most popular fork KoboldCpp is in active development, and was the first to adopt the Min P sampler, and even distincts itself with the context shift feature. Just wondering what this means for the future. Thanks!