bloop VS llama.cpp

https://github.com/BloopAI/bloop Is fully open source and has full text + regex search built on tantivy fyi

Thanks! Bloop really helped with the documentation! they index your code base and uses RAG locally to talk against it. It works shockingly well.

In this blog post, I’ll be comparing 3 distinct AI-first code search tools I recently came across: Cody (developed by late-stage startup, Sourcegraph), SeaGOAT (an open-source project that was trending on HN last week), and Bloop (an early-stage YC startup). I’ll be evaluating them along the dimensions of user-friendliness as well as their accuracy.

If you're confused about any of the code snippets above, you can check out bloop.ai and phind.com (along with its VSCode extension) to answer any of your questions about the repository, noting that both have free plans.

COMPANY: bloop (https://bloop.ai)

bloop (YC S21) | Rust AI Engineer | Onsite | London, UK

We're a YC + tier-1 VC backed startup, working on a AI chat assistant for software developers that can answer questions about any large codebase.

About you: you hack around with GPT on the weekend, maybe you even fine-tune some of these new open source models. But you're also a solid programmer, capable of not just prototyping a nice demo but also shipping performant code that can scale. You're probably quite good at what you do, maybe you were top of your class, or maybe you rose the ranks to your comfortable management role, but yearn for the days of writing code, redbull and deploying on a Friday.

We're a small team, looking for one individual to join us in London, in-person at least 3 days per week.

Please reach out to me at join [at] bloop [dot] ai with "HN AI Engineer" in the subject line and a note about why this is exciting and a link to your portfolio or linkedin/resume.

P.S. Our stack is Rust, so it's likely the role will go to a candidate with Rust experience but if you think you have something to offer that compensates for lack of Rust we should chat anyway!

https://github.com/BloopAI/bloop

Bro let me turn your life inside out: https://bloop.ai

Another example is Bloop, it is a code search engine built on top of tantivy and qdrant.

Bloop is a developer assistant that uses GPT-4 to answer questions about your codebase (https://github.com/BloopAI/bloop)

well https://github.com/ggerganov/llama.cpp/issues/6849

Have just done this recently for local chat with pdf feature in https://recurse.chat. (It's a macOS app that has built-in llama.cpp server and local vector database)

Running an embedding server locally is pretty straightforward:

- Get llama.cpp release binary: https://github.com/ggerganov/llama.cpp/releases

Ah, thanks for this! I can't edit my parent comment that you replied to any longer unfortunately.

As I said, I only compared the contributors graphs [0] and checked for overlaps. But those apparently only go back about year and only list at most 100 contributors ranked by number of commits.

[0]: https://github.com/ollama/ollama/graphs/contributors and https://github.com/ggerganov/llama.cpp/graphs/contributors

KodiBot is a desktop app that enables users to run their own AI chat assistants locally and offline on Windows, Mac, and Linux operating systems. KodiBot is a standalone app and does not require an internet connection or additional dependencies to run local chat assistants. It supports both Llama.cpp compatible models and OpenAI API.

There are already some implementations out there which attempt to accomplish this!

Here's an example: https://github.com/silphendio/sliced_llama

A gist pertaining to said example: https://gist.github.com/silphendio/535cd9c1821aa1290aa10d587...

Here's a discussion about integrating this capability with ExLlama: https://github.com/turboderp/exllamav2/pull/275

And same as above but for llama.cpp: https://github.com/ggerganov/llama.cpp/issues/4718#issuecomm...

For those who might not be aware of this, there is also an open source project on GitHub called "Twinny" which is an offline Visual Studio Code plugin equivalent to Copilot: https://github.com/rjmacarthy/twinny

It can be used with a number of local model services. Currently for my setup on a NVIDIA 4090, I'm running both the base and instruct model for deepseek-coder 6.7b using 5_K_M Quantization GGUF files (for performance) through llama.cpp "server" where the base model is for completions and the instruct model for chat interactions.

llama.cpp: https://github.com/ggerganov/llama.cpp/

deepseek-coder 6.7b base GGUF files: https://huggingface.co/TheBloke/deepseek-coder-6.7B-base-GGU...

deepseek-coder 6.7b instruct GGUF files: https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct...