quivr VS khoj

3. Quivr | GitHub | tutorial

I used Ollama (with Mistral 7B) and Quivr to get a local RAG up and running and it works fine, but was surprised to find there are no easy user-friendly ways to do it. Most other local LLM UIs don't implement this use case (I looked here), even though it is one of the most useful local LLM use-cases I can think of: search and summarize information from sensitive / confidential documents.

In my opinion LLM based document search tools such as OSS Quivr may be better suited for documentation search for startups.

A highly customed Quivr with one of the 'Open Source LLMs' may provides great 'semantic search' for product documentation.

https://github.com/StanGirard/quivr

Hey thanks for creating this, will try later if i have time. Meanwhile, do you try some of other second brain app such as this, and how was the comparison? The one i mentioned was trending on github so i think its decent (been playing with it since last week or so, also). But i already starred your repo so i can come back later.

There are already several RAG chat open source solutions available. Two that immediately come to mind are:

Danswer

https://github.com/danswer-ai/danswer

Khoj

https://github.com/khoj-ai/khoj

I'm a fan of Khoj. Been using it for months. https://github.com/khoj-ai/khoj

https://github.com/khoj-ai/khoj

This is the easiest I found, on here too.

Thanks for the feedback. Does your machine have a GPU? 32GB CPU RAM should be enough but GPU speeds up response time.

We have fixes for the seg fault[1] and improvement to the query speed[2] that should be released by end of day today[3].

Update khoj to version 0.10.1 with pip install --upgrade khoj-assistant to see if that improves your experience.

The number of documents/pages/entries doesn't scale memory utilization as quickly and doesn't affect the search, chat response time as much

[1]: The seg fault would occur when folks sent multiple chat queries at the same time. A lock and some UX improvements fixed that

[2]: The query time improvements are done by increasing batch size, to trade-off increased memory utilization for more speed

[3]: The relevant pull request for reference: https://github.com/khoj-ai/khoj/pull/393

We recently integrated Llama 2 into Khoj. I wanted to share a short real-world evaluation of using Llama 2 for the chat with docs use-cases and hear which models have worked best for you all. The standard benchmarks (ARC, HellaSwag, MMLU etc.) are not tuned for evaluating this

Self-host your Notion assistant using the instructions here. You'll need Python >= 3.8 to get started.

Here's an early example: https://github.com/khoj-ai/khoj