bench-warmers VS khoj

i have more ideas than I know what to do with, help yourself: https://github.com/dmarx/bench-warmers

you're welcome to help yourself to my ideas (no guarantees that they're any good or even comprehensible, I do a lot of my brainstorming while high). here's my brainstorming space, scroll down for a categorized ToC: https://github.com/dmarx/bench-warmers

I've decided to just lean into it and am literally just giving my ideas away. https://github.com/dmarx/bench-warmers

it might sound like a lot, but it's actually really lightweight and easy to use. Check it out: https://github.com/dmarx/bench-warmers

you've sort of become victims of your own success: as another pandas dev mentioned, you want to preserve backwards compatibility and this significantly complicates any restructuring. I'm sympathetic and am not sure what the best solution here would be. I had this idea last night but i'm not sure I like this approach either.

To be clear, i'm talking about e.g. vision impaired, hearing impaired, etc. Here's an example of a project idea in this space (possibly a bit more ambitious than what you're looking for but if you think you could tackle this I encourage you to take a stab at it): https://github.com/dmarx/bench-warmers/blob/main/automated-video-description.md

There are already several RAG chat open source solutions available. Two that immediately come to mind are:

Danswer

https://github.com/danswer-ai/danswer

Khoj

https://github.com/khoj-ai/khoj

I'm a fan of Khoj. Been using it for months. https://github.com/khoj-ai/khoj

https://github.com/khoj-ai/khoj

This is the easiest I found, on here too.

Thanks for the feedback. Does your machine have a GPU? 32GB CPU RAM should be enough but GPU speeds up response time.

We have fixes for the seg fault[1] and improvement to the query speed[2] that should be released by end of day today[3].

Update khoj to version 0.10.1 with pip install --upgrade khoj-assistant to see if that improves your experience.

The number of documents/pages/entries doesn't scale memory utilization as quickly and doesn't affect the search, chat response time as much

[1]: The seg fault would occur when folks sent multiple chat queries at the same time. A lock and some UX improvements fixed that

[2]: The query time improvements are done by increasing batch size, to trade-off increased memory utilization for more speed

[3]: The relevant pull request for reference: https://github.com/khoj-ai/khoj/pull/393

We recently integrated Llama 2 into Khoj. I wanted to share a short real-world evaluation of using Llama 2 for the chat with docs use-cases and hear which models have worked best for you all. The standard benchmarks (ARC, HellaSwag, MMLU etc.) are not tuned for evaluating this

Self-host your Notion assistant using the instructions here. You'll need Python >= 3.8 to get started.

Here's an early example: https://github.com/khoj-ai/khoj