Went down the rabbit hole of 100% local RAG, it works but are there better options?

• ollama

  196 62,615 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

I used Ollama (with Mistral 7B) and Quivr to get a local RAG up and running and it works fine, but was surprised to find there are no easy user-friendly ways to do it. Most other local LLM UIs don't implement this use case (I looked here), even though it is one of the most useful local LLM use-cases I can think of: search and summarize information from sensitive / confidential documents.

• quivr

  22 32,386 9.9 TypeScript

  Your GenAI Second Brain 🧠 A personal productivity assistant (RAG) ⚡️🤖 Chat with your docs (PDF, CSV, ...) & apps using Langchain, GPT 3.5 / 4 turbo, Private, Anthropic, VertexAI, Ollama, LLMs, Groq that you can share with users ! Local & Private alternative to OpenAI GPTs & ChatGPT powered by retrieval-augmented generation.

  

I used Ollama (with Mistral 7B) and Quivr to get a local RAG up and running and it works fine, but was surprised to find there are no easy user-friendly ways to do it. Most other local LLM UIs don't implement this use case (I looked here), even though it is one of the most useful local LLM use-cases I can think of: search and summarize information from sensitive / confidential documents.

• SurveyJS

  surveyjs.io featured

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

• reor

  7 4,175 9.9 TypeScript

  Self-organizing AI note-taking app that runs models locally.

  

I am working on local RAG that encapsulates all the embedding & llm logic inside of an electron app. It is meant to be kind of a writing/personal knowledge management app powered by local models: https://github.com/reorproject/reor

• h2ogpt

  28 10,458 10.0 Python

  Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://codellama.h2o.ai/

  

Take a look at h2ogpt. It's open and local with API (incoming and outgoing) and impressive feature set, including RAG from docs, images, and web search.

• hallucination-leaderboard

  13 1,030 8.8

  Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents

  

Check this leaderboard, it is specific for RAG use case: https://github.com/vectara/hallucination-leaderboard

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project