DeepRAG: Thinking to Retrieval Step by Step for Large Language Models

1. semantifly

   1 1 14 7.2 Go

   

   My company (actually our two amazing interns) was working on this over the summer, we abandoned it but it’s 85% of the way to doing what you want it to do: https://github.com/accretional/semantifly

   We stopped working on it mostly because we had higher priorities and because I became pretty disillusioned with top-K rag. We had to build out a better workflow system anyway, and with that we could instead just have models write and run specific queries (eg list all .ts files containing the word “DatabaseClient”), and otherwise have their context set by users explicitly.

   The problem with RAG is that simplistic implementations distract and slow down models. You probably need an implementation that makes multiple passes to prune the context down to what you need to get good results, but that’s complicated enough that you might want to build something else that gives you more bang for your buck.

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. chonkie

   2 3 2,845 9.8 Python

   🦛 CHONK your texts with Chonkie ✨ - The no-nonsense RAG chunking library

   

   Semantic chunking is where I would start with now. Also check this out: https://github.com/chonkie-ai/chonkie

4. tldw

   3 4 666 9.9 Python

   tl/dw (Too Long, Didn't Watch): Your Personal Research Multi-Tool - a naive attempt at 'A Young Lady's Illustrated Primer' (Open Source NotebookLM)

   

   Not the person you asked, but it's dependent on what you're trying to chunk. I've written a standalone chunking library for an app I'm building: https://github.com/rmusser01/tldw/blob/main/App_Function_Lib...

   It's setup so that you can perform whatever type of chunking you might prefer.

5. webwright

   4 3 39 9.0 Python

   Webwright is an AI-powered terminal emulator that lives within your OS. It eliminates time spent on repetitive tasks, conjures code, summons software, and bends the OS to its will. Are you ready to release the ghost in your shell?

   

   https://github.com/MittaAI/webwright

   Let me know if you want to go over the code or want to discuss what works and what doesn’t. We had a loop on the action/function call “pipeline” but I changed it to just test if there was a function call or not and then just keep calling.

Suggest a related project