Chunking your data for RAG

1. txtai

   1 385 10,749 9.6 Python

   💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

   

2. Judoscale

   judoscale.com featured

   Save 47% on cloud hosting with autoscaling that just works. Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.

   

3. chonkie

   2 3 2,845 9.8 Python

   Discontinued 🦛 CHONK your texts with Chonkie ✨ - The no-nonsense RAG chunking library [GET https://api.github.com/repos/chonkie-ai/chonkie: 404 - Not Found // See: https://docs.github.com/rest]

   

   The Textractor extracts chunks of text from files and the Embeddings takes those chunks and builds an index/database. We'll use a late chunker backed by Chonkie.

4. txtai.js

   3 13 155 5.2 JavaScript

   JavaScript client for txtai

   

5. txtai.java

   4 14 38 5.1 Java

   Java client for txtai

   

6. txtai.rs

   5 8 111 5.2 Rust

   Rust client for txtai

   

7. txtai.go

   6 12 78 0.0 Go

   Go client for txtai

   

8. transformers

   7 207 143,133 10.0 Python

   🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

   

9. InfluxDB

   influxdata.com featured

   InfluxDB high-performance time series database. Collect, organize, and act on massive volumes of high-resolution data to power real-time intelligent systems.

   

10. fastapi

    8 4 75,036 - Python

    Discontinued FastAPI framework, high performance, easy to learn, fast to code, ready for production [Moved to: https://github.com/fastapi/fastapi] (by tiangolo)

    

11. rag

    9 3 352 6.9 Python

    🚀 Retrieval Augmented Generation (RAG) with txtai. Combine search and LLMs to find insights with your own data.

    

12. ragdata

    10 1 17 4.5 Python

    📚 Build knowledge bases for RAG

    

13. paperai

    11 22 1,392 5.3 Python

    📄 🤖 Semantic search and workflows for medical/scientific papers

    

14. annotateai

    12 3 315 7.6 Python

    📝 Automatically annotate papers using LLMs

    

Suggest a related project