Exploring Methods to Improve Text Chunking in RAG Models (and other things...)

• vectorboard

  2 43 5.5 Python

  Open Source Embeddings Optimisation and Eval Framework for RAG/LLM Applications. Documentations at https://docs.vectorboard.ai/introduction

  

Hi, about chunking, if the text is structured (markdown or html), you can take headding and paragraph as a chunking unit, but the result is also affected by the applied embeddings, which can be evaluated separately first, for standard chunking methods with different chunk lengths, for example with this tool https://github.com/VectorBoard/vectorboard.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project