surya
llmsherpa
| surya | llmsherpa | |
|---|---|---|
| 6 | 6 | |
| 6,743 | 993 | |
| - | 16.2% | |
| 8.4 | 6.6 | |
| 5 days ago | 10 days ago | |
| Python | Jupyter Notebook | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
surya
-
New open source AI model for document segmentation and unstructured ETL
Would this be able to incorporate the models from Surya —
https://github.com/VikParuchuri/surya
-
Show HN: Beyond text splitting – improved file parsing for LLM's
This looks great! You might be interested in surya - https://github.com/VikParuchuri/surya (I'm the author). It does OCR (much more accurate than tesseract), layout analysis, and text detection.
The OCR is slow on CPU (working on it), but faster than tesseract (CPU-only) on GPU.
Happy to discuss more, feel free to email me (in profile).
-
LlamaCloud and LlamaParse
You may want to try https://github.com/VikParuchuri/surya (I'm the author). I've only benchmarked against tesseract, but it outperforms it by a lot (benchmarks in repo). Happy to discuss.
You could also try https://github.com/VikParuchuri/marker for general PDF parsing (I'm also the author) - it seems like you're more focused on tables.
- Show HN: Surya – OCR and line detection in 93 languages
- Surya: Multilingual Document OCR Toolkit
llmsherpa
-
LlamaCloud and LlamaParse
To get good RAG performance you will need a good chunking strategy. Simply getting all the text is not good enough and knowing the boundaries of table, list, paragraph, section etc. is helpful.
Great work by llamaindex team. Also feel free to try https://github.com/nlmatics/llmsherpa which takes into account some of the things I mentioned.
-
Show HN: Open-source Rule-based PDF parser for RAG
I wrote about split points and the need for including section hierarchy in this post: https://ambikasukla.substack.com/p/efficient-rag-with-docume...
All this is automated in the llmsherpa parser https://github.com/nlmatics/llmsherpa which you can use as an API over this library.
What are some alternatives?
cmdf - this thing will fix misspelled commands by learning from your history.
unstructured - Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.
stable-diffusion-webui - Stable Diffusion web UI
txtai - 💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows
Auto-GPT - An experimental open-source attempt to make GPT-4 fully autonomous. [Moved to: https://github.com/Significant-Gravitas/Auto-GPT]
llama_parse - Parse files for optimal RAG
Parsr - Transforms PDF, Documents and Images into Enriched Structured Data
marker - Convert PDF to markdown quickly with high accuracy
paperetl - 📄 ⚙️ ETL processes for medical and scientific papers
nlm-ingestor - This repo provides the server side code for llmsherpa API to connect. It includes parsers for various file formats.
open-webui - User-friendly WebUI for LLMs (Formerly Ollama WebUI)
llama-hub - A library of data loaders for LLMs made by the community -- to be used with LlamaIndex and/or LangChain