pdfGPT VS Parsr

From "Artificial intelligence is ineffective and potentially harmful for fact checking" (2023) https://news.ycombinator.com/item?id=37226233 : pdfgpt, knowledge_gpt, elasticsearch

pdfGPT: https://github.com/bhaskatripathi/pdfGPT :

> PDF GPT allows you to chat with the contents of your PDF file by using GPT capabilities.

GH "pdfgpt" topic: https://github.com/topics/pdfgpt

knowledge_gpt: https://github.com/mmz-001/knowledge_gpt

From https://news.ycombinator.com/item?id=39112014 : paperai

neuml/paperai:

What about systems like pdfgpt and knowledge_gpt; that, in returning citations from the trained texts, might be useful for legal discovery with admissibility?

pdfgpt: https://github.com/bhaskatripathi/pdfGPT

PDFChatter (Universal Sentence Encoder + GPT-3.5): out of the box, this solution works quite well. However, the need for Q&A prompting (and 'black-box' nature of the solution) seems to disqualify it as an optimal solution.

I was recommended pdfGPT and it looks good. Before I start setting it up, is there anything I should know? Are there better solutions? And how secure would it or they be?

https://github.com/bhaskatripathi/pdfGPT/blob/main/app.py it's a 200 line python script, all you need to do is rewrite pdf_to_text function for whatever format your file is. gpt4 can do it.

I'm part of the team that build LlamaParse. It's net improvement compare to other PDF->Structured Text extractors (I build several in the past, includig https://github.com/axa-group/Parsr).

For character extraction, LlamaParse use a mixture of OCR / character extraction from the PDF (it's the only parser I'm aware of that address some of the buggy PDF font issues, check the 'text' mode to see raw document before reconstruction), use a mixture of heuristic and Machine learning models to reconstruct the document.

Once plug with a Recursive retrieval strategy, allow you to get Sota result on question answering over complexe text (see notebook: https://github.com/run-llama/llama_parse/blob/main/examples/...).

AMA

Link to the Github Repository and pre defined Docker Compose Build file for Parsr

I would check out https://github.com/Unstructured-IO/unstructured (what lang chain uses) or https://github.com/axa-group/Parsr (probably what unstructured copied to get their startup off the ground lol)

Things I still want to try: - Parsr

I personally haven't used it yet, but I heard some good things about Parsr: https://github.com/axa-group/Parsr

Parsing PDFs is very non-trivial process. Google and Amazon parses are largely based on OCRing. There are some advanced state-of-the-art NN-based OCR approaches but they are not very stable, but a stable industry standard is Tesseract, and nice all-in-one open source tools that brings a ton of tools together is https://github.com/axa-group/Parsr . hope this helps