ColossalAI VS PaddleOCR

> open-source a complete RLHF pipeline ... based on the LLaMA pre-trained model

I've gotten to where when I see "open source AI" I now know it's "well, except for $some_other_dependencies"

Anyway: https://scribe.rip/@yangyou_berkeley/colossalchat-an-open-so... and https://github.com/hpcaitech/ColossalAI#readme (Apache 2) can save you some medium.com heartache at least

Quick Read: https://www.marktechpost.com/2023/04/01/meet-colossalchat-an-open-source-ai-solution-for-cloning-chatgpt-with-a-complete-rlhf-pipeline/ Github: https://github.com/hpcaitech/ColossalAI Examples: https://chat.colossalai.org/

One of the current methods for training competing models is to have ChatGPT literally create prompt -> completion data sets. That's what was used for https://github.com/hpcaitech/ColossalAI. A model based off of the Llama weights released by facebook, then fine tuned on ChatGPT3.5 prompt + completions. So yes, there is a good chance that google is literally using ChatGPT in the training loop.

Here's the github from the article:

https://github.com/hpcaitech/ColossalAI

The article talks about their RLHF implementation briefly. There’s details on their RLHF implementation here: https://github.com/hpcaitech/ColossalAI/blob/a619a190df71ea3...

Here’s the project on GitHub: https://github.com/hpcaitech/ColossalAI

PyTesseract Module [ Github ] EasyOCR Module [ Github ] PaddlePaddle OCR [ Github ]

My default recommendation for OCR is https://github.com/PaddlePaddle/PaddleOCR but most of the examples there are not handwritten - so I'm not sure how well it'll handle it this time.

Other than EasyOCR and Tesseract, PaddleOCR (https://github.com/PaddlePaddle/PaddleOCR) is probably the most well known open-source OCR solution.

What are you planning to do with the text after detecting / recognizing it? How fast does the detection / recognition need to be in order to be useful?

Yup! But I'm still exploring options. (any recommendations would be welcomed!) Here are some candidates I'm considering:

- https://github.com/mindee/doctr

- https://github.com/open-mmlab/mmocr

- https://github.com/PaddlePaddle/PaddleOCR (honestly I don't know Mandarin so I'm a bit stuck)

- https://github.com/clovaai/donut - While it's primarily an "OCR-free document understanding transformer," I think it's worth experimenting with. Think I can sort this out by letting the LLM reason through it multiple times (although this will impact performance)

- yesterday got a suggestion to consider https://github.com/kakaobrain/pororo - I don't think development is still active but the results are pretty great on Korean text

For images with text, if you want to do visual qa, document classification, table/key information extraction, checkout https://huggingface.co/blog/document-ai https://github.com/philschmid/document-ai-transformers https://github.com/google-research/pix2struct https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.6/ppstructure/README.md

EasyOCR is a popular project if you are in an environment where you can use run Python and PyTorch (https://github.com/JaidedAI/EasyOCR). Other open source projects of note are PaddleOCR (https://github.com/PaddlePaddle/PaddleOCR) and docTR (https://github.com/mindee/doctr).

I think you can train tesseract with custom data if you have enough, or you can use deep learning models like https://pyimagesearch.com/2020/08/17/ocr-with-keras-tensorflow-and-deep-learning or https://www.google.com/amp/s/nanonets.com/blog/attention-ocr-for-text-recogntion/amp/ or try other existing tools like paddle-ocr https://github.com/PaddlePaddle/PaddleOCR

https://huggingface.co/blog/document-ai https://github.com/microsoft/table-transformer https://github.com/google-research/pix2struct https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.6/ppstructure/table/README.md

when following the installation commands present in the paddleocr repo(https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.6/doc/doc_en/quickstart_en.md) im still unable to install paddleocr. paddlepaddle is successfully installed on my m1 mac with python3.9.16 but while installing paddleocr im getting this error after long pip backtracking

When I was evaluating options a few months ago I found https://github.com/PaddlePaddle/PaddleOCR to be a very strong contender for my use case (reading product labels), but you'll definitely want to put together some representative docs/images and test a bunch of solutions to see what works for you.