pikepdf VS django-pgpubsub

pikepdf – A Python library for reading and writing PDF, powered by qpdf

Not off the shelf but here are some tools. I have no experience with them.

Wolf binarization - I think it makes the text more clear before OCR.

https://github.com/chriswolfvision/local_adaptive_binarizati...

This thing OCRs the pdf using Tesseract OCR

https://github.com/ocrmypdf/OCRmyPDF/

Two other pdf tools

https://github.com/qpdf/qpdf

https://github.com/pikepdf/pikepdf

i recently transitioned a large PDF processing pipeline[1] away from PyPDF3 to use the pikepdf[2] library instead. pikepdf is based on the C++ `qpdf` library, and this switch has cut the necessary special-case manual checking + error handling in half compared to pypdf for these sorts of tasks.

[1]: i'm helping out with the CV Open Access archive, https://openaccess.thecvf.com/menu. broadly, our pipeline needs to ingest tons of author-provided PDFs of varying quality and output a canonical PDF with corrected page numbers, proper metadata, and a banner stamped on the first page. it's a lot of work, and it's not uncommon for pypdf to simply fail for slightly invalid input or give garbled results. this isn't optimal since we can't review 2,000 papers per conference release lol. pypdf has been nice, but pikepdf has handled everything i've thrown at it.

[2] https://pikepdf.readthedocs.io

First I would stick with celery if possible. I am not aware of something like that that is widely adopted. you could write your own crontab scripts or a combination of crontab script that writes to db with some listener like this https://github.com/Opus10/django-pgpubsub or use this one https://github.com/kraiz/django-crontab But both projects are not widely used or do not seem to be well maintained. I hope I was able to be of any help.

If you're using Postgres, you could consider https://github.com/Opus10/django-pgpubsub

django-pgpubsub – A distributed task processing framework for Django built on top of the Postgres NOTIFY/LISTEN protocol

Rather than relying on signals and celery, we may want to consider borrowing a pattern from the land of Postgres: Triggers and NOTIFY/LISTEN. The library django-pgpubsub offers a django application layer abstraction of this pattern. We can refactor our above solution using django-pgpubsub to look as follows:

You could try https://github.com/Opus10/django-pgpubsub. This is distributed task processing framework for Django built on top of the Postgres NOTIFY/LISTEN protocol. Does what you're looking for and isn't as operationally heavy as something like celery.

A common pattern in modern web development is the requirement to process data asynchronously after some user action or database event. In the below article, I describe via a concrete example a traditional approach to solving this problem for a Django/Postgres based application using django signals and Celery. I then proceed to discuss some of the shortcomings of this approach and demonstrate how using django-pgpubsub can offer a lightweight and more robust solution.