library-of-alexandria VS Archive.org-Downloader

After a quick Google search, I figured out that only less than 1% of ancient texts survived to the modern day. This unfortunate fact was my inspiration to start working on an ambitious web crawling and archival project, called the Library of Alexandria.

Here is a link to the latest releases: https://github.com/bottomless-archive-project/library-of-alexandria/releases

What do you mean by storage system? Just curious because I'm working on a similar project.

If anybody is interested about the project mentioned in the interview, it's available here: https://github.com/bottomless-archive-project/library-of-alexandria

I'm collecting documents. Working on an app suite called Library of Alexandria. Got 91 million docs atm (mostly PDFs) and it's only going up. All of that fits on around 100 TB with gzip compression.

I'm working on an OSS called Library of Alexandria. It is an application that is built to collect, archive, and make searchable various (mostly PDF) documents. I have a little bit more than 90 million documents archived. My next step is to somehow label/classify them.

90 TB of PDFs. I'm working on the Library of Alexandria project. Just a fun little library, nothing more. 😅😅😅

Not sure if the OP is saying they tried this already without success. I see a lot of people reporting problems recently getting ASCMs from IA to load the PDF, even with ADE. I've been using this with good results but it's not user friendly: https://github.com/MiniGlome/Archive.org-Downloader

By the way, you can skip the DRM process with tools like this, but yes I understand that there is a significant share of humanity who won't be able to use them.

No, I meant https://github.com/MiniGlome/Archive.org-Downloader. It's a command-line tool and works perfectly and very quickly for me. There are also several other options on GitHub - great minds think alike, I guess. Any of them is probably a better solution than getting individual pages one by one, which is definitely doing it the hard way.

also this: https://github.com/MiniGlome/Archive.org-Downloader

100% zoomed one is from archive.org-downloader.py

I've gotten quite a lot of mileage out of the Archive.org downloader.

I am trying to download books only available for 1 hour borrowing on the Internet Archive using this, but I am not the best at technology and seem to have hit a snag.