rarbg VS imdb-rename

Thank You for all the hard work you have been doing!!!! We understand this has been a lot! People would be extremely happy if you can add https://github.com/2004content/rarbg ! Please! Others have been asking also and getting no reply!

Install libreoffice and copy and paste one of the text versions from github.com/2004content/rarbg

https://github.com/rdavydov/rarbg-dump

LIVE at https://rdavydov.github.io/rarbg-dump/

Made a DB out of "everything.txt" from https://github.com/2004content/rarbg

Made a back-end with FastAPI that talks to the DB and serves requests

Made a front-end with day/night theme to search the DB via the back-end

Deployed the front-end to the GitHub Pages

Dump is fully migrated into 2 DBs, total number of magnets is 3.468.029.

The largest RARBG + EZTV dump so far, around 3.5 million magnets. No file sizes, dates or categories, only torrent names and hashes.

Now it finds keywords in any order. Like "x265 batman" or "rarbg batman".

Special thanks to 2004content for the SQLite backup, without it I won't be able to make this site.

You can get the full database here: https://github.com/2004content/rarbg

RARBG's magnet links have been preserved in multiple places, such as this one in github. https://github.com/2004content/rarbg

I wrote a tool that did something related a while back using IMDb data: https://github.com/BurntSushi/imdb-rename

This might be of interest: https://github.com/BurntSushi/imdb-rename

What a shit take. The article itself is perhaps a nice light overview of 101-ish level concepts, although knowing how and when to apply them in a real engineering context is not something I would consider 101 level. And certainly, building something that is actually at the scale of GitHub Search is nowhere near 101 level.

This is what a 101-level inverted index implementation looks like: https://github.com/BurntSushi/imdb-rename

In other words, absolutely nothing like what GitHub built. Nowhere close.

imdb-rename is an example of a tool that memory maps FSTs on disk in order to execute fulltext searches very quickly on the command line.

> How else would you lazy-load a database of (say) 32GB into memory, almost instantly?

That's what the fst crate[1] does. It's likely working at a lower level of abstraction than you intend. But the point is that it works, is portable and doesn't require any cooperation from the OS other than the ability to memory map files. My imdb-rename tool[2] uses this technique to build an on-disk database for instantaneous searching. And then there is the regex-automata crate[3] that permits deserializing a regex instantaneously from any kind of slice of bytes.[4]

I think you should maybe provide some examples of what you're suggesting to make it more concrete.

[1] - https://crates.io/crates/fst

[2] - https://github.com/BurntSushi/imdb-rename

[3] - https://crates.io/crates/regex-automata

[4] - https://docs.rs/regex-automata/0.1.9/regex_automata/#example...