mldht
shiny-adventure
| mldht | shiny-adventure | |
|---|---|---|
| 2 | 1 | |
| 144 | 0 | |
| - | - | |
| 10.0 | 10.0 | |
| over 2 years ago | 8 months ago | |
| Java | Python | |
| Mozilla Public License 2.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
mldht
-
Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally
Another alternative is https://github.com/the8472/mldht which, contrary to magnetico, strives to be a nice citizen (its author is active in the bittorrent community AFAIU)
-
Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
Oh, I already have implemented something[0] like that. It doesn't need lambdas or anything "cloud scale" like that. You "just" need a few dozen to a hundred IP addresses assigned to one machine and run a multi-homed DHT node on that to passively observe traffic from multiple points in the keyspace.
But neither of these approaches is what I'd call a "super simple DHT crawler CLI tool" that the initial comment was asking about. BEP51 is intended to make crawling simple enough that it can run on a single home internet connection, but a proper implementation still isn't trivial.
[0] https://github.com/the8472/mldht
shiny-adventure
-
Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
> The DHT crawler is not quite unique to bitmagnet; another open-source project, magnetico was first (as far as I know) to implement a usable DHT crawler, and was a crucial reference point for implementing this feature.
Heh. That was one of my first projects when I was still learning to code back in 2012.
https://github.com/laino/shiny-adventure
The DHT crawler/worker lived seperately, and I eventually put it here to rescue it from a dying HDD:
https://github.com/laino/DHT-Torrent-database-Worker
The code is abhorrent and you absoltely shouldn't use it, but it worked.
What are some alternatives?
dhtcrawler2 - dhtcrawler is a DHT crawler written in erlang. It can join a DHT network and crawl many P2P torrents. The program save all torrent info into database and provide an http interface to search a torrent by a keyword
bitmagnet - A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI, GraphQL API and Servarr stack integration.
DHT-Torrent-database-Worker - I was learning to code. No bully. Old project built sometime in 2012. Crawls the DHT network, downloads torrent metadata from peers and uploads that torrent information into a database.
rarbg-selfhosted - A self-hosted Torznab API for the RARBG backup, compatible with Prowlarr, Radarr, Sonarr etc.
content-seeder
Magnetico - Autonomous (self-hosted) BitTorrent DHT search engine suite.