calishot
By Krazybug
spider
spider is an OD crawler that crawls through opendirectories and indexes the urls (by pyDiablo)
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
calishot
Posts with mentions or reviews of calishot.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2021-06-08.
-
CALISHOT 2022-01: Find ebooks among 373 Calibre sites this month
Other resources will be proposed on it soon, like a wiki, tips, the datasets, original calibres, and some news about related tools like calisuck, calishot ... which are now turning into a single new project and will be released soon.
-
CALISHOT 2021-06: Find ebooks among 383 Calibre sites
If you want to build the db with your own list of servers, here is the python project on Github with commands to run on you own list of servers.
-
Need help with an OD indexer that I am writing in Python
This way you can also evolve your application to become async. As your using requests rather than aiohttp, may I suggest you to use gevent with a pool of requests in parallel (not too much ~ 10). You can look at this file as an example.
spider
Posts with mentions or reviews of spider.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2021-02-24.
-
lots of movies and tv shows
Crawled via spider by u/babars95
-
Need help with an OD indexer that I am writing in Python
The tips are based on his code as of 27-02-2021: https://github.com/pyDiablo/spider/tree/5662e9732456bcd6614c96a9813def196a0b9f89
What are some alternatives?
When comparing calishot and spider you can also consider the following projects:
open-directory-downloader - A NodeJS wrapper around KoalaBear84/OpenDirectoryDownloader
demeter - Demeter is a tool for scraping the calibre web ui
odcrawler-scanner - A reddit bot that scans ODs over at /r/OpenDirectories and submits the results to the ODCrawler discovery server
spider - scripts and baselines for Spider: Yale complex and cross-domain semantic parsing and text-to-SQL challenge
DiskCache - Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.
ODmovieindexer - Extract and index movie information of movies found in open directories posted on r/opendirectories.