spider
crab
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
spider
crab
-
Crab – offline first scraper & parser in rust
So here it is – https://github.com/bazhenov/crab. The project is in early stage, but it is functional and I was able to parse several sites I need for research purposes. Feedback appreciated.
What are some alternatives?
colly - Elegant Scraper and Crawler Framework for Golang
doonop - Doonop is a crawler and scrapper on behalf. Focused on collecting arbitrary information from an entire site or its part.
crusty-core - A small library for building fast and highly customizable web crawlers
Crawler4j - Open Source Web Crawler for Java
scraping-with-rust - 👾 scraping hacker news with rust
rust-bitcoin-indexer - Powerful & versatile Bitcoin Indexer, in Rust
nipper - A Rust crate for manipulating HTML with CSS selectors
chan-downloader - CLI to download all images/webms in a 4chan thread
hltv-rust - A client to fetch and parse data from HLTV.org (origin at https://foss.alic.dev/dist1ll/hltv-rust)
shrike - Data analysis infrastructure for the Neo N3 blockchain.