estela-cli
mlscraper
| estela-cli | mlscraper | |
|---|---|---|
| 1 | 10 | |
| 4 | 1,229 | |
| - | - | |
| 5.8 | 0.6 | |
| 8 months ago | about 2 months ago | |
| Python | Python | |
| - | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
estela-cli
-
Deploying Scrapy Projects on the Cloud
We are currently running a closed beta of Bitmaker Cloud (free and unlimited). Bitmaker Cloud gives you easy management of scraping workloads via a web dashboard and API. Only Scrapy spiders are supported at the moment (additional languages/frameworks are on the roadmap). Bitmaker Cloud is powered by estela, an elastic web scraping cluster running on Kubernetes. estela is a modern alternative to proprietary platforms such as Scrapy Cloud, as well as OSS projects such as scrapyd. The source code of estela and estela-cli is available on Github.
mlscraper
-
What are the best tools for web scraping and analysis of natural language to populate a dataset?
See if something like autoscraper or mlscraper suits your needs.
-
Experimental library for scraping websites using OpenAI's GPT API
Why GPT-based then? There are libraries that do this: You give examples, they generate the rules for you and give you a scraper object that takes any html and returns the scraped data.
Mine: https://github.com/lorey/mlscraper
-
Could someone recommend me a library for c# like one of these two (they are for python) : mlscraper and autoscraper
GitHub - lorey/mlscraper: 🤖 Scrape data from HTML websites automatically by just providing examples
-
Smart Scraper
Check it out here: https://github.com/lorey/mlscraper Example: https://github.com/lorey/mlscraper/blob/master/examples/quotes\_to\_scrape.py
- Pre-trained Webscraping Models
- 🤖 Scrape data from HTML websites automatically by just providing examples
- mlscraper: Scrape data from HTML pages automatically with Machine Learning
-
Show HN: RSS feeds for arbitrary websites using CSS selectors
In case anyone wants to detect the selectors automatically, here's a small python library I wrote that does it for you: https://github.com/lorey/mlscraper
What are some alternatives?
Grab - Web Scraping Framework
scrapingant-client-python - ScrapingAnt API client for Python.
ttrss_plugin-feediron - Evolution of ttrss_plugin-af_feedmod
furss - Fix Up RSS (and atom): Make full-text versions of rss/atom feeds
feed-me-up-scotty
rssify - Tool that generates an rss feed out of websites that don't have one
RSSHub - 🧡 Everything is RSSible
feedgen - Generates RSS/ATOM/JSON feeds. Can be reasonably extended or create a feed using the CSS generator.
scrapy-proxycrawl-middleware - Scrapy middleware interface to scrape using ProxyCrawl proxy service
rssify - script that generates an rss feed out of websites that don't have one
sneakpeek - Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis
autoscraper - A Smart, Automatic, Fast and Lightweight Web Scraper for Python