WebDumper
ChromeController
WebDumper | ChromeController | |
---|---|---|
2 | 1 | |
131 | 209 | |
- | - | |
0.0 | 3.3 | |
almost 3 years ago | 8 months ago | |
TypeScript | Python | |
- | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
WebDumper
-
Mastering Web Scraping in Python: Crawling from Scratch
I've had decent experience using [1] WebDumper to scrape SPA (single page applications), which rely almost entirely DOM rendering and client-side JS.
I'm curious if folks here had any other recommendations for scraping SPAs, ie React or Angular applications.
1: https://github.com/EllyMandliel/WebDumper
- WebDumper - A tool I made for unpaking Single Page Applications using source maps. It works pretty nicely.
ChromeController
What are some alternatives?
rod - A Devtools driver for web automation and scraping
skyscraper - Structural scraping for the rest of us.
grub-2.0 - Grub is an AI powered Web crawler.
ccl_chrome_indexeddb - (Sometimes partial) Python re-implementations of the technologies involved in reading various data sources in Chrome-esque applications.
selenium-python-helium - Lighter web automation for Python [Moved to: https://github.com/mherrmann/helium]
colly - Elegant Scraper and Crawler Framework for Golang
ArchiveBox - 🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...