ChromeController
WebDumper
ChromeController | WebDumper | |
---|---|---|
1 | 2 | |
209 | 131 | |
- | - | |
3.3 | 0.0 | |
8 months ago | almost 3 years ago | |
Python | TypeScript | |
GNU General Public License v3.0 or later | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ChromeController
WebDumper
-
Mastering Web Scraping in Python: Crawling from Scratch
I've had decent experience using [1] WebDumper to scrape SPA (single page applications), which rely almost entirely DOM rendering and client-side JS.
I'm curious if folks here had any other recommendations for scraping SPAs, ie React or Angular applications.
1: https://github.com/EllyMandliel/WebDumper
- WebDumper - A tool I made for unpaking Single Page Applications using source maps. It works pretty nicely.
What are some alternatives?
rod - A Devtools driver for web automation and scraping
grub-2.0 - Grub is an AI powered Web crawler.
skyscraper - Structural scraping for the rest of us.
ccl_chrome_indexeddb - (Sometimes partial) Python re-implementations of the technologies involved in reading various data sources in Chrome-esque applications.
selenium-python-helium - Lighter web automation for Python [Moved to: https://github.com/mherrmann/helium]
colly - Elegant Scraper and Crawler Framework for Golang
ArchiveBox - 🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...