Similar projects and alternatives to scrapy-redis
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
estela, an elastic web scraping cluster 🕸
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
Headless Chrome Node.js API
Command-line JSON processor
Elegant Scraper and Crawler Framework for Golang
Parsing HTML at the command line
The browserless Chrome service in Docker. Run on our cloud, or bring your own.
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
Issue tracking for the Steam for Linux beta client
Chromium Binary for AWS Lambda and Google Cloud Functions
A service daemon to run Scrapy spiders
curl-impersonate: A special build of curl that can impersonate Chrome & Firefox
Webscraping Open Project
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python
A triple-linked lists based DOM implementation.
OUTDATED!!!!! - Replaced by "The Bumblebee Project" and "Ironhide"
Google Search Results PHP API via Serp Api
Wistalk : Analyze Wikipedia User's Activity
A crawler/scraper based on golang + colly, configurable via JSON
a portable, lightweight web crawler using Powerpage.
Rank Wikipedia Article's Contributors by Byte Counts.
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
scrapy-redis reviews and mentions
Ask HN: What are the best tools for web scraping in 2022?
33 projects | news.ycombinator.com | 10 Aug 2022
11. With some work, you can use Scrapy for distributed projects that are scraping thousands (millions) of domains. We are using https://github.com/rmax/scrapy-redis.
rmax/scrapy-redis is an open source project licensed under MIT License which is an OSI approved license.