scrapy-redis

Redis-based components for Scrapy. (by rmax)

Scrapy-redis Alternatives

Similar projects and alternatives to scrapy-redis

  1. Playwright

    466 scrapy-redis VS Playwright

    Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

  2. Judoscale

    Save 47% on cloud hosting with autoscaling that just works. Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.

    Judoscale logo
  3. steam-for-linux

    Issue tracking for the Steam for Linux beta client

  4. puppeteer

    390 scrapy-redis VS puppeteer

    JavaScript API for Chrome and Firefox

  5. jq

    Discontinued Command-line JSON processor [Moved to: https://github.com/jqlang/jq] (by stedolan)

  6. bumblebee-Old-and-abbandoned

    OUTDATED!!!!! - Replaced by "The Bumblebee Project" and "Ironhide"

  7. pup

    Parsing HTML at the command line

  8. colly

    Elegant Scraper and Crawler Framework for Golang

  9. InfluxDB

    InfluxDB high-performance time series database. Collect, organize, and act on massive volumes of high-resolution data to power real-time intelligent systems.

    InfluxDB logo
  10. undetected-chromedriver

    Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

  11. curl-impersonate

    curl-impersonate: A special build of curl that can impersonate Chrome & Firefox

  12. google-search-results-php

    Google Search Results PHP API via Serp Api

  13. browserless

    21 scrapy-redis VS browserless

    Deploy headless browsers in Docker. Run on our cloud or bring your own. Free for non-commercial uses.

  14. powerpage-web-crawler

    a portable, lightweight web crawler using Powerpage.

  15. linkedom

    A triple-linked lists based DOM implementation.

  16. Webscraping Open Project

    Discontinued The web scraping open project repository aims to share knowledge and experiences about web scraping with Python [Moved to: https://github.com/TheWebScrapingClub/webscraping-from-0-to-hero]

  17. scrapyd

    6 scrapy-redis VS scrapyd

    A service daemon to run Scrapy spiders

  18. chrome-aws-lambda

    Chromium Binary for AWS Lambda and Google Cloud Functions

  19. estela

    estela, an elastic web scraping cluster 🕸

  20. wi-page

    Rank Wikipedia Article's Contributors by Byte Counts.

  21. polite

    Be nice on the web

  22. wistalk

    Wistalk : Analyze Wikipedia User's Activity

  23. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better scrapy-redis alternative or higher similarity.

scrapy-redis discussion

Log in or Post with

scrapy-redis reviews and mentions

Posts with mentions or reviews of scrapy-redis. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-06-26.

Stats

Basic scrapy-redis repo stats
4
5,590
6.0
10 months ago

Sponsored
Save 47% on cloud hosting with autoscaling that just works
Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.
judoscale.com

Did you know that Python is
the 2nd most popular programming language
based on number of references?