Scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python. (by scrapy)

Scrapy Alternatives

Similar projects and alternatives to Scrapy

  1. Django

    536 Scrapy VS Django

    The Web framework for perfectionists with deadlines.

  2. InfluxDB

    InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

    InfluxDB logo
  3. Playwright

    478 Scrapy VS Playwright

    Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

  4. Pandas

    426 Scrapy VS Pandas

    Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

  5. Pytorch

    393 Scrapy VS Pytorch

    Tensors and Dynamic neural networks in Python with strong GPU acceleration

  6. puppeteer

    392 Scrapy VS puppeteer

    JavaScript API for Chrome and Firefox

  7. NumPy

    303 Scrapy VS NumPy

    The fundamental package for scientific computing with Python.

  8. examples

    177 Scrapy VS examples

    TensorFlow examples (by tensorflow)

  9. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  10. scikit-learn

    88 Scrapy VS scikit-learn

    scikit-learn: machine learning in Python

  11. Keras

    87 Scrapy VS Keras

    Deep Learning for humans

  12. seaborn

    83 Scrapy VS seaborn

    Statistical data visualization in Python

  13. NLTK

    69 Scrapy VS NLTK

    NLTK Source

  14. Robot Framework

    Generic automation framework for acceptance testing and RPA

  15. crawlee

    47 Scrapy VS crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

  16. colly

    41 Scrapy VS colly

    Elegant Scraper and Crawler Framework for Golang

  17. undetected-chromedriver

    Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

  18. lxml

    19 Scrapy VS lxml

    The lxml XML toolkit for Python

  19. scrapy-playwright

    🎭 Playwright integration for Scrapy

  20. estela

    10 Scrapy VS estela

    estela, an elastic web scraping cluster 🕸

  21. requests-html

    14 Scrapy VS requests-html

    Pythonic HTML Parsing for Humans™

  22. splash

    9 Scrapy VS splash

    Lightweight, scriptable browser as a service with an HTTP API

  23. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better Scrapy alternative or higher similarity.

Scrapy discussion

Log in or Post with
  1. User avatar
    7c91971f
    · 12 months ago
    · Reply

    Review ★★★★★ 10/10

Scrapy reviews and mentions

Posts with mentions or reviews of Scrapy. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2025-01-16.

Stats

Basic Scrapy repo stats
189
57,159
9.6
7 days ago

Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com

Did you know that Python is
the 2nd most popular programming language
based on number of references?