crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation. (by apify)

Crawlee-python Alternatives

Similar projects and alternatives to crawlee-python

  1. Next.js

    2,483 crawlee-python VS Next.js

    The React Framework

  2. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  3. fastapi

    FastAPI framework, high performance, easy to learn, fast to code, ready for production

  4. Poetry

    Python packaging and dependency management made easy

  5. uv

    231 crawlee-python VS uv

    An extremely fast Python package and project manager, written in Rust.

  6. flow-pipeline

    A set of tools and examples to run a flow-pipeline (sFlow, NetFlow)

  7. zstd

    Zstandard - Fast real-time compression algorithm

  8. httpx

    A next generation HTTP client for Python. 🦋

  9. crawlee

    54 crawlee-python VS crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

  10. pyxel

    A retro game engine for Python

  11. undetected-chromedriver

    Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

  12. sqlite-utils

    Python CLI utility and library for manipulating SQLite databases

  13. jmespath.py

    JMESPath is a query language for JSON.

  14. playwright-python

    Python version of the Playwright testing and automation library.

  15. orjson

    Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

  16. botasaurus

    The All in One Framework to Build Undefeatable Scrapers

  17. httpcore

    A minimal HTTP client. ⚙️

  18. scrapy-playwright

    🎭 Playwright integration for Scrapy

  19. q2bs

    Python toolkit for analyzing industrial-scale automated content generation. Scrapes, analyzes, and visualizes blog publishing patterns. Built to investigate a case of automated plagiarism producing 8,400+ articles/day (Based on analysis of the period November 20 - December 7, 2025)

  20. aiofiles

    File support for asyncio

  21. how-to-make-web-scraping-faster

    How to Make Web Scraping Faster - Python Tutorial

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better crawlee-python alternative or higher similarity.

crawlee-python discussion

Log in or Post with

crawlee-python reviews and mentions

Posts with mentions or reviews of crawlee-python. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2025-09-30.
  • Launching Crawlee for Python v1.0 to simplify building web scrapers and crawlers
    1 project | news.ycombinator.com | 30 Sep 2025
  • How we moved Crawlee for Python out of Beta
    2 projects | dev.to | 30 Sep 2025
    Today is the day we move Crawlee for Python out of beta 🥳 In this post I will summarize the decisions and development that went into this process. I hope it will inform — and even validate — your product decisions.
  • How to scrape YouTube using Python [2025 guide]
    4 projects | dev.to | 16 Jul 2025
    In this guide, we'll explore how to efficiently collect data from YouTube using Crawlee for Python. The scraper will extract video metadata, video statistics, and transcripts - giving you structured YouTube data perfect for content analysis, ML training, or trend monitoring.
  • How to scrape TikTok using Python
    5 projects | dev.to | 30 Apr 2025
    Which hashtags are trending now? What is an influencer's engagement rate? What topics are important for a content creator? You can find answers to these and many other questions by analyzing TikTok data. However, for analysis, you need to extract the data in a convenient format. In this blog, we'll explore how to scrape TikTok using Crawlee for Python.
  • How to scrape Bluesky with Python
    6 projects | dev.to | 21 Mar 2025
    Bluesky is an emerging social network developed by former members of the Twitter(now X) development team. The platform has been showing significant growth recently, reaching 140.3 million visits according to SimilarWeb. Like X, luesky generates a vast amount of data that can be used for analysis. In this article, we’ll explore how to collect this data using Crawlee for Python.
  • Inside implementing SuperScraper with Crawlee.
    3 projects | dev.to | 5 Mar 2025
    Crawlee for Python is open for early adopters. 🐍 👉 Checkout the source code 👈.
  • How to scrape Crunchbase using Python in 2024 (Easy Guide)
    5 projects | dev.to | 15 Jan 2025
    By the end of this blog, we'll explore three different ways to extract data from Crunchbase using Crawlee for Python. We'll fully implement two of them and discuss the specifics and challenges of the third. This will help us better understand how important it is to properly choose the right data source.
  • How to scrape Google Maps data using Python and Crawlee
    2 projects | dev.to | 30 Dec 2024
    Note: Before going ahead with the project, I request to star Crawlee for Python on GitHub, it helps us to spread the world to fellow scraping developers.
  • How to scrape Google search results with Python
    2 projects | dev.to | 1 Dec 2024
    In this guide, we'll create a Google Search scraper using Crawlee for Python that can handle result ranking and pagination.
  • Web scraping of a dynamic website using Python with HTTP Client
    5 projects | dev.to | 28 Sep 2024
    The library already has support for an HTTP client that allows bypassing Cloudflare - CurlImpersonateHttpClient. Since we have to work with JSON responses we could use parsel_crawler added in version 0.3.0, but I think this is excessive for such tasks, besides I like the high speed of orjson.. Therefore, we'll need to implement our crawler rather than using one of the ready-made ones.
  • A note from our sponsor - SaaSHub
    www.saashub.com | 14 Jun 2026
    SaaSHub helps you find the best software and product alternatives Learn more →

Stats

Basic crawlee-python repo stats
15
9,157
9.7
3 days ago

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com

Did you know that Python is
the 1st most popular programming language
based on number of references?