Scrape

Top 22 Scrape Open-Source Projects

  1. autoscraper

    A Smart, Automatic, Fast and Lightweight Web Scraper for Python

  2. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  3. cloudflare-scrape

    A Python module to bypass Cloudflare's anti-bot page.

  4. metascraper

    Get unified metadata from websites using Open Graph, Microdata, RDFa, Twitter Cards, JSON-LD, HTML, and more.

    Project mention: Show HN: I made a tool to clean and convert any webpage to Markdown | news.ycombinator.com | 2024-04-14
  5. twitter-api-client

    Implementation of X/Twitter v1, v2, and GraphQL APIs (by trevorhobenshield)

  6. Scweet

    A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers, user info, images...

  7. stweet

    Advanced python library to scrap Twitter (tweets, users) from unofficial API

  8. scrape

    Scrape any website, article or RSS/Atom Feed with ease!

  9. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  10. goq

    A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library

  11. raise

    A simple (and unofficial) GitHub Trending client that lives in your menubar.

  12. html2rss

    📰 Build RSS 2.0 feeds from websites (and JSON APIs) automatically or with a few CSS selectors.

  13. visdom

    A library use jQuery like API for html parsing & node selecting & node mutation, suitable for web scraping and html confusion. (by fefit)

  14. extract-css-core

    Extract all CSS from a given url, both server side and client side rendered.

  15. imgur-scraper

    Retrieve years of imgur.com's data without any authentication.

  16. squirm

    This was the night of the crawling terror!

  17. FONTS_DOT_COM_RIPPER

    Script to extract entire font families from Fonts.com, rips them as woff2 and final output includes woff2 and ttf files

  18. scrapyteer

    Web crawling & scraping framework for Node.js on top of headless Chrome browser

  19. Blind-App-Reviews

    Scraped reviews of over 25 companies from the Blind App ⚡️

  20. airbnb-scraper

    Apify public actor for scraping Airbnb homes.

  21. dozent

    Dozent is a powerful downloader that is used to collect large amounts of Twitter data from the internet archive.

  22. bchydro-outages

    Track BCHydro Outages via Git history

  23. real_estate_hungary

  24. weheartpy

    A fast, reliable API wrapper for weheartit.com

  25. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Scrape discussion

Log in or Post with

Scrape related posts

  • Show HN: AboutIdeasNow – search /about, /ideas, /now pages of 7k+ personal sites

    15 projects | news.ycombinator.com | 26 Feb 2024
  • Streamate closed my account

    1 project | /r/CamGirlProblems | 25 Jun 2023
  • Reverse Engineering Twitter Spaces - Capture 500 Audio Streams/Live Transcripts per IP

    1 project | /r/programming | 11 Jun 2023
  • Show HN: Twitter Spy Tools – Capture large volumes of audio and transcript data

    1 project | news.ycombinator.com | 1 Jun 2023
  • Twitter Spy Tools - Capture large volumes of audio and transcript data

    1 project | /r/programming | 1 Jun 2023
  • Veliko berem, da če nimaš službe, dobiš takoj zastonj občinsko stanovanje, kjer ni treba plačevati elektrike itd. Jaz bi tudi to naredila. Mi poveste, kako vsi to dobite, sklepam da je zelo lahko in vsak to dobi?

    1 project | /r/Slovenia | 20 May 2023
  • Twitter will be purging accounts with no activity for several years soon. We need to archive as many as we can. Any ideas on Methods

    1 project | /r/Archiveteam | 8 May 2023
  • A note from our sponsor - CodeRabbit
    coderabbit.ai | 9 Feb 2025
    Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR. Learn more →

Index

What are some of the best open-source Scrape projects? This list will help you:

# Project Stars
1 autoscraper 6,611
2 cloudflare-scrape 3,413
3 metascraper 2,394
4 twitter-api-client 1,708
5 Scweet 1,109
6 stweet 593
7 scrape 329
8 goq 262
9 raise 155
10 html2rss 122
11 visdom 112
12 extract-css-core 37
13 imgur-scraper 37
14 squirm 31
15 FONTS_DOT_COM_RIPPER 24
16 scrapyteer 19
17 Blind-App-Reviews 14
18 airbnb-scraper 11
19 dozent 7
20 bchydro-outages 7
21 real_estate_hungary 5
22 weheartpy 4

Sponsored
CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai

Did you know that Python is
the 2nd most popular programming language
based on number of references?