Python web-scraper

Open-source Python projects categorized as web-scraper | Edit details

Top 14 Python web-scraper Projects

  • lightnovel-crawler

    Generate and download e-books from online sources.

    Project mention: EpubPress, turn web content into ebooks | news.ycombinator.com | 2022-04-15
  • Monkey-DL (Anime Downloader)

    Bulk download your favourite anime episodes from your favourite anime websites

  • Scout APM

    Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.

  • onlyfans-dl

    OnlyFans content downloader

    Project mention: Working method for downloading paid Onlyfans content? | reddit.com/r/Piracy | 2021-12-11

    GitHub

  • web-scraping

    Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

  • summarizer

    A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.

    Project mention: ¿Cómo se llamaba el alma de código que resumía noticias?¿Sigue vivo? | reddit.com/r/mexicow | 2022-02-21

    https://github.com/PhantomInsights/summarizer y https://github.com/PhantomInsights/tweet-transcriber

  • facebook_page_scraper

    Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV

  • mexican-jobs-2020

    Data ETL & Analysis on thousands of job listings from the official Mexican job board (2020 edition).

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • tweet-transcriber

    A Reddit bot that transcribes tweets from comments and submissions links, mirrors their images and replies back with a formatted Markdown message.

    Project mention: ¿Cómo se llamaba el alma de código que resumía noticias?¿Sigue vivo? | reddit.com/r/mexicow | 2022-02-21

    https://github.com/PhantomInsights/summarizer y https://github.com/PhantomInsights/tweet-transcriber

  • git-pull

    Parallelized web scraper for Github

  • tagalog-dictionary-scraper

    Builds a Tagalog dictionary by collecting Tagalog words from tagalog.pinoydictionary.com

    Project mention: Any techy people here know how to convert some json dictionary scrapes into a kindle friendly format? | reddit.com/r/Tagalog | 2021-08-10

    I'm wondering the same thing! My only lead so far is this scraper that could maybe be repurposed and this guide on how to get dictionaries correctly formatted for kindle.

  • reddit-bots

    A collection of Reddit bots that I use to enhance the subreddits I manage.

  • varieteebot

    A telegram bot that sends today's tee of some tee shops.

  • nanoscrape

    Simple scraping program that can download webpages, Discord embeds, and more.

  • iw-scraper

    Web scraper for imovelweb listings

    Project mention: Feedback: Web Scraper para o imovelweb | reddit.com/r/datasciencebr | 2022-01-08
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-04-15.

Python web-scraper related posts

Index

What are some of the best open-source web-scraper projects in Python? This list will help you:

Project Stars
1 lightnovel-crawler 635
2 Monkey-DL (Anime Downloader) 619
3 onlyfans-dl 349
4 web-scraping 299
5 summarizer 238
6 facebook_page_scraper 41
7 mexican-jobs-2020 20
8 tweet-transcriber 15
9 git-pull 14
10 tagalog-dictionary-scraper 13
11 reddit-bots 13
12 varieteebot 3
13 nanoscrape 0
14 iw-scraper 0
Find remote jobs at our new job board 99remotejobs.com. There are 9 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com