SaaSHub helps you find the best software and product alternatives Learn more →
Top 18 Python web-scraper Projects
-
Monkey-DL (Anime Downloader)
Bulk download your favourite anime episodes from your favourite anime websites
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
web-scraping
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
-
summarizer
A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.
-
facebook_page_scraper
Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
-
CobWeb-lnx
CobWeb is a Python library for web scraping. The library consists of two classes: Spider and Scraper.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
tagalog-dictionary-scraper
Builds a Tagalog dictionary by collecting Tagalog words from tagalog.pinoydictionary.com
-
mexican-jobs-2020
Data ETL & Analysis on thousands of job listings from the official Mexican job board (2020 edition).
-
tweet-transcriber
A Reddit bot that transcribes tweets from comments and submissions links, mirrors their images and replies back with a formatted Markdown message.
-
Python-Web-Scraper
An adaptive Python Web Scraper App to catch the best deals by scraping and parsing data from select E-Commerce sites.
-
Abosar
অবসর 📚 A collection of short Bengali stories web scraped from various Bengali eMagazines and eNewspapers.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Use Lightnovel crawler on a computer in terminal or in their discord bot to find series across multiple LN / webnovel sites then choose the format to download (epub,pdf, txt, and many more)
It's been a cool learning experience making a Product Hunt listing, a small demo video, and allll the social posts (Twitter, LinkedIn, etc).
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python web-scraper related posts
- Help with Paperback IOS.
- Multiparadigmatic Web Scraping Tool!
- ISSTH left me disappointed
- a discord server and bot to fetch epub chapters from novels?
- Python Web Scraper/Crawler for E-Commerce sites. Currently supports only a few websites but im looking to expand that list. Tips/criticism are welcomed. This is the first project for my student CV (0 working experience) so I'd like it to be as polished as possible.
- Wat is jullie ervaring met e-readers?
- Does the kindle have a search function? A working one? I’ve seen videos but those are like years old.
-
A note from our sponsor - SaaSHub
www.saashub.com | 25 Apr 2024
Index
What are some of the best open-source web-scraper projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | lightnovel-crawler | 1,283 |
| 2 | Monkey-DL (Anime Downloader) | 811 |
| 3 | web-scraping | 617 |
| 4 | summarizer | 267 |
| 5 | facebook_page_scraper | 195 |
| 6 | Senpwai | 128 |
| 7 | CobWeb-lnx | 38 |
| 8 | reddit-bots | 23 |
| 9 | tagalog-dictionary-scraper | 22 |
| 10 | mexican-jobs-2020 | 21 |
| 11 | tweet-transcriber | 19 |
| 12 | git-pull | 16 |
| 13 | Python-Web-Scraper | 12 |
| 14 | Abosar | 12 |
| 15 | varieteebot | 3 |
| 16 | gli99 | 2 |
| 17 | iw-scraper | 0 |
| 18 | nanoscrape | 0 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com