Top 6 Python python-scraper Projects

python-scraper

• Add a project

1. SeleniumBase

   1 22 11,572 9.7 Python

   Python APIs for web automation, testing, and bypassing bot-detection.

   

   Project mention: Scraping German Rental Price Data – Part I: Whole Lotta Captchas | news.ycombinator.com | 2025-07-29

   Not yet! But it's on my list to try out next after giving SeleniumBase[1] a chance.

   [1] https://github.com/seleniumbase/SeleniumBase

2. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

3. botasaurus

   2 8 2,952 9.6 Python

   The All in One Framework to Build Undefeatable Scrapers

   

   Project mention: 12 tips on how to think like a web scraping expert | dev.to | 2024-11-04

   I especially recommend paying attention to curl_cffi and frameworks botasaurus and Crawlee for Python.

4. scrapfly-scrapers

   3 5 615 8.8 Python

   Scalable Python web scraping scripts for +40 popular domains

   

   Project mention: A Comprehensive Guide to TikTok API | dev.to | 2025-03-20

   At scrapfly, we are dedicated to provide developer withs all the resources they need to reach their scraping goals. Check out our comprehensive guide on scraping tiktok as well as our example tiktok scraper using Scrapfly's APIs on github.

5. Reddit_Scrapper

   4 1 64 8.6 Python

   Scrape Reddit for marketing pain points using GPT — built to fuel Cronlytic's growth. (by Mohamedsaleh14)

   

   Project mention: Show HN: Open-source AI tool for Reddit lead generation | news.ycombinator.com | 2025-06-21

6. weheartpy

   5 2 4 7.3 Python

   A fast, reliable API wrapper for weheartit.com

   

7. youtube-scrapy-scraper

   6 1 1 2.3 Python

   Professional YouTube data scraper built with Scrapy framework. Extract video search results, channel analytics, subscriber counts, view metrics, and trending content. Features anti-bot protection, proxy rotation, CSV/JSON export, and comprehensive data validation. Perfect for content research, competitor analysis, and YouTube API alternatives.

   

   Project mention: Building a YouTube Scraper with Scrapy: A Developer's Journey | dev.to | 2025-07-10

   The complete code is available on GitHub: youtube-scrapy-scraper

Index

Sponsored

InfluxDB – Built for High-Performance Time Series Workloads

InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

www.influxdata.com

Do not miss the trending Python projects with our weekly report!