Top 5 Python web-crawler Projects

PSpider

0 1,727 4.4 Python

简单易用的Python爬虫框架，QQ交流群：597510560
kochat

0 335 0.0 Python

Opensource Korean chatbot framework
InfluxDB

www.influxdata.com
sponsored

Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Data Platform where developers build real-time applications for analytics, IoT and cloud-native services in less time with less code.
spidy Web Crawler

0 297 2.7 Python

The simple, easy to use command line web crawler.
Ignareo-ISML-auto-voter

0 181 6.8 Python

Ignareo the Carillon, a web crawler/spider template of ultimate high concurrency built for leprechauns. Carillons as the best web spiders; Long live the golden years of leprechauns! (2022 ISML is last ISML)
GoodreadsScraper

1 68 4.1 Python

Scrape data from Goodreads using Scrapy and Selenium :books:

Project mention: [OC] Top 10 Fantasy Books of the 21st Century According to GoodReads | reddit.com/r/dataisbeautiful | 2022-05-20

Used this code to get the data: https://github.com/havanagrawal/GoodreadsScraper
Sonar

www.sonarsource.com
sponsored

Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-05-20.