Top 7 Python web-crawler Projects

PSpider

0 1,811 0.0 Python

简单易用的Python爬虫框架，QQ交流群：597510560
kochat

1 442 0.0 Python

Opensource Korean chatbot framework
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
spidy Web Crawler

0 323 0.0 Python

The simple, easy to use command line web crawler.
Ignareo-ISML-auto-voter

1 189 4.5 Python

Ignareo the Carillon, a web crawler/spider template of ultimate high concurrency built for leprechauns. Carillons as the best web spiders; Long live the golden years of leprechauns! (ISML=international saimoe; 2022 ISML is last ISML)
GoodreadsScraper

1 115 0.0 Python

Scrape data from Goodreads using Scrapy and Selenium :books:
CobWeb-lnx

5 38 4.7 Python

CobWeb is a Python library for web scraping. The library consists of two classes: Spider and Scraper.

Project mention: Quem já contribuiu e quem já usou projectos open-source? | /r/devpt | 2023-06-30

Python

1 4 6.3 Python

This repository contains the python source code, containing more than 40 python projects, involving many fields.仓库用于储存python源代码, 包含40多个python项目，涉及爬虫、算法、OpenGL、tkinter、面向对象编程等多个领域。 (by qfcy)

Project mention: Potentially malicious package | /r/learnpython | 2023-12-05

Recently, I tried to install the PyObject package, but accidentally ran pip install pyobject. The install failed with a Unicode error, but I am assuming that some code was run during the setup process. The main repo seems to be here. I'm just paranoid that it might be malicious, as there is a file called myMalware.py in the root of the project. I've looked over it and am of the opinion that it is most likely safe, but would just like a second opinion.

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).