Python webscraper

Open-source Python projects categorized as webscraper

Top 17 Python webscraper Projects


    Python class to scrape data from and return listings in a pandas DataFrame object

  • Stocker

    Financial Web Scraper & Sentiment Classifier (by dwallach1)

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  • CoWin-Vaccine-Notifier

    Automated Python Script to retrieve vaccine slots availability and get notified when a slot is available.

  • iSubRip

    A Python package for scraping and downloading subtitles from AppleTV / iTunes movie pages.

    Project mention: I'm looking for official subtitles. | /r/Piracy | 2023-02-10
  • Jobs_LinkedIn

    Finds Jobs on LinkedIn using web-scraping

  • scraperx

    Library for scraping websites or apis at any scale

  • SearchifyX

    Fast flashcard searcher

    Project mention: FULL GUIDE FOR EDGENUITY | /r/edgenuity | 2023-02-02

    There's a program called searchifyx that scrapes brainly, quizziz, and Quizlet to find answers

  • Sonar

    Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

  • otakuapuri

    Otakuapuri is a manga downloader and anime streaming application that provides an easy and convenient platform for manga and anime enthusiasts. Users can download their favorite manga in PDF format and stream their favorite anime series.

    Project mention: Otakuapuri (v1.2.1) - Manga Downloader in PDF / Anime streaming [VF|VOSTFR] / User-friendly / Listen to / Watch openings-endings / MAL Ranking... | /r/coolgithubprojects | 2023-04-05
  • raspberry-pi-stock-checker

    A configurable python webscraper that checks raspberry pi stocks from verified sellers

  • letterboxdpy

    A letterboxd webscraper

  • YellowPage-scraper

    A YellowPage scraper is a Python program/script that extracts data from the website using the Python programming language. The scraper can be used to gather information such as business names, addresses, phone numbers, emails and reviews from the YellowPages website.

    Project mention: How do you get girl clothes in secret | /r/MtF | 2023-07-12
  • hes-dead-jim

    A command-line tool for finding and reporting dead/broken links in a file or webpage.

  • PotParser

    Python package which allows you to scrape information about cannabis strains and calculate the amount of THC or CBD in a given amount of flower

    Project mention: PotParser - a cli tool for getting information's about a strain from different websites | /r/trees | 2023-05-12
  • jobsearch-python-webcrawler

    Automatically scrap all the job titles and links, then organize into an excel file. Using Selenium and Openpyxl to scrape the information.

    Project mention: jobsearch-python-webscraper: Automatically scrap all the job titles and links, then organize into an excel file. Using Selenium and Openpyxl to scrape the information. | /r/CKsTechNews | 2022-12-21
  • CL-Checker

    An automated Craigslist webscrapper, written in Python for Windows.

    Project mention: Automated Email Alerts for Craigslist Search | /r/opensource | 2023-01-27
  • webcrawler

    This repository contains Python code for web crawling. It is built using the BeautifulSoup library and allows you to extract text from web pages and store it in text files. The crawler can also extract hyperlinks from web pages and crawl them recursively.This code will be a great starting point for your own web scraping projects

    Project mention: I have made a simple webscraper in python.pls checkout this github project. | /r/madeinpython | 2023-05-23
  • poolbooru_gelscraper

    a simple python script for scraping images off gelbooru pools.

  • Revelo Payroll

    Free Global Payroll designed for tech teams. Building a great tech team takes more than a paycheck. Zero payroll costs, get AI-driven insights to retain best talent, and delight them with amazing local benefits. 100% free and compliant.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-07-12.

Python webscraper related posts


What are some of the best open-source webscraper projects in Python? This list will help you:

Project Stars
1 212
2 Stocker 144
3 CoWin-Vaccine-Notifier 104
4 iSubRip 65
5 Jobs_LinkedIn 54
6 scraperx 53
7 SearchifyX 51
8 otakuapuri 16
9 raspberry-pi-stock-checker 13
10 letterboxdpy 10
11 YellowPage-scraper 6
12 hes-dead-jim 5
13 PotParser 3
14 jobsearch-python-webcrawler 2
15 CL-Checker 2
16 webcrawler 2
17 poolbooru_gelscraper 1
Learn any GitHub repo in 59 seconds
Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at