Where to start: Learning Web-scraping

This page summarizes the projects mentioned and recommended in the original post on /r/learnpython

Scout Monitoring - Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
  • Scrapy

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    The most well-known one (and actually the only one I know) is scrapy. I won't go into too much detail, but among others it offers:

  • Scout Monitoring

    Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

    Scout Monitoring logo
  • lxml

    The lxml XML toolkit for Python

    lxml is an XML parser however, it also supports HTML parsing. It's blazing fast and supports XPath. I think it isn't as beginner friendly to use, though it has detailed documentation. It works less well with heavily broken HTML documents and the encoding detection isn't as good as the one of BS4.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Scrapy: A Fast and Powerful Scraping and Web Crawling Framework

    1 project | news.ycombinator.com | 16 Feb 2024
  • Implementing case sensitive headers in Scrapy (not through `_caseMappings`)

    4 projects | /r/scrapy | 3 Jul 2023
  • Dicas para projetos usando web scraping

    1 project | /r/brdev | 27 Jun 2023
  • Best tools to use for web scraping ??

    1 project | /r/learnpython | 25 Jun 2023
  • I'm using python to scrape web page content and extract keywords, how can I make it faster to process?

    1 project | /r/datascience | 10 Jun 2023

Did you konow that Python is
the 1st most popular programming language
based on number of metions?