Writing a simple Web crawler in python

This page summarizes the projects mentioned and recommended in the original post on /r/u_DevGenious
Python webscraper Webscraping webscraping-beautifulsoup webscraping-data

InfluxDB
Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
Our great sponsors

  • webcrawler

    This repository contains Python code for web crawling. It is built using the BeautifulSoup library and allows you to extract text from web pages and store it in text files. The crawler can also extract hyperlinks from web pages and crawl them recursively.This code will be a great starting point for your own web scraping projects

    • I have a wrote a simple web crawler in python which can scan an entire website and store its contents into text files github link : https://github.com/ksn-developer/webcrawler.git

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts