Storm-crawler Alternatives

Similar projects and alternatives to storm-crawler

• Apache Nutch

  3 2,812 8.0 Java storm-crawler VS Apache Nutch

  Apache Nutch is an extensible and scalable web crawler

  

• jsoup

  27 10,645 9.1 Java storm-crawler VS jsoup

  jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Crawler4j

  2 4,493 0.0 Java storm-crawler VS Crawler4j

  Open Source Web Crawler for Java

• Sparkler

  0 409 3.0 Java storm-crawler VS Sparkler

  Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

  

• PeARS-orchard

  1 35 0.0 HTML storm-crawler VS PeARS-orchard

  This is the development version of PeARS, the people's search engine. More compact but less robust than PeARS-lite. If you just want to use PeARS as a local indexer, use PeARS-lite instead.

  

• scaling-to-distributed-crawling

  5 36 0.0 HTML storm-crawler VS scaling-to-distributed-crawling

  Repository for the Mastering Web Scraping in Python: Scaling to Distributed Crawling blogpost with the final code.

  

• crawlee

  29 12,129 9.8 TypeScript storm-crawler VS crawlee

  Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives