Web Search and Scrape

This page summarizes the projects mentioned and recommended in the original post on /r/javahelp
Web Crawling Scraper Puppeteer Scraping Crawling

SurveyJS - Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App
With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
surveyjs.io
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

  • puppet-scraper

    Discontinued Scraping using Puppeteer the sane way 🤹🏻‍♂️

    • I'd look into scraping with JavaScript. Something like this could meet your needs.

  • Scrapy

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    • scrapy a python web scraping library

  • SurveyJS

    Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

    SurveyJS logo

  • crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

    • Crawlee a javascript web scraping and browser automation library

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Scrapy: A Fast and Powerful Scraping and Web Crawling Framework

    1 project | news.ycombinator.com | 16 Feb 2024

  • Implementing case sensitive headers in Scrapy (not through `_caseMappings`)

    4 projects | /r/scrapy | 3 Jul 2023

  • Dicas para projetos usando web scraping

    1 project | /r/brdev | 27 Jun 2023

  • Best tools to use for web scraping ??

    1 project | /r/learnpython | 25 Jun 2023

  • I'm using python to scrape web page content and extract keywords, how can I make it faster to process?

    1 project | /r/datascience | 10 Jun 2023