Headless Browser for Web Scraping: Usage Features

This page summarizes the projects mentioned and recommended in the original post on dev.to
Testing Frameworks Runner headless-chrome Testing Web

SurveyJS - Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App
With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
surveyjs.io
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

  • puppeteer

    Node.js API for Chrome

    • Puppeteer by Google is a Node.js library providing an API for headless browser control via DevTools Protocol. Originally, it was an automated testing library, but it has been successfully used for web scraping. Supported browsers: Chrome / Chromium, limited – Firefox. Supported programming languages: Node.js; has an unofficial Python library, “Pyppeteer.”

  • kimuraframework

    Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites

    • Kimurai is a Web Scraping framework for Ruby with headless browser functionality. Supported browsers: Chromium and Firefox. Supported programming languages: Ruby.

  • SurveyJS

    Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

    SurveyJS logo

  • Playwright

    Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

    • Playwright.js by Microsoft is a recent alternative to Puppeteer. It positions itself as end-to-end testing for modern web apps. Its primary purpose is testing, and it has an excellent toolset, including the error tracer capable of capturing and investigating the test failure. The same rendering engine works for mobile browsers, desktop, and the cloud. Supported browsers: Chromium, Firefox, and WebKit. Supported programming languages: Playwright API can be used with TypeScript, JavaScript, Python, .NET, and Java.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • A question about web-scraping

    1 project | /r/learnprogramming | 1 Sep 2023

  • Can a puppetry major survive a flagship’s financial crisis? Should it?

    1 project | news.ycombinator.com | 10 Jul 2023

  • Is there a mobile program that can find specific words on screen and click on them?

    1 project | /r/AskProgramming | 9 Jul 2023

  • landing page website image storing.

    1 project | /r/Supabase | 25 Jun 2023

  • Help! How to download an entire website with it contents behind a login page

    1 project | /r/webdev | 22 Jun 2023