Top 8 headless-browser Open-Source Projects

ArchiveBox

248 19,915 9.8 Python

🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

Project mention: Ask HN: What Underrated Open Source Project Deserves More Recognition? | news.ycombinator.com | 2024-03-07

Two projects I greatly appreciate, allowing me to easily archive my bandcamp and GOG purchases (after the initial setup anyways):
https://github.com/easlice/bandcamp-downloader
https://github.com/Kalanyr/gogrepoc
And I recently learned about archivebox, which I think is going to be a fast favorite and finally let me clear out my mess of tabs/bookmarks: https://github.com/ArchiveBox/ArchiveBox

puppeteer-extra

28 6,122 0.0 JavaScript

💯 Teach puppeteer new tricks through plugins.

Project mention: What are your favorite Data Scraping tools? | /r/dataengineering | 2023-06-22

You could use https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth A plugin to escape anti bot detection

InfluxDB

www.influxdata.com featured

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
scrapy-playwright

11 847 7.8 Python

🎭 Playwright integration for Scrapy

Project mention: Web Scraping Dynamic Websites With Scrapy Playwright | dev.to | 2024-03-06

scrapy-playwright is an integration between Scrapy and Playwright. It enables scraping dynamic web pages with Scrapy by processing the web scraping requests using a Playwright instance.

html2image

1 322 6.7 Python

A package acting as a wrapper around the headless mode of existing web browsers to generate images from URLs and from HTML+CSS strings or files.

Project mention: Ask HN: What's your "it's not stupid if it works" story? | news.ycombinator.com | 2023-12-22

It uses the headless version of Chrome/Chromium or Edge behind the scenes.
It made me realize that even big projects have features that just don't work. Edge headless wouldn't let you take screenshots up until recently, and I still encountered issues with Firefox last time I tried to add support for it in the package. I also stumbled upon weird behaviors of Chrome CDP when trying to implement an alternative to using the headless mode, and these issues eventually fixed themselves after some Chrome updates.
[1] https://github.com/vgalin/html2image

marionette

2 177 2.8 Crystal

Selenium alternative for Crystal. Browser manipulation without the Java overhead.

Project mention: marionette: A Selenium alternative written in Crystal | /r/crystal_programming | 2023-06-20

Python-Selenium-Action

17 150 0.0 Python

Run Selenium with Python via Github Actions using Headless or Non-Headless browsers!

Project mention: Saturday Daily Thread: Resource Request and Sharing! Daily Thread | /r/Python | 2023-06-24

Shared this multiple times - but always seems to be helpful to someone. A GitHub action / template to run your Selenium based scripts on GitHub with ease. https://github.com/MarketingPipeline/Python-Selenium-Action

autoprotonvpn

1 10 0.0 JavaScript

1-click anon protonvpn account
SaaSHub

www.saashub.com featured

SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
phantomime

1 8 3.1 Python

An embeddable headless browser package for Python that provides a simplified interface for interacting with web pages using Selenium and Selenium Hub.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

headless-browser related posts

Show HN: Generate a concatenated file of all CSS used on a given website

3 projects | news.ycombinator.com | 25 Sep 2023
Saturday Daily Thread: Resource Request and Sharing! Daily Thread

1 project | /r/Python | 24 Jun 2023
marionette: A Selenium alternative written in Crystal

1 project | /r/crystal_programming | 20 Jun 2023
youtube bandwidth throttled for cloud addresses?

1 project | /r/youtubedl | 18 May 2023
Python-Selenium-Action: Run Selenium with Python via Github Actions!

2 projects | /r/github | 25 Mar 2023
Python-Selenium-Action: Easily Run Selenium with Python via Github Actions using Headless or Non-Headless browsers!

1 project | /r/Python | 25 Mar 2023
Ask HN: What's the best way to get all the HTML from a JavaScript site?

1 project | news.ycombinator.com | 5 Mar 2023
A note from our sponsor - SaaSHub
www.saashub.com | 15 May 2024

SaaSHub helps you find the best software and product alternatives Learn more →