SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 TypeScript web-scraping Projects
-
Project mention: firecrawl VS lookacrawler - a user suggested alternative | libhunt.com/r/firecrawl | 2026-09-03
-
Kargo
Stop Scripting Promotions. Start Shipping with Kargo. Kargo automates promotion across dev, staging, and prod with approval gates and verification. Open source, built by the team behind Argo CD. Download now.
-
crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Project mention: Fingerprinting a website's tech stack from a single HTTP request (no headless browser) | dev.to | 2026-09-05One plain HTTP fetch of the homepage via Crawlee's CheerioCrawler, then check three things against a signature table:
-
maxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
-
wigolo
The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.
-
Project mention: How to Build a LinkedIn Outreach Pipeline (Without Getting Your Account Banned) | dev.to | 2026-06-12
What worked was switching to Patchright, a Playwright fork that patches those patterns at the Chrome DevTools Protocol level, before any page JavaScript runs. It was a one-line replacement: import { chromium } from 'patchright' and the rest of the Playwright API stays the same. We paired that with installing real google-chrome-stable in the image instead of relying on bundled Chromium, which gives a more believable TLS handshake. LinkedIn's edge can fingerprint the handshake itself through JA3 and JA4, and bundled Chromium has a slightly different signature from the Chrome a real person runs.
-
dembrandt
Extract any website’s design system into tokens in seconds: logo, colors, typography, borders & more. One command.
On the tools front: ExposedByDefault reveals everything your browser hands out automatically (eye-opening privacy check!), Remend auto-closes broken markdown mid-stream (essential for AI content generation), Walrus delivers 1.2M writes/sec on consumer hardware as the Rust-powered Kafka alternative, and Dembrandt extracts any site's design tokens with one npx command—logo, colors, typography, the works!
-
google-search
A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration. (by web-agent-master)
Project mention: Show HN: Google Dawg – Python program to automate Google dorking | news.ycombinator.com | 2025-11-24I have an assignment to make a google dork program from my professor and this is the result. Currently on beta, try it out if you are interested.
So this is a program to automate google dorking for penetration testing and educational purposes. It use web-agent-master/google-search to search the dork. google-search simulate human behaviour so it's kind of bot detection proof. Though captcha might showed up every now and them. Do check them out if you are interested: https://github.com/web-agent-master/google-search
google-dawg features:
-
AppSignal
Monitoring that respects your time & budget. APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product.
-
reader
Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents. (by vakra-dev)
Project mention: Reader – web scraping that outputs clean Markdown for LLMs | news.ycombinator.com | 2026-02-21 -
teracrawl
High-performance web crawler API optimized for LLMs. Turn any search or website into clean Markdown using remote browsers. Firecrawl alternative
Project mention: Stop crawling my HTML you dickheads – use the API | news.ycombinator.com | 2025-12-14The only way you can block these "AI" scrapers is a combination of IP filtering (https://spur.us/) and Fingerprinting (https://abrahamjuliot.github.io/creepjs/).
Things like browserbase are easy to block with this. It's a losing battle though, personally moved entirely to real environments for https://browser.cash/developers
-
GEO AEO Tracker
Open-source, local-first AI visibility intelligence dashboard. Track your brand across 6 AI models.
-
-
charlotte
Token-efficient browser MCP server — structured web pages for AI agents, not raw accessibility dumps (by TickTockBent)
Project mention: Anatomy of a GitHub Actions Supply Chain Attack Targeting MCP Repos | dev.to | 2026-04-08Charlotte is a browser automation MCP server. The PR came from an account called internet-dot and added .github/workflows/hol-skill-validate.yml:
-
geoleadscraper
Free open-source Google Maps scraper — Chrome extension to extract business leads (name, address, phone, website, email) from Google Maps, Yandex Maps & 2GIS and export to CSV/Excel. No API key, no account.
Project mention: Show HN: GeoLeadScraper – Open-source Google Maps scraper (Chrome extension) | news.ycombinator.com | 2026-06-11 -
browserclaw
The AI-native browser automation library. Snapshot + ref targeting — born from OpenClaw, built for agents, by agents.
Here's a complete AI browser agent using browserclaw and Claude:
-
super-scraper
Generic REST API for scraping websites. Drop-in replacement for ScrapingBee, ScrapingAnt, and ScraperAPI services. And it is open-source!
-
crawlee-cloud
Self-hosted, open-source platform for running Apify Actors. Drop-in compatible with the Apify SDK.
Project mention: Show HN: Self-hosted Apify-compatible platform for web scraping | news.ycombinator.com | 2025-12-26- CLI: Push, run, and manage Actors from your terminal
Stack: Node.js, Fastify, PostgreSQL, Redis, S3/MinIO, Next.js
GitHub: https://github.com/crawlee-cloud/crawlee-cloud
Happy to answer questions!
-
veilbrowser
Stealth browser for AI agents — real Chrome over raw CDP, no Playwright/Puppeteer. TypeScript + MCP-native. Passes sannysoft 57/57, bypasses Cloudflare.
Project mention: Show HN: Veil – stealth browser for AI agents (real Chrome no Playwright) | news.ycombinator.com | 2026-06-09 -
-
-
-
cbrowser
Cognitive Browser: The browser automation that thinks. Constitutional safety • Persona UX testing • Natural language interface • Self-healing selectors • Built for AI agents
Project mention: Show HN: CBrowser – Simulate how a confused first-timer experiences your website | news.ycombinator.com | 2026-02-27 -
-
capmonster-nodejs-captcha-solver
Official Node.js / TypeScript SDK for CapMonster Cloud. Bypass reCAPTCHA, Cloudflare Turnstile, DataDome in Puppeteer and Playwright scripts.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
TypeScript web-scraping discussion
TypeScript web-scraping related posts
-
Fingerprinting a website's tech stack from a single HTTP request (no headless browser)
-
Atlas – open-source deep research you own
-
hasdata-cli VS crawlee - a user suggested alternative
2 projects | 4 Jun 2026 -
Building a self-hosted browser scraping service (is it more hassle than its worth?)
-
I Built 34 Web Scrapers — Here's What I Learned About Anti-Bot Detection
-
BrowserClaw – Accessibility snapshot and ref targeting for AI browser agents
-
I Built 23 Free Web Scrapers on Apify — Here is What I Learned
-
A note from our sponsor - SaaSHub
www.saashub.com | 8 Sep 2026
Index
What are some of the best open-source web-scraping projects in TypeScript? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | firecrawl | 176,641 |
| 2 | crawlee | 25,664 |
| 3 | maxun | 17,361 |
| 4 | wigolo | 5,161 |
| 5 | patchright | 4,273 |
| 6 | dembrandt | 3,299 |
| 7 | google-search | 609 |
| 8 | reader | 558 |
| 9 | teracrawl | 282 |
| 10 | GEO AEO Tracker | 250 |
| 11 | ayakashi | 216 |
| 12 | charlotte | 178 |
| 13 | geoleadscraper | 46 |
| 14 | browserclaw | 44 |
| 15 | super-scraper | 43 |
| 16 | crawlee-cloud | 42 |
| 17 | veilbrowser | 42 |
| 18 | botasaurus-starter | 36 |
| 19 | LeMondeRssReader | 35 |
| 20 | scrapyteer | 20 |
| 21 | cbrowser | 19 |
| 22 | atlas | 17 |
| 23 | capmonster-nodejs-captcha-solver | 16 |
Sponsored
Stop Scripting Promotions. Start Shipping with Kargo
Kargo automates promotion across dev, staging, and prod with approval gates and verification. Open source, built by the team behind Argo CD. Download now.
akuity.io