With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js. Learn more →
Top 17 TypeScript Scraping Projects
-
crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
-
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
-
-
facebook-group-members-scraper
Facebook Group Members Extractor. Download Facebook group members in CSV.
-
scraper
Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom. (by get-set-fetch)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
freenom-auto-renew-domains
A scraper built with puppeteer that auto renew free domains on Freenom and send discord message using bot
-
-
timetable-grabber-sit
Timetable Grabber - SIT is a tool that allows you to grab and export your trimester's timetable to the .ics format where you can import it to your favourite calendar.
-
linguabook.github.io
Just-in-time scraper of linguistic information from different sources like Cambridge, Merriam-Webster. It also has a satellite Chrome Extension to see linguistic information for selected word while you surfing the internet
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
In this guide, we'll be extracting information from Amazon product pages using the power of TypeScript in combination with the Cheerio and Crawlee libraries. We'll explore how to retrieve and extract detailed product data such as titles, prices, image URLs, and more from Amazon's vast marketplace. We'll also discuss handling potential blocking issues that may arise during the scraping process.
Project mention: Tutorial: Extracting structured data from websites using Groq and Firecrawl | news.ycombinator.com | 2024-04-22
Project mention: How to quickly grab all members of a AWDTSG Group - Database w/full names, pictures, an ddetails coming soon to know who to avoid | /r/AWDTSGisToxic | 2023-07-01For those who want to do it for themselves: https://github.com/floriandiud/facebook-group-members-scraper
Install SerpApi package Don't forget to install the API package you want to use. In this sample, we need to install the serpapi package*:* https://github.com/serpapi/serpapi-javascript
Here's the repo if anyone wants to take a look: https://github.com/MaximumOverflow/Philia-GUI/releases
Hi guys, I've created an open-source low-code Node.js web scraping tool on top of the Puppeteer - https://github.com/miroshnikov/scrapyteer. It offers a small set of functions that are combined in pipelines to define a crawling workflow and a shape of output data. Maybe somebody will find it useful.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
TypeScript Scraping related posts
- Connect OpenAI with external APIs with Function calling
- Build and run your Python web scrapers in the cloud with Apify SDK for Python
- GitHub - le-quentin/free-stock-tickers: Freely fetch live stock data by scraping web pages
- Free backend app to integrate live stock prices in your spreadsheets
- For developers: free open source backend app to fetch stock prices directly in your spreadsheet
- Web Search and Scrape
- Crawlee · Build reliable crawlers. Fast. | Crawlee
-
A note from our sponsor - SurveyJS
surveyjs.io | 24 Apr 2024
Index
What are some of the best open-source Scraping projects in TypeScript? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | crawlee | 12,044 |
| 2 | firecrawl | 1,659 |
| 3 | fingerprint-suite | 692 |
| 4 | secret-agent | 633 |
| 5 | libremdb | 257 |
| 6 | facebook-group-members-scraper | 157 |
| 7 | scraper | 98 |
| 8 | freenom-auto-renew-domains | 48 |
| 9 | serpapi-javascript | 40 |
| 10 | Philia | 23 |
| 11 | headless-task-server | 21 |
| 12 | timetable-grabber-sit | 19 |
| 13 | scrapyteer | 16 |
| 14 | game-watch | 12 |
| 15 | dofus-scraper | 7 |
| 16 | pattern-grab | 7 |
| 17 | linguabook.github.io | 1 |
Sponsored
The modern identity platform for B2B SaaS
The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
workos.com