The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 17 JavaScript Crawler Projects
-
browser-fingerprinting
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web?
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
work_crawler
Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫画 webtoons 咚漫 ニコニコ静画 ComicWalker ヤングエースUP モアイ pixivコミック サイコミ;アルファポリス カクヨム ハーメルン 小説家になろう 起点中文网 八一中文网 顶点小说 落霞小说网 努努书坊 笔趣阁→epub.
-
fakebrowser
🤖 Fake fingerprints to bypass anti-bot systems. Simulate mouse and keyboard operations to make behavior like a real person.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
th-music-video-generator
Touhou Project random music video generator/player, crawling image and video from websites to generate MV.
-
images-downloader
A Node.js module for downloading a single image or multiple images to disk from a given Url
-
CodexDrake
An open source, privacy-first, self-hosting capable and blazing fast search engine written in JavaScript. Browse anonymously and safely without the need to pay third-party APIs. 👀
-
finance-news-crawler
Finance News Crawler uses News API to fetch some latest articles and generates a sentiment report with the OpenAI API or VADER
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Project mention: A site that tracks the price of a Big Mac in every US McDonald's | news.ycombinator.com | 2024-01-13Yes, there is a lot written about it. Here is one link I have saved:
https://github.com/niespodd/browser-fingerprinting
Project mention: How do Instapaper and Pocket apps extract the content of the articles? | /r/opensource | 2023-12-04Edit: I found this library in NodeJs useful for article extraction. Anyone looking for something like you can take a look. https://github.com/extractus/article-extractor
Project mention: Show HN: A Node.js script powered by Puppeteer for undetectable web scraping | news.ycombinator.com | 2024-01-17
You can view the package on NPM and you can look at the code on Github
Project mention: Would it be worth publishing my Chrome Extension even if I anticipate that it will have very few users? | /r/developersIndia | 2023-06-16I have recently developed a Chrome extension called Netflix Hotkeys, which adds a variety of shortcuts to enhance the Netflix experience. Initially, I created it for my own personal use, but I believe it could be beneficial for others as well. To cater to a wider audience, I have incorporated additional shortcuts and created a comprehensive readme. However, I am now hesitant about publishing it on the Chrome Web Store due to the expected lack of traction. I am concerned that it might not gain much popularity and could be a disappointing endeavor.
JavaScript Crawler related posts
- Cool ChatGPT Finance Sentiment Analysis
- GitHub - simwai/finance-news-crawler: Finance News Crawler uses News API to fetch some latest articles and generates a sentiment report with the OpenAI API or VADER
- Would it be worth publishing my Chrome Extension even if I anticipate that it will have very few users?
- Who here is developing extensions?
- Netflix Hotkeys: A Chrome Extension to enhance your Netflix Experience
- Netflix Hotkeys: A Chrome Extension to enhance your Netflix Experience
- FAQs on my side project
-
A note from our sponsor - WorkOS
workos.com | 24 Apr 2024
Index
What are some of the best open-source Crawler projects in JavaScript? This list will help you:
Project | Stars | |
---|---|---|
1 | node-crawler | 6,612 |
2 | browser-fingerprinting | 3,830 |
3 | work_crawler | 2,848 |
4 | google-play-scraper | 2,218 |
5 | article-extractor | 1,375 |
6 | fakebrowser | 1,043 |
7 | sitemap-generator | 395 |
8 | JSSoup | 360 |
9 | th-music-video-generator | 266 |
10 | undetectable-crawler | 22 |
11 | images-downloader | 19 |
12 | selector-finder | 19 |
13 | airbnb-scraper | 9 |
14 | Netflix-Hotkeys | 9 |
15 | CodexDrake | 7 |
16 | tumblweed | 6 |
17 | finance-news-crawler | 4 |
Sponsored