Top 13 JavaScript Scraper Projects
-
Project mention: ❗Slow Download Speed in 4K Video Downloader/4K YouTube to MP3❗ | reddit.com/r/4kdownloadapps | 2021-10-19
-
browser-fingerprinting
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web?
Project mention: what are the rules to identify web visits as bot/scraper | reddit.com/r/datascience | 2021-11-23 -
Scout APM
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
-
Visit the official Github repository for more information here.
-
freeDictionaryAPI
There was no free Dictionary API on the web when I wanted one for my friend, so I created one.
I am making an app that is the word of the day but is in any* language. I found this AMAZING API that provides several+ different languages with definitions, origins, phonetics, sometimes audio the works.
-
-
amazon_scraper
Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt
Project mention: We have updated our Amazon Headless Chrome + Proxy scraper at Github. Feedback highly appreciated! | reddit.com/r/webscraping | 2021-02-16 -
Project mention: Facebook sues two Chrome extension devs for scraping user data | news.ycombinator.com | 2021-01-14
https://github.com/niczem/trawler
Still a d*ckmove from FB, especially considering that a browser plugin can not really be used to scale up. Anyway, looking forward to hear from FB :)
-
-
nba-topshop-scraper
Node script that will use Selenium to scrape card information from NBA Topshot including card names, rarity, and lowest cost at the moment. Data is scraped once per day.
Project mention: I just made a quick selenium scraper that scrapes the marketplace for the current cards. I made it open-source and will update it with daily datasets (automating this). There's already websites that do this but in case someone wants the data in JSON format, it's here. Use however you want. | reddit.com/r/nbatopshot | 2021-03-14 -
Project mention: Another demo of the "Godot Game Library" app. It's now fully localized, can auto detect the game's platform, search on itch.io to gather info about the game, supported platforms: macOS, Linux and Windows (for now) | reddit.com/r/godot | 2021-04-22
The scraping is done with another tool I created https://github.com/hiulit/itchio-godot-scraper (I talked with the itch people and they were ok with it) and it's only used to get info about the game: title, author, etc.
-
html_tag_annotator
A Machine Learning tool to create the training dataset very quickly & easily by using a smart chrome extension
Project mention: Create the training dataset very easily by using a smart chrome extension | reddit.com/r/datascienceproject | 2021-08-10 -
freenom-auto-renew-domains
A scraper built with puppeteer that auto renew free domains on Freenom and send discord message using webhook
Here is the source code : https://github.com/Sorok-Dva/freenom-auto-renew-domains
-
This is the repo for the API. Check free-exchange-rates-scraper for the actual scraper code.
-
twitter-image-downloader
This is a tool to download images posted/retweeted from a Twitter user's timeline. (by Kimkykie)
Twitter Image Downloader is the tool I built to be able to scrap images from Twitter accounts, of course for educational purposes. I know several such tools exist but I decided to expand my Puppeteer and JS skills by building one myself.
JavaScript Scraper related posts
- ❗Slow Download Speed in 4K Video Downloader/4K YouTube to MP3❗
- Freenom auto renew domains tools
- I made an open source YouTube Video Archiver
- Website Scraper | NodeJS
- How do you convert YouTube video to mp3 ?
- How to un-CMS a website
- I just made a quick selenium scraper that scrapes the marketplace for the current cards. I made it open-source and will update it with daily datasets (automating this). There's already websites that do this but in case someone wants the data in JSON format, it's here. Use however you want.
Index
What are some of the best open-source Scraper projects in JavaScript? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | node-ytdl-core | 2,967 |
| 2 | browser-fingerprinting | 2,590 |
| 3 | node-website-scraper | 1,054 |
| 4 | freeDictionaryAPI | 872 |
| 5 | avbot-charts | 23 |
| 6 | amazon_scraper | 21 |
| 7 | trawler | 10 |
| 8 | nba-topshop-scraper | 8 |
| 9 | itchio-godot-scraper | 7 |
| 10 | html_tag_annotator | 5 |
| 11 | freenom-auto-renew-domains | 5 |
| 12 | free-exchange-rates-scraper | 1 |
| 13 | twitter-image-downloader | 0 |
Are you hiring? Post a new remote job listing for free.
