Top 4 TypeScript Crawler Projects
web scraping extension (by get-set-fetch)Project mention: Scraping Sub-page on a password protected site | reddit.com/r/webscraping | 2021-02-03
Once such open source extension is https://github.com/get-set-fetch/extension . Happy scraping :)
A lightweight crawler frontier implementation in TypeScript using Redis.Project mention: MicroFrontier – an open source URL frontier for Web Crawlers built with Redis | news.ycombinator.com | 2021-11-02
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
Reconnaissance Platform (by salugi)Project mention: Owl - Rust Port Analyzer and Network Mapper | reddit.com/r/rust | 2021-10-20
I'm currently building out a recon platform (https://github.com/yugely/Pelican) and I have gotten about as far as I can with whats available currently in Deno to continue building. That, and I realized that Rust doesn't have a legit "nmap" equivalent, because I wanted to implement one with the tooling I'm currently building. So I decided to start (https://github.com/yugely/Owl only initial commit, nothing really there now).
Filesystem crawlerProject mention: Introducing filespy | dev.to | 2021-03-16
Filespy mixes globbing with regular expressions, a concept borrowed from Recrawl.
TypeScript Crawler related posts
What are some of the best open-source Crawler projects in TypeScript? This list will help you:
Are you hiring? Post a new remote job listing for free.