TypeScript Crawler

Open-source TypeScript projects categorized as Crawler | Edit details

Top 4 TypeScript Crawler Projects

  • GitHub repo extension

    web scraping extension (by get-set-fetch)

    Project mention: Scraping Sub-page on a password protected site | reddit.com/r/webscraping | 2021-02-03

    Once such open source extension is https://github.com/get-set-fetch/extension . Happy scraping :)

  • GitHub repo MicroFrontier

    A lightweight crawler frontier implementation in TypeScript using Redis.

    Project mention: MicroFrontier – an open source URL frontier for Web Crawlers built with Redis | news.ycombinator.com | 2021-11-02
  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • GitHub repo Pelican

    Reconnaissance Platform (by salugi)

    Project mention: Owl - Rust Port Analyzer and Network Mapper | reddit.com/r/rust | 2021-10-20

    I'm currently building out a recon platform (https://github.com/yugely/Pelican) and I have gotten about as far as I can with whats available currently in Deno to continue building. That, and I realized that Rust doesn't have a legit "nmap" equivalent, because I wanted to implement one with the tooling I'm currently building. So I decided to start (https://github.com/yugely/Owl only initial commit, nothing really there now).

  • GitHub repo recrawl

    Filesystem crawler

    Project mention: Introducing filespy | dev.to | 2021-03-16

    Filespy mixes globbing with regular expressions, a concept borrowed from Recrawl.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-11-02.

TypeScript Crawler related posts


What are some of the best open-source Crawler projects in TypeScript? This list will help you:

Project Stars
1 extension 20
2 MicroFrontier 5
3 Pelican 1
4 recrawl 0
Find remote jobs at our new job board 99remotejobs.com. There are 29 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
OPS - Build and Run Open Source Unikernels
Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.