gogetcrawl

Extract web archive data using Wayback Machine and Common Crawl (by karust)

Gogetcrawl Alternatives

Similar projects and alternatives to gogetcrawl based on common topics and language

  • ghost

    Discontinued Use ghost for passive recon: get the Wayback Machine history for a URL, search for term(s) or regular expression matches, save all archived links, save an archived robots.txt and sitemap.xml, run a whois lookup, and get IP addresses, all without touching the target. (by davemolk)

  • xurlfind3r

    A command-line interface (CLI) based passive URLs discovery utility. It is designed to efficiently identify known URLs of given domains by tapping into a multitude of curated online passive sources.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • colly

    Elegant Scraper and Crawler Framework for Golang

  • Ferret

    Declarative web scraping

  • lux

    👾 Fast and simple video download library and CLI tool written in Go

  • Rendora

    dynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better gogetcrawl alternative or higher similarity.

gogetcrawl reviews and mentions

Posts with mentions or reviews of gogetcrawl. We have used some of these posts to build our list of alternatives and similar projects.
  • A tool/package for Web Archive data extraction
    1 project | /r/golang | 31 May 2023
    I've developed yet another solution that can help you extract data from web archives :) You can use it as a separate tool, or import it into your Go project. Github: https://github.com/karust/gogetcrawl

Stats

Basic gogetcrawl repo stats
1
126
5.2
11 months ago

karust/gogetcrawl is an open source project licensed under MIT License which is an OSI approved license.

The primary programming language of gogetcrawl is Go.


Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com