Go Crawling

Open-source Go projects categorized as Crawling | Edit details

Top 6 Go Crawling Projects

  • colly

    Elegant Scraper and Crawler Framework for Golang

    Project mention: First project - The API for engineering multiple choice questions | reddit.com/r/golang | 2022-05-08

    There is the scraper struct that spawns goroutine. Then, each goroutine scraps the web URL using go colly, and the structured MCQ questions are served from API endpoints. Users can choose between persistent Postgres model or in-memory data structure to store the questions and retrieve them from API. I have used dependency injection to inject the models via the interface to my application struct.

  • Ferret

    Declarative web scraping

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • rod

    A Devtools driver for web automation and scraping

    Project mention: Rod - A Devtools driver for web automation and scraping | reddit.com/r/github_trends | 2022-04-29
  • Dataflow kit

    Extract structured data from web sites. Web sites scraping.

  • antch

    Antch, a fast, powerful and extensible web crawling & scraping framework for Go

  • bathyscaphe

    Fast, highly configurable, cloud native dark web crawler.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-05-08.

Go Crawling related posts


What are some of the best open-source Crawling projects in Go? This list will help you:

Project Stars
1 colly 16,585
2 Ferret 4,981
3 rod 2,394
4 Dataflow kit 514
5 antch 226
6 bathyscaphe 73
Find remote jobs at our new job board 99remotejobs.com. There are 7 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives