Go Spider

Open-source Go projects categorized as Spider | Edit details

Top 6 Go Spider Projects

  • colly

    Elegant Scraper and Crawler Framework for Golang

    Project mention: First project - The API for engineering multiple choice questions | reddit.com/r/golang | 2022-05-08

    There is the scraper struct that spawns goroutine. Then, each goroutine scraps the web URL using go colly, and the structured MCQ questions are served from API endpoints. Users can choose between persistent Postgres model or in-memory data structure to store the questions and retrieve them from API. I have used dependency injection to inject the models via the interface to my application struct.

  • Pholcus

    Pholcus is a distributed high-concurrency crawler software written in pure golang

  • SonarLint

    Deliver Cleaner and Safer Code - Right in Your IDE of Choice!. SonarLint is a free and open source IDE extension that identifies and catches bugs and vulnerabilities as you code, directly in the IDE. Install from your favorite IDE marketplace today.

  • DHT

    BitTorrent DHT Protocol && DHT Spider.

    Project mention: DHT VS dht - a user suggested alternative | libhunt.com/r/dht | 2022-01-13
  • Geziyor

    Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

  • ant

    A web crawler for Go (by yields)

  • gospider

    ⚡ Light weight Golang spider framework | 轻量的 Golang 爬虫框架

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-05-08.

Go Spider related posts


What are some of the best open-source Spider projects in Go? This list will help you:

Project Stars
1 colly 16,585
2 Pholcus 7,114
3 DHT 2,500
4 Geziyor 1,704
5 ant 261
6 gospider 144
Find remote jobs at our new job board 99remotejobs.com. There are 7 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives