Go Spider

Open-source Go projects categorized as Spider

Top 12 Go Spider Projects

  1. colly

    Elegant Scraper and Crawler Framework for Golang

    Project mention: Golang with Colly: Use Random Fake User-Agents When Scraping | dev.to | 2025-01-10

    https://github.com/lib4u/fake-useragent https://github.com/gocolly/colly

  2. InfluxDB

    InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

    InfluxDB logo
  3. crawlab

    Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架

  4. Pholcus

    Pholcus is a distributed high-concurrency crawler software written in pure golang

  5. DHT

    BitTorrent DHT Protocol && DHT Spider.

  6. Geziyor

    Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

  7. webpalm

    🕸️ Crawl in the web network

  8. ant

    A web crawler for Go (by yields)

  9. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  10. spidy

    Domain names collector - Crawl websites and collect domain names along with their availability status. (by twiny)

  11. scrapemate

    Golang Crawling and scraping framework (by gosom)

  12. goscrapy

    GoScrapy: Harnessing Go's power for blazingly fast web scraping, inspired by Python's Scrapy framework.

    Project mention: GoScrapy: Harnessing Go's power for fast web scraping | news.ycombinator.com | 2024-09-06
  13. wbot

    A simple & efficient web crawler.

  14. turbo-tor-crawl

    Recursive hostnames crawler

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Go Spider discussion

Log in or Post with

Go Spider related posts

Index

What are some of the best open-source Spider projects in Go? This list will help you:

# Project Stars
1 colly 24,156
2 crawlab 11,740
3 Pholcus 7,588
4 DHT 2,747
5 Geziyor 2,710
6 webpalm 370
7 ant 278
8 spidy 157
9 scrapemate 118
10 goscrapy 95
11 wbot 16
12 turbo-tor-crawl 6

Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com

Did you know that Go is
the 4th most popular programming language
based on number of references?