Top 13 Go Scraping Projects

• colly

  39 22,165 6.0 Go

  Elegant Scraper and Crawler Framework for Golang

  

Project mention: Scraping the full snippet from Google search result | dev.to | 2024-01-01

SerpApi focuses on scraping search results. That's why we need extra help to scrape individual sites. We'll use GoColly package.

• Ferret

  0 5,616 3.1 Go

  Declarative web scraping

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• Geziyor

  2 2,475 0.6 Go

  Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

  

Project mention: Show HN: I scraped 25M Shopify products to build a search engine | news.ycombinator.com | 2023-12-13

As someone who has scraped millions of items myself, I had success using Geziyor (https://github.com/geziyor/geziyor) built in Go. Shopify sites are especially easy to scrape because they tend to share the same product data formatting and don't hide it behind JS rendering.

• till

  5 807 1.8 Go

  DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.

  

• Dataflow kit

  0 636 0.0 Go

  Extract structured data from web sites. Web sites scraping.

• antch

  0 255 0.0 Go

  Antch, a fast, powerful and extensible web crawling & scraping framework for Go

  

• JsonGenius

  12 153 6.1 Go

  Get structured JSON data from any page.

Project mention: Show HN: SingleAPI – Convert the Internet into your own API | news.ycombinator.com | 2023-10-17

isn’t this just using jsongenius[1]

[1] https://github.com/semanser/JsonGenius

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• newser

  2 83 3.2 Go

  Newser is a simple utility to generate a pdf with you favorite news articles

• goClone

  1 54 6.2 Go

  🌱 goClone - clone websites in a matter of seconds

Project mention: Show HN: Goclone – your ultimate tool for offline web browsing | news.ycombinator.com | 2023-07-31

• xdsl-exporter

  3 45 0.9 Go

  xDSL Prometheus Exporter

• goGetJS

  4 35 0.0 Go

  a tool for extracting, searching, and saving JavaScript files (with optional headless browser)

• moviestills

  1 14 5.7 Go

  A small CLI app to scrap high-quality movie snapshots from various websites.

• go-scraper

  1 0 1.8 Go

  This repo show how to Scrape different type of data

Go Scraping related posts

• Show HN: Flyscrape – A standalone and scriptable web scraper in Go
  6 projects | news.ycombinator.com | 11 Nov 2023
• Colly: Elegant Scraper and Crawler Framework for Golang
  1 project | news.ycombinator.com | 23 Aug 2023
• Do I really need to grind leetcode to get a software job? It's been the bane of my existence
  1 project | /r/learnprogramming | 22 Jun 2023
• Hello, I am new to Python. I know the basics but I have no idea what should I do next.
  1 project | /r/learnpython | 22 Jun 2023
• colly VS scrapemate - a user suggested alternative
  2 projects | 15 Apr 2023
• Web scraper help
  1 project | /r/golang | 1 Mar 2023
• Web Scraping in Golang
  2 projects | dev.to | 7 Feb 2023
• A note from our sponsor - WorkOS
  workos.com | 25 Apr 2024

  The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Go projects with our weekly report!