Go for web scraping

• colly

  4 22,072 6.0 Go

  Elegant Scraper and Crawler Framework for Golang

  

  http://go-colly.org/ is excellent

• rod

  2 4,727 7.9 Go

  A Devtools driver for web automation and scraping

  

  I recently tried out https://github.com/go-rod/rod. I think it's based on chromedp (so, Chrome dev tools and headless browser) but it also has code to download and run a supported version of Chrome that doesn't interfere with your local browser.

• WorkOS

  workos.com

  sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• go-wiki

  0 83 4.7 Go

  This is a Golang open-source module that makes it easy to access and parse data from Wikipedia (Wikipedia API wrapper)

  If you ever need to scrap the Wikipedia, use this: https://github.com/trietmn/go-wiki

• goGetJS

  0 35 0.0 Go

  a tool for extracting, searching, and saving JavaScript files (with optional headless browser)

• katana

  0 8,618 9.1 Go

  A next-generation crawling and spidering framework.

  

  Checkout https://github.com/projectdiscovery/katana It is right up there with any advanced web scraper you will find in python world or beyond.

• InfluxDB

  www.influxdata.com

  sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project