Top 15 Go Scraper Projects

lux

5 17,804 8.4 Go

👾 Fast and simple video download library and CLI tool written in Go

Project mention: lux VS youtube - a user suggested alternative | libhunt.com/r/lux | 2022-02-02
colly

13 16,540 5.3 Go

Elegant Scraper and Crawler Framework for Golang

Project mention: First project - The API for engineering multiple choice questions | reddit.com/r/golang | 2022-05-08

There is the scraper struct that spawns goroutine. Then, each goroutine scraps the web URL using go colly, and the structured MCQ questions are served from API endpoints. Users can choose between persistent Postgres model or in-memory data structure to store the questions and retrieve them from API. I have used dependency injection to inject the models via the interface to my application struct.
SonarQube

www.sonarqube.org
sponsored

Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
Ferret

0 4,973 7.4 Go

Declarative web scraping
rod

7 2,376 9.1 Go

A Devtools driver for web automation and scraping

Project mention: Rod - A Devtools driver for web automation and scraping | reddit.com/r/github_trends | 2022-04-29
Geziyor

0 1,704 4.6 Go

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
finance-go

15 542 0.0 Go

:bar_chart: Financial markets data library implemented in go.

Project mention: finance-go: NEW Data - star count:528.0 | reddit.com/r/algoprojects | 2022-04-09
Dataflow kit

0 514 0.0 Go

Extract structured data from web sites. Web sites scraping.
Scout APM

scoutapm.com
sponsored

Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
ant

0 261 3.2 Go

A web crawler for Go (by yields)
GMDB

0 214 0.0 Go

GMDB is the ultra-simple, cross-platform Movie Library with Features (Search, Take Note, Watch Later, Like, Import, Learn, Instantly Torrent Magnet Watch)
dorkscout

13 170 5.3 Go

DorkScout - Golang tool to automate google dork scan against the entiere internet or specific targets

Project mention: automatic google dorking scan tool | reddit.com/r/AccessCyber | 2021-08-17
demeter

2 160 1.1 Go

Demeter is a tool for scraping the calibre web ui

Project mention: GitHub - gnur/booksing: web-based epub indexer | reddit.com/r/selfhosted | 2021-09-21

Maybe https://github.com/gnur/demeter, from the same author?
torrodle

1 134 0.0 Go

⚡️ A magnet link scraper for streaming videos (movies, tv shows, anime, porn) along with subtitles.

Project mention: Search and Stream torrent in your terminal | reddit.com/r/commandline | 2022-04-14
llvm-c-search

6 11 5.3 Go

Terminal interface to search the LLVM-C API

Project mention: Terminal interface to search the LLVM-C API | reddit.com/r/golang | 2021-12-26
moviestills

1 7 9.0 Go

A small CLI app to scrap high-quality movie snapshots from various websites.

Project mention: Get high-quality movie snapshots from various websites | reddit.com/r/coolgithubprojects | 2021-10-14
rrip

2 2 10.0 Go

Bulk image downloader for reddit.

Project mention: rrip - program to bulk-download images from reddit | reddit.com/r/golang | 2022-05-15

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-05-15.

Go Scraper related posts

rrip - program to bulk-download images from reddit
1 project | reddit.com/r/golang | 15 May 2022
First project - The API for engineering multiple choice questions
2 projects | reddit.com/r/golang | 8 May 2022
How to collaborate on web scraping?
2 projects | reddit.com/r/webscraping | 1 May 2022
How I Scraped Michelin Guide Using Go
2 projects | dev.to | 21 Mar 2022
lux VS youtube - a user suggested alternative
2 projects | 2 Feb 2022
Terminal interface to search the LLVM-C API
1 project | reddit.com/r/golang | 26 Dec 2021
Terminal interface to search the LLVM-C API
1 project | reddit.com/r/Compilers | 26 Dec 2021

Index

What are some of the best open-source Scraper projects in Go? This list will help you:

	Project	Stars
1	lux	17,804
2	colly	16,540
3	Ferret	4,973
4	rod	2,376
5	Geziyor	1,704
6	finance-go	542
7	Dataflow kit	514
8	ant	261
9	GMDB	214
10	dorkscout	170
11	demeter	160
12	torrodle	134
13	llvm-c-search	11
14	moviestills	7
15	rrip	2

Find remote jobs at our new job board 99remotejobs.com. There are 12 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.