SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Go Scraper Projects
👾 Fast and simple video download library and CLI tool written in GoProject mention: Bilibili download stalls at around 30-60% | /r/youtubedl | 2023-05-18
Not a fix, but I tend to use lux when downloading from bilibili. It is faster too.
Elegant Scraper and Crawler Framework for GolangProject mention: New modern web crawling tool | news.ycombinator.com | 2023-04-30
Sounds cool, but how is this different from Colly: https://github.com/gocolly/colly?
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
Declarative web scraping
A Devtools driver for web automation and scrapingProject mention: Library to convert HTML to pdf in Golang | /r/golang | 2023-05-22
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.
📖 The most advanced (yet simple) cli manga downloader in the entire universe! Lua scrapers, export formats, anilist integration, fancy TUI and more!Project mention: What application handles manga downloads? | /r/selfhosted | 2023-05-19
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
:bar_chart: Financial markets data library implemented in go.Project mention: finance-go: NEW Data - star count:602.0 | /r/algoprojects | 2023-05-13
Extract structured data from web sites. Web sites scraping.
A web crawler for Go (by yields)
GMDB is the ultra-simple, cross-platform Movie Library with Features (Search, Take Note, Watch Later, Like, Import, Learn, Instantly Torrent Magnet Watch)
DorkScout - Golang tool to automate google dork scan against the entiere internet or specific targetsProject mention: Automatizovani Google Dorking | /r/programiranje | 2023-04-14
Demeter is a tool for scraping the calibre web ui
Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog. (by odpf)Project mention: Modern open-source data platform that empowers organizations to discover, transform, analyse and secure data faster and efficiently. | /r/dataengineering | 2022-06-21
Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog.
Scraply a simple dom scraper to fetch information from any html based websiteProject mention: scraply v3.0.0 - a very simple scraping too using CSS selectors and jQuery like functions | /r/golang | 2022-07-06
Domain names collector - Crawl websites and collect domain names along with their availability status. (by twiny)Project mention: Share Your Code.. Share your most unique piece of Go code. | /r/golang | 2022-10-15
1 - Expired domain scrapper => https://github.com/twiny/spidy 2 - A sample & efficient web crawler => https://github.com/twiny/wbot 3 - A mini blockchain scanner => https://github.com/twiny/blockscan 4 - A Snake Game => https://github.com/twiny/snaky
Configurable Scraper & Downloader, Powered by RegExp and Go (by everdrone)Project mention: Looking for approachable OSS project or mentor | /r/golang | 2022-09-15
Hey, I am also relatively new to Go The project I’m maintaining is Grab
Bulk image downloader for reddit.Project mention: Command line tool to bulk download images from reddit | /r/commandline | 2023-02-08
xDSL Prometheus ExporterProject mention: I created Prometheus Exporter with Go to scrape my xDSL Modem stats | /r/golang | 2023-03-17
New way for collect information from the API's/Websites (by PxyUp)Project mention: Show HN: Library for scrape internet like GQL | news.ycombinator.com | 2023-03-24
Go package for scraping website recipes
A configurable command-line web scraper written in go with auto configuration capabilityProject mention: No code command line webscraper | /r/webscraping | 2023-03-09
I am currently building a webscraper, called goskyr, that can be run from the command line and is supposed to be easily configurable. So instead of having to write code to scrape a website you'd just write a configuration snippet and run the scraper. I realize that there are a number of gui based scraping services that make it extremely easy to setup a scraping process for any website, so for people having no coding experience whatsoever that would probably be the easiest solution. I'm trying to come close to those gui based solutions in terms of functionality by providing a 'smart' way of finding potentially interesting data/fields and letting the user select a subset in a terminal based ui. Also date extraction & parsing and the newly added machine learning capability is probably worth mentioning. Still, those other, gui based solutions are really awesome, eg octoparse or scrapestorm.
A small CLI app to scrap high-quality movie snapshots from various websites.
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Go Scraper related posts
Bilibili download stalls at around 30-60%
1 project | /r/youtubedl | 18 May 2023
colly VS scrapemate - a user suggested alternative
2 projects | 15 Apr 2023
Web Scraping in Python: Avoid Detection Like a Ninja
2 projects | dev.to | 5 Apr 2023
Web scraping with Go
5 projects | /r/golang | 2 Apr 2023
Show HN: Library for scrape internet like GQL
1 project | news.ycombinator.com | 24 Mar 2023
No code command line webscraper
3 projects | /r/webscraping | 9 Mar 2023
Fitter - GQL for web scrawling, multiple sources, defining fields, emulating browser and more other
1 project | /r/programming | 7 Mar 2023
A note from our sponsor - #<SponsorshipServiceOld:0x00007f0922f73d48>
www.saashub.com | 5 Jun 2023
What are some of the best open-source Scraper projects in Go? This list will help you: