|over 2 years ago||about 1 month ago|
|GNU General Public License v3.0 or later||MIT License|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Does Golang have stable support for Selenium WebDriver?
4 projects | /r/golang | 1 May 2021
GitHub - sourcegraph/go-selenium: Selenium WebDriver client for Go
Web scraping with Go
5 projects | /r/golang | 2 Apr 2023
Web Crawler in Go: Extracting Keyword-Relevant Text with Text Density
2 projects | /r/golang | 18 Mar 2023
There are a billion things that you need to consider when building a decent web crawler, especially interacting with pages in the modern web. For example, a lot of content is dynamically loaded by the browser nowadays, and won't show up if you make a simple HTTP request. Open your browser devtools and look at the network tab after you make a request, and you'll see it makes loads of auxiliary requests. Some content is also only loaded after you interact with it (e.g. hover, click). For that reason I'd recommend using something like chromedp and do browser based crawling, even if it's much slower.
Web Automation: Don't Use Selenium, Use Playwright
5 projects | news.ycombinator.com | 9 Nov 2022
Thanks. I've found Chromedp  to be a good Selenium replacement when programming in Go. Used in conjunction with HeadlessShell  you can deploy a Go app into a container and do the testing all within the same container with low overhead.
Run single JS func in Go app
4 projects | /r/golang | 22 Oct 2022
Is it node specific? Could you just use https://github.com/chromedp/chromedp and run it as an actually webpage and get the value from that?
Thoughts on Go headless browser tools for testing and scraping?
4 projects | /r/golang | 14 Oct 2022
It looks like https://github.com/chromedp/chromedp is the most popular option. There's also https://github.com/playwright-community/playwright-go. Any thoughts or advice about using these tools?
Is it possible to scrape a website protected by Cloudflare?
6 projects | /r/golang | 10 Sep 2022
Is there a library similar to HTMLUnit in GO?
5 projects | /r/golang | 19 Aug 2022
Easiest way to MITM my browser requests/responses using Go
4 projects | /r/golang | 9 Aug 2022
The alternative is to drive, for example, Chrome with the DevTools protocol (using something like https://github.com/chromedp/chromedp or https://github.com/raff/godet (this is my version of a DevTools driver).
Web scraping libraries
3 projects | /r/golang | 17 Jul 2022
Scrape and automate with golang
2 projects | /r/golang | 11 Jun 2022
Docs are kind of meh but I always have a good time “remote controlling” chrome with this https://github.com/chromedp/chromedp
What are some alternatives?
rod - A Devtools driver for web automation and scraping
cdp - Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language.
playwright-go - Playwright for Go a browser automation library to control Chromium, Firefox and WebKit with a single API.
colly - Elegant Scraper and Crawler Framework for Golang
gomate.io - Behavior-driven development tool for GoLang
realize - Realize is the #1 Golang Task Runner which enhance your workflow by automating the most common tasks and using the best performing Golang live reloading.
httpexpect - End-to-end HTTP and REST API testing for Go.
godet - Remote client for Chrome DevTools
gorip - Facebook scraper and a basic facebook bot, made with Golang.
prettybenchcmp - cmd tool for automatic storage and comparison of benchmarks results
selenoid - Selenium Hub successor running browsers within containers. Scalable, immutable, self hosted Selenium-Grid on any platform with single binary.