chromedp
rod
Our great sponsors
chromedp | rod | |
---|---|---|
27 | 20 | |
10,341 | 4,750 | |
2.0% | 3.3% | |
5.5 | 7.9 | |
8 days ago | 3 days ago | |
Go | Go | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
chromedp
- Web scraping with Go
-
Web Crawler in Go: Extracting Keyword-Relevant Text with Text Density
There are a billion things that you need to consider when building a decent web crawler, especially interacting with pages in the modern web. For example, a lot of content is dynamically loaded by the browser nowadays, and won't show up if you make a simple HTTP request. Open your browser devtools and look at the network tab after you make a request, and you'll see it makes loads of auxiliary requests. Some content is also only loaded after you interact with it (e.g. hover, click). For that reason I'd recommend using something like chromedp and do browser based crawling, even if it's much slower.
-
How can I do infinte scroll with go http
You could use something like the chrome dev tools (https://github.com/chromedp/chromedp) or look into JS runtimes in Go and try to handle the I/O yourself.
-
Web Automation: Don't Use Selenium, Use Playwright
Thanks. I've found Chromedp [0] to be a good Selenium replacement when programming in Go. Used in conjunction with HeadlessShell [1] you can deploy a Go app into a container and do the testing all within the same container with low overhead.
-
Run single JS func in Go app
Is it node specific? Could you just use https://github.com/chromedp/chromedp and run it as an actually webpage and get the value from that?
-
Thoughts on Go headless browser tools for testing and scraping?
It looks like https://github.com/chromedp/chromedp is the most popular option. There's also https://github.com/playwright-community/playwright-go. Any thoughts or advice about using these tools?
- Is it possible to scrape a website protected by Cloudflare?
- Is there a library similar to HTMLUnit in GO?
-
Easiest way to MITM my browser requests/responses using Go
The alternative is to drive, for example, Chrome with the DevTools protocol (using something like https://github.com/chromedp/chromedp or https://github.com/raff/godet (this is my version of a DevTools driver).
- Web scraping libraries
rod
-
Need help authenticating to Okta programatically.
I have tried the following. 1. Login to Okta via browser programatically using go-rod. Which I managed to do so successfully, but I'm failing to load up Slack as it's stuck in the browser loader screen for Slack. 2. I tried to authenticate via Okta RESTful API. So far, I have managed to authenticate using {{domain}}/api/v1/authn, and then subsequently using MFA via the verify endpoint {{domain}}/api/v1/authn/factors/{{factorID}}/verify which returns me a sessionToken. From here, I can successfully create a sessionCookie which have proven quite useless to me. Perhaps I am doing it wrongly.
- Library to convert HTML to pdf in Golang
- Web scraping with Go
- Best option for browser automation
-
I’m messed up with Go libraries
I usually find libraries by googling them or searching awesome go on GitHub, for selenium/puppeteer I've always found go-rod useful and easy in every way, Sometimes I also Google "X in Nodejs for Golang"
-
Go for web scraping
I recently tried out https://github.com/go-rod/rod. I think it's based on chromedp (so, Chrome dev tools and headless browser) but it also has code to download and run a supported version of Chrome that doesn't interfere with your local browser.
- Reducir tiempo de Web Scraping con concurrencia - GO
-
VHS: CLI Home Video Recorder
One of the dependencies is `rod`[0], which is a web scraping/automation library, and I believe requires a browser to work. I don't know what they're using it for though as I haven't looked at the code (and I'm not familiar with Go anyways).
-
Thoughts on Go headless browser tools for testing and scraping?
I don't have personal experience, but https://github.com/go-rod/rod is far more active than chromedp
- Project with a Web scraper GO binary
What are some alternatives?
go-selenium - Selenium WebDriver client for Go
playwright-go - Playwright for Go a browser automation library to control Chromium, Firefox and WebKit with a single API.
cdp - Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language.
colly - Elegant Scraper and Crawler Framework for Golang
WebDumper - A tool for scraping, dumping and unpacking (webpacked) javascript source files.
realize - Realize is the #1 Golang Task Runner which enhance your workflow by automating the most common tasks and using the best performing Golang live reloading.
gomate.io - Behavior-driven development tool for GoLang
gotests - Automatically generate Go test boilerplate from your source code.
httpexpect - End-to-end HTTP and REST API testing for Go.