katana
goGetJS
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
katana
-
Originally a Covid project. Now a discount search engine.
Using a few different methods. Pulling the sites I'm using Puppeteer and Katana (https://github.com/projectdiscovery/katana). To process and extract the information is tricky, most websites selling things put time into their metadata; this does make it easier. Additionally, a lot of the larger stores have common patterns between them. Failing all of this, I trained a Tensor flow model to understand how to read product pages. However, it's far from perfect and a journey of continual improvement.
-
I’m messed up with Go libraries
Anyway, check out katana https://github.com/projectdiscovery/katana
- Katana
-
Go for web scraping
Checkout https://github.com/projectdiscovery/katana It is right up there with any advanced web scraper you will find in python world or beyond.
- A next-gen crawling and spidering framework
- New crawler released by projectdiscovery - Katana
- Katana: A next-generation crawling and spidering framework
goGetJS
- Go for web scraping
- Share Your Code.. Share your most unique piece of Go code.
-
Tool for extracting, searching, and saving JavaScript files
I’m still tweaking, but it’s stable and available at https://github.com/davemolk/goGetJS. Please let me know if this helps your workflow or if there are features you’d like to see!
-
Golang tool for extracting, searching, and saving JavaScript files
You can find it at https://github.com/davemolk/goGetJS
What are some alternatives?
rod - A Devtools driver for web automation and scraping
goClone - 🌱 goClone - clone websites in a matter of seconds
koin - Koin - a pragmatic lightweight dependency injection framework for Kotlin & Kotlin Multiplatform
Kodein - Painless Kotlin Dependency Injection
KODI - light-weight KOtlin Dependency Injection (KODI)
jellyfin-sdk-kotlin - Kotlin SDK for Jellyfin, supporting Android and JVM targets
butterknife - Bind Android views and callbacks to fields and methods.
Dagger2 - A fast dependency injector for Android and Java.
kapsule - Minimalist dependency injection library for Kotlin.
Toothpick - A scope tree based Dependency Injection (DI) library for Java / Kotlin / Android.
Guice - Guice (pronounced 'juice') is a lightweight dependency injection framework for Java 11 and above, brought to you by Google.
slrp - rotating open proxy multiplexer