Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free. Learn more →
Top 9 Go Crawling Projects
-
Project mention: Colly: Elegant Scraper and Crawler Framework for Golang | news.ycombinator.com | 2023-08-23
-
-
Mergify
Tired of breaking your main and manually rebasing outdated pull requests?. Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.
-
I have tried the following. 1. Login to Okta via browser programatically using go-rod. Which I managed to do so successfully, but I'm failing to load up Slack as it's stuck in the browser loader screen for Slack. 2. I tried to authenticate via Okta RESTful API. So far, I have managed to authenticate using {{domain}}/api/v1/authn, and then subsequently using MFA via the verify endpoint {{domain}}/api/v1/authn/factors/{{factorID}}/verify which returns me a sessionToken. From here, I can successfully create a sessionCookie which have proven quite useless to me. Perhaps I am doing it wrongly.
-
hakrawler
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
This week i have used https://github.com/hakluke/hakrawler and I think it meets youre requirements.
-
cariddi
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
cariddi is an open source (https://github.com/edoardottt/cariddi) web security tool. It takes as input a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more.
-
-
webpalm
WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive approach, it can generate a complete tree of all webpages and their links on a website. It can also extract data from the body of each page using regular expressions, making it an ideal tool for web scraping and data extraction.
-
SonarQube
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
-
-
Go Crawling related posts
- 🌐 "WebPalm: Unleash Websites" 🌐
- What is a good bug Bounty methodology?
- webpalm
- webpalm
- New Modern Crawling tool written with go
- New Modern Fast Crawler
- Webpalm - Modern fast web crawling tool in go
-
A note from our sponsor - Mergify
blog.mergify.com | 22 Sep 2023
Index
What are some of the best open-source Crawling projects in Go? This list will help you:
Project | Stars | |
---|---|---|
1 | colly | 20,763 |
2 | Ferret | 5,465 |
3 | rod | 4,182 |
4 | hakrawler | 3,941 |
5 | cariddi | 984 |
6 | Dataflow kit | 616 |
7 | webpalm | 285 |
8 | antch | 252 |
9 | bathyscaphe | 84 |