hakrawler
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application (by hakluke)
serviceq
Super fault-tolerant HTTP load balancer & queue. White paper for reference - https://github.com/gptankit/serviceq-paper (by gptankit)
Our great sponsors
| hakrawler | serviceq | |
|---|---|---|
| 3 | 3 | |
| 4,230 | 74 | |
| - | - | |
| 0.0 | 2.6 | |
| 3 months ago | 4 months ago | |
| Go | Go | |
| GNU General Public License v3.0 only | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
hakrawler
Posts with mentions or reviews of hakrawler.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-07-25.
-
Find all src img on a public website?
This week i have used https://github.com/hakluke/hakrawler and I think it meets youre requirements.
- 2 open source tools to find subdomains (ft. HakLuke)
-
Looking for an open-source Golang project to work on
I coded a popular CLI web crawler a while back when I first learned go. The code base is not great and it's buggy! I've been meaning to basically refactor the whole thing. https://github.com/hakluke/hakrawler
serviceq
Posts with mentions or reviews of serviceq.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-03-05.
-
Open source Go projects to contribute (beginners)
https://github.com/gptankit/serviceq - a load balancer and queue (need http2 support) https://github.com/gptankit/go-wasm - wasm experiments in go (if you want to help community adapt to wasm using go)
-
Looking for an open-source Golang project to work on
If interested in load balancers/proxies and networking in general, you may want to look at https://github.com/gptankit/serviceq.
-
I'm trying to create a load balancer that routes based on CPU availability on server. Do you guys know the best algorithm for such a case?
When you have so many metrics to consider, ideal choice is to assign weights to each metric and let the load balancer distribute load probabilistically (think weighted random). Implemented a load balancer in Go on similar lines (https://github.com/gptankit/serviceq) by calculating 'effective error' as weight from each node and inversely distributing load to the cluster. This ensures that a) one single node doesn't get overwhelmed with all the requests and b) even if some node is down, it keeps receiving a small percentage of requests (albeit lesser and lesser as time progresses).
What are some alternatives?
When comparing hakrawler and serviceq you can also consider the following projects:
Gokapi - Lightweight selfhosted Firefox Send alternative without public upload. AWS S3 supported.
enpass-cli - Enpass commandline client
gau - Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.
ent - An entity framework for Go
dsq - Commandline tool for running SQL queries against JSON, CSV, Excel, Parquet, and more.
fixme - Scan for NOTE, OPTIMIZE, TODO, HACK, XXX, FIXME, and BUG comments within your source, and print them to stdout so you can deal with them.
reddit-rss - improved rss feed for reddit
dbmigrat - DB schema migration lib designed for monorepos.
Mantis - Sphire Mantis is a broadly featured Go helper library with standalone packages