SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Go Distributed System Projects
-
Project mention: Transitioning from more traditional OOP like C# to Go, what are the biggest coding style differences. | /r/golang | 2023-11-16
Reading the standard library will give you ideas/insight about various Go idiomatic patterns/approaches, and you can see a full website/API implementation in the pkg.go.dev repository (https://github.com/golang/pkgsite). Projects like https://github.com/etcd-io/etcd may be interesting too.
-
Interesting. What are you thoughts on NSQ?
Was looking at it earlier today, but haven't ever tried it out.
-
InfluxDB
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
-
Project mention: Ask HN: What are some examples of cloud lock-in? | news.ycombinator.com | 2022-12-05
Had similar goals. Started by writing Go interfaces for it with Go Micro - https://go-micro.dev then opted for the platform service model as you mentioned with Micro - https://micro.dev
I think whether it's Dapr, Micro or something else, the platform service model with well defined interfaces is the way to go. I don't think a lot of people get this yet so it's still going to be a few years before it takes off.
-
First, I started with gRPC’s recommended starter repository for learning gRPC, their **helloworld **example, which is a part of the official gRPC repository.
-
seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
Project mention: Show HN: OpenSign – The open source alternative to DocuSign | news.ycombinator.com | 2023-10-28> Theoretically they could swap with minio but last time we used it it was not a drop-in replacement yet.
Depends on whether AGPL v3 works for you or not (or whether you decide to pay them), I guess: https://min.io/pricing
I've actually been looking for more open alternatives, but haven't found much.
Zenko CloudServer seemed to be somewhat promising, but doesn't seem to be managed very actively: https://github.com/scality/cloudserver/issues/4986 (their Docker images on DockerHub were last updated 10 months ago, which is what the homepage links to; blog doesn't seem active since 2019, forums don't have much going on, despite some action on GitHub still)
There was also Garage, but that one is also AGPL v3: https://garagehq.deuxfleurs.fr/
The closest I got was discovering that SeaweedFS has an S3 compatible mode: https://github.com/seaweedfs/seaweedfs
-
I found simple distributed RQlite https://github.com/rqlite/rqlite based on raft and sqlite. How hard is to add it?
-
Nomad
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
Project mention: Google Kubernetes Engine incident spanning 9 days | news.ycombinator.com | 2023-10-10 -
Onboard AI
Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev.
-
require nats.io
-
Project mention: Show HN: Mu – A community first app platform proposal | news.ycombinator.com | 2023-06-13
Hi HN
I'm Asim, an engineer and author of the open source project Micro (https://micro.dev). I've been on this forum in various forms since 2009. In that time I've seen a lot of shifts in the technology landscape, participated in a few and even tried to start a company myself (which ended up VC funded). In typical HN fashion, I drank the kool-aid and applied to YC around 8-10 times. I got so far as going to mountain view for an interview but never got in. I played right into the idea of starting a fast growing company and "changing the world" by solving my own problems in the developer ecosystem but somewhere along the way lost sight of the problem I was solving. I think it speaks to my own need to seek validation (in the wrong place) but also the general nature of the ecosystem that became all about the pursuit of more funding, growth, users, etc. It really feels like while there's 70-80% value created, the other 20-30% is doing us a lot of harm. Consumer software has become addictive, corporations are becoming huge profit seeking blackholes that mostly cater to their shareholders and we have yet to see any great alternative to the existing tools and services we're now beholden too.
A couple weeks ago I posted Micro Chat (https://micro.mu) on HN - https://news.ycombinator.com/item?id=36135683. This was my first effort in trying to solve the problems mentioned above. But I don't think it's enough. I think what we're talking about is something much bigger. And it's not going to be solved by open source, federation or a single tool. I'm starting to think we need to build something bigger, an entirely new community led app platform.
So here's my idea. Mu (pronounced mew) is a proposal to build a new community app platform from the ground up. It would address all the problems we seem to mention here about ownership of services, addictive scrolling, profit seeking agendas and do it by empowering a community to crowdfund the creation of an alternative system that focuses on being a utility for the people. It would generally be open source, potentially self hostable, but the larger goal would be to empower a handful of people to host and run the service for everyone else, a built like how Let's Encrypt is run today.
I've thrown this idea around a lot, started many discussions here and even tried to push a few things out but never really gotten anywhere with it. I think part of it is to just, build the smallest piece and see where it goes, hence Micro Chat, but the other part is to gauge the sentiment in the room and see whether we can actually fund something much bigger totally aligned with the community. I'm a huge fan of Google services and the utility they offer, but the nature of technology has veered so far into addiction with scrolling feeds, streaming and social media that I feel there's a need to do something now.
If there's interest, please comment and provide feedback.
Cheers
-
If you have any questions or would like to learn more, feel free to join discussions about JuiceFS on GitHub and the JuiceFS community on Slack.
-
Project mention: temporal VS laravel-workflow - a user suggested alternative | libhunt.com/r/temporal | 2023-08-23
-
Project mention: Gothub: Alternative front-end for GitHub written with Go | news.ycombinator.com | 2023-09-01
Neither do the issues support. But there is git-bug [0].
-
rpcx
Best microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! 𝐉𝐚𝐯𝐚有𝐝𝐮𝐛𝐛𝐨, 𝐆𝐨𝐥𝐚𝐧𝐠有𝐫𝐩𝐜𝐱! build for cloud!
Project mention: Ask HN: Cool side project you have written using Golang | news.ycombinator.com | 2023-02-02I have the microservices framework https://rpcx.io, which is used by many companies.
-
Depending on your exact needs, you could try HashiCorp's Raft implementation: https://github.com/hashicorp/raft
-
cadence
Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
Uber | Software Engineers | Hybrid (Denmark) | https://www.uber.com/dk/en/careers/locations/aarhus/
Work with an amazing team responsible for the infrastructure software that makes Uber’s data centers around the world reliable and scalable. If you want to solve the toughest engineering challenges alongside some of the smartest people in the industry, Uber Aarhus is the right place for you.
The team in Aarhus build and operate the stateless and stateful compute platforms used by nearly every other engineer in the company (Up - https://www.uber.com/en-GB/blog/up-portable-microservices-re... and Odin - https://www.uber.com/en-GB/blog/how-uber-optimized-cassandra...) as well as other related infrastructure projects such as Cadence - https://github.com/uber/cadence.
-
-
There are a couple of other contenders in this space. DVC (https://dvc.org/) seems most similar.
If you're interested in something you can self-host... I work on Pachyderm (https://github.com/pachyderm/pachyderm), which doesn't have a Git-like interface, but also implements data versioning. Our approach de-duplicates between files (even very small files), and our storage algorithm doesn't create objects proportional to O(n) directory nesting depth as Xet appears to. (Xet is very much like Git in that respect.)
The data versioning system enables us to run pipelines based on changes to your data; the pipelines declare what files they read, and that allows us to schedule processing jobs that only reprocess new or changed data, while still giving you a full view of what "would" have happened if all the data had been reprocessed. This, to me, is the key advantage of data versioning; you can save hundreds of thousands of dollars on compute. Being able to undo an oopsie is just icing on the cake.
Xet's system for mounting a remote repo as a filesystem is a good idea. We do that too :)
-
-
Project mention: Thoughts on new tool - easegress - 5k stars in 8 months | /r/kubernetes | 2023-03-03
Since there is no discussion on reddit, wanted to get people's thoughts on a relatively new project easegress which looks to have gained a lot of stars. Seems interesting and something that can't easily be compared to any other individual tool since it combines a lot of features in one.
-
There are good use-cases, but to much focus on cryptocurrencies. Frameworks are not well designed to be used in other use-cases. Best that I have found that doesn't forces you to a specific architecture is Tendermint, but is not even close to state-of-the-art performance and scalability. Everything else is designed around cryptocurrency and smart contracts.
-
-
https://github.com/anacrolix/torrent/blob/master/cmd/torrent... does exactly that. Install with `go get github.com/anacrolix/torrent/cmd/torrent@latest`, and then run `torrent serve `.
-
Jocko
Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-12-01.
Go Distributed Systems related posts
- Adding new database engine support
- Transitioning from more traditional OOP like C# to Go, what are the biggest coding style differences.
- Build Your Own BitTorrent
- Spacedrive: Unify files from all your devices and clouds into one easy explorer
- Durable Coroutines for Go
- Fault Tolerance in Distributed Systems: Strategies and Case Studies
- Coroutines for Golang
-
A note from our sponsor - #<SponsorshipServiceOld:0x00007f0fa33642f0>
www.saashub.com | 3 Dec 2023
Index
What are some of the best open-source Distributed System projects in Go? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | etcd | 45,000 |
| 2 | nsq | 23,949 |
| 3 | go-micro | 21,001 |
| 4 | grpc-go | 19,169 |
| 5 | seaweedfs | 18,752 |
| 6 | rqlite | 14,194 |
| 7 | Nomad | 14,117 |
| 8 | NATS | 13,900 |
| 9 | micro | 11,917 |
| 10 | juicefs | 8,898 |
| 11 | temporal | 8,574 |
| 12 | git-bug | 7,906 |
| 13 | rpcx | 7,790 |
| 14 | raft | 7,520 |
| 15 | cadence | 7,455 |
| 16 | kitex | 6,382 |
| 17 | pachyderm | 6,004 |
| 18 | KrakenD | 5,818 |
| 19 | easegress | 5,575 |
| 20 | tendermint | 5,566 |
| 21 | gaia | 5,124 |
| 22 | torrent | 4,998 |
| 23 | Jocko | 4,799 |
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com