Top 23 Go Distributed System Projects
-
etcd
Distributed reliable key-value store for the most critical data of a distributed system
etcd-io/etcd (Go): Distributed reliable key-value store for the most critical data of a distributed system
-
nsq
A realtime distributed messaging platform
-
go-micro
Go Micro is a framework for distributed systems development
asim/go-micro (Go): A Go microservices development framework
-
go-micro
Go Micro is a framework for distributed systems development
after you said that, i changed to github..com/micro/go-micro/v2 all the packages that use go-micro repo, but i keep receiving the same errors. What does "go: converting Gopkg.lock: stat github.com/micro/go-micro/v2/[email protected]: github.com/micro/[email protected]: invalid version: go.mod has post-v1 module path "github.com/micro/go-micro/v2" at revision eb1e22bd10b0" mean? Why it says "github.com/micro/[email protected]" if i set to version 2.9.1?
-
grpc-go
The Go language implementation of gRPC. HTTP/2 based RPC
Latest mention: protoc plugin, with support for google/api/annotations.proto | reddit.com/r/golang | 2021-01-22 -
Seaweed File System
SeaweedFS is a distributed object store and file system to store and serve billions of files fast! Object store has O(1) disk seek, transparent cloud integration. Filer supports cross-cluster active-active replication, Kubernetes, POSIX, S3 API, encryption, Erasure Coding for warm storage, FUSE mount, Hadoop, WebDAV.
Latest mention: I received first-ever donation on my open-source side project and it felt great | news.ycombinator.com | 2021-01-12I am receiving $337 per month as of now for https://github.com/chrislusf/seaweedfs
Not something to be proud of if including the time spent to evolve the project.
-
micro
Micro is a platform for cloud native development
Latest mention: Show HN: Micro – A platform for cloud native development in Go | news.ycombinator.com | 2021-01-25The license[0] for this project might be problematic. It is worded very vague and far reaching.
-
NATS
High-Performance server for NATS, the cloud native messaging system.
Latest mention: Tiny message broker written in Go - nothing serious. Born out of COVID-boredom. Would love your suggestions. | reddit.com/r/golang | 2020-12-23NATS has it's own custom text based protocol, and the client ecosystem is not as mature as others', like MQTT. In fact, I developed the NATS Dart client initially to be used from Flutter apps, but now can be used on Dart servers as well. Ivan from NATS is currently working on native MQTT v3.5 support (releasing sometime soon) and v5 support landing in Q1 2021 (Source: NATS Roadmap). I see here that they're building the Go implementation from scratch, which is interesting. They also seem to be converting every MQTT topic to a NATS topic for backward compatibility (obviously, duh) - I'm not sure if this brings any challenges down the line.
-
Nomad
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
hashicorp/nomad (Go): Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
-
rqlite
The lightweight, distributed relational database built on SQLite.
-
rpcx
A zero cost, faster multi-language bidirectional microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! 𝐉𝐚𝐯𝐚有𝐝𝐮𝐛𝐛𝐨, 𝐆𝐨𝐥𝐚𝐧𝐠有𝐫𝐩𝐜𝐱!
-
git-bug
Distributed, offline-first bug tracker embedded in git, with bridges
(Optionally with something like git-bug that lets you sync issues with GitHub, GitLab, etc.)
-
cadence
Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
In 2015, Uber opened a development center in Seattle and I decided to take the leap and join the engineering team. Coincidently enough, Maxim Fateev ended up joining the Uber team in Seattle only a month after I did. At the time, Uber was running on Kafka 7 as the messaging backbone. Based on the scale they were running, they were encountering some serious operational issues. Considering Maxim and I had more than a decade worth of experience building messaging systems similar to Kafka, we decided to create the OSS project Cherami to address this Uber sized problem. After a year of working on the project, we were observing a very similar pattern to the one that we tried to solve with SWF and Durable Task Framework. When engineers needed to build with raw infrastructure primitives like queues and databases they were spending 80% of their time building resiliency into the application. This was clearly not sustainable for Uber, which was growing at an amazing pace and building a brand of "Transportation as Reliable as Running Water". This need to increase developer productivity without compromising on reliability of the system was the motivation for Maxim and I to create the OSS project Cadence. Within a very short period of time, we built a multi-tenant service hosted by our team. Cadence provided a great developer experience by enabling developers to use Golang to build and run stateful applications with very little operational overhead. Cadence grew organically within Uber and quickly became popular among developers. It slowly but surely began to emerge as the standard way to build stateful applications when reliability cannot be compromised.
-
raft
Golang implementation of the Raft consensus protocol
-
Jocko
Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
-
tendermint
⟁ Tendermint Core (BFT Consensus) in Go
-
torrent
Full-featured BitTorrent client package and utilities
-
KrakenD
Ultra performant API Gateway with middlewares
-
dragonboat
A feature complete and high performance multi-group Raft library in Go.
-
glow
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
-
gleam
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
-
emitter-io
High performance, distributed and low latency publish-subscribe platform.
-
juicefs
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
Latest mention: A distributed Posix file system built on top of Redis and S3 | news.ycombinator.com | 2021-01-12Most of network file system provide open-after-close consistency, same to JuiceFS, it's mentioned here [1]:
Index
What are some of the best open-source Distributed System projects in Go? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | etcd | 34,514 |
| 2 | nsq | 19,105 |
| 3 | go-micro | 15,279 |
| 4 | go-micro | 15,270 |
| 5 | grpc-go | 13,011 |
| 6 | Seaweed File System | 11,340 |
| 7 | micro | 9,537 |
| 8 | NATS | 8,776 |
| 9 | Nomad | 7,898 |
| 10 | rqlite | 7,367 |
| 11 | rpcx | 5,392 |
| 12 | git-bug | 4,896 |
| 13 | cadence | 4,614 |
| 14 | raft | 4,262 |
| 15 | Jocko | 4,118 |
| 16 | tendermint | 3,874 |
| 17 | torrent | 3,766 |
| 18 | KrakenD | 3,486 |
| 19 | dragonboat | 3,454 |
| 20 | glow | 2,915 |
| 21 | gleam | 2,731 |
| 22 | emitter-io | 2,706 |
| 23 | juicefs | 2,634 |