Top 23 Go Distributed System Projects
Distributed reliable key-value store for the most critical data of a distributed systemLatest mention: Gopher Gold #16 - Wed Oct 21 2020 | dev.to | 2020-10-21
etcd-io/etcd (Go): Distributed reliable key-value store for the most critical data of a distributed system
A realtime distributed messaging platform
Go Micro is a framework for distributed systems developmentLatest mention: Gopher Gold #16 - Wed Oct 21 2020 | dev.to | 2020-10-21
asim/go-micro (Go): A Go microservices development framework
Go Micro is a framework for distributed systems developmentLatest mention: Tturning dep project into go modules | reddit.com/r/golang | 2021-01-11
after you said that, i changed to github..com/micro/go-micro/v2 all the packages that use go-micro repo, but i keep receiving the same errors. What does "go: converting Gopkg.lock: stat github.com/micro/go-micro/v2/[email protected]: github.com/micro/[email protected]: invalid version: go.mod has post-v1 module path "github.com/micro/go-micro/v2" at revision eb1e22bd10b0" mean? Why it says "github.com/micro/[email protected]" if i set to version 2.9.1?
The Go language implementation of gRPC. HTTP/2 based RPCLatest mention: protoc plugin, with support for google/api/annotations.proto | reddit.com/r/golang | 2021-01-22
SeaweedFS is a distributed object store and file system to store and serve billions of files fast! Object store has O(1) disk seek, transparent cloud integration. Filer supports cross-cluster active-active replication, Kubernetes, POSIX, S3 API, encryption, Erasure Coding for warm storage, FUSE mount, Hadoop, WebDAV.
Micro is a platform for cloud native developmentLatest mention: Show HN: Micro – A platform for cloud native development in Go | news.ycombinator.com | 2021-01-25
The license for this project might be problematic. It is worded very vague and far reaching.
High-Performance server for NATS, the cloud native messaging system.Latest mention: Tiny message broker written in Go - nothing serious. Born out of COVID-boredom. Would love your suggestions. | reddit.com/r/golang | 2020-12-23
NATS has it's own custom text based protocol, and the client ecosystem is not as mature as others', like MQTT. In fact, I developed the NATS Dart client initially to be used from Flutter apps, but now can be used on Dart servers as well. Ivan from NATS is currently working on native MQTT v3.5 support (releasing sometime soon) and v5 support landing in Q1 2021 (Source: NATS Roadmap). I see here that they're building the Go implementation from scratch, which is interesting. They also seem to be converting every MQTT topic to a NATS topic for backward compatibility (obviously, duh) - I'm not sure if this brings any challenges down the line.
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.Latest mention: Gopher Gold #14 - Wed Oct 07 2020 | dev.to | 2020-10-07
hashicorp/nomad (Go): Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
The lightweight, distributed relational database built on SQLite.Latest mention: rqlite/rqlite | reddit.com/r/devopsish | 2021-01-23
A zero cost, faster multi-language bidirectional microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! 𝐉𝐚𝐯𝐚有𝐝𝐮𝐛𝐛𝐨, 𝐆𝐨𝐥𝐚𝐧𝐠有𝐫𝐩𝐜𝐱!
Distributed, offline-first bug tracker embedded in git, with bridgesLatest mention: Anyone using Fossil or Pijul with Rust projects? | reddit.com/r/rust | 2021-01-22
(Optionally with something like git-bug that lets you sync issues with GitHub, GitLab, etc.)
Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.Latest mention: A 10+ year Journey | dev.to | 2020-10-21
In 2015, Uber opened a development center in Seattle and I decided to take the leap and join the engineering team. Coincidently enough, Maxim Fateev ended up joining the Uber team in Seattle only a month after I did. At the time, Uber was running on Kafka 7 as the messaging backbone. Based on the scale they were running, they were encountering some serious operational issues. Considering Maxim and I had more than a decade worth of experience building messaging systems similar to Kafka, we decided to create the OSS project Cherami to address this Uber sized problem. After a year of working on the project, we were observing a very similar pattern to the one that we tried to solve with SWF and Durable Task Framework. When engineers needed to build with raw infrastructure primitives like queues and databases they were spending 80% of their time building resiliency into the application. This was clearly not sustainable for Uber, which was growing at an amazing pace and building a brand of "Transportation as Reliable as Running Water". This need to increase developer productivity without compromising on reliability of the system was the motivation for Maxim and I to create the OSS project Cadence. Within a very short period of time, we built a multi-tenant service hosted by our team. Cadence provided a great developer experience by enabling developers to use Golang to build and run stateful applications with very little operational overhead. Cadence grew organically within Uber and quickly became popular among developers. It slowly but surely began to emerge as the standard way to build stateful applications when reliability cannot be compromised.
Golang implementation of the Raft consensus protocol
Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
⟁ Tendermint Core (BFT Consensus) in Go
Full-featured BitTorrent client package and utilities
Ultra performant API Gateway with middlewares
A feature complete and high performance multi-group Raft library in Go.
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
High performance, distributed and low latency publish-subscribe platform.
JuiceFS is a distributed POSIX file system built on top of Redis and S3.Latest mention: A distributed Posix file system built on top of Redis and S3 | news.ycombinator.com | 2021-01-12
Most of network file system provide open-after-close consistency, same to JuiceFS, it's mentioned here :
What are some of the best open-source Distributed System projects in Go? This list will help you:
|6||Seaweed File System||11,340|