Go Distributed Systems

Open-source Go projects categorized as Distributed Systems | Edit details

Top 23 Go Distributed System Projects

  • GitHub repo etcd

    Distributed reliable key-value store for the most critical data of a distributed system

    Project mention: Package Management Nightmare | reddit.com/r/golang | 2021-10-12

    They have an open issue for it and looks like no blockers, and a PR bumping the otel version, so it looks like it's moving.

  • GitHub repo nsq

    A realtime distributed messaging platform

    Project mention: Fluvio: The Programmable Data Platform | news.ycombinator.com | 2021-07-10

    If we are talking about "cloud-native", I would really recommend taking a look at nsq or nats. I am really big fanboy of NSQ [1], it is simple, easy to setup and conceptually easy to learn.

    [1] https://nsq.io

  • Scout APM

    Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

  • GitHub repo go-micro

    A Go framework for distributed systems development

    Project mention: Show HN: A microservice framework, listed in CNCF Landscape, 1 year 10k+ stars | news.ycombinator.com | 2021-09-20

    Been striving in the way of micro services for a few years. My contribution is https://go-micro.dev (17k stars)

  • GitHub repo grpc-go

    The Go language implementation of gRPC. HTTP/2 based RPC

    Project mention: Testing gRPC services - request collections and modern load testing | dev.to | 2021-09-25

    From there you need to set an endpoint and request body in JSON format (Insomnia supports both unary and streaming calls) and hit Send. Here I'm using Greeter service from grpc-go repository ⬇️ It appears that gRPC support in Insomnia is indeed very young as there's no support yet for request headers or any response metadata other than status ⬇️ Good news is that Insomnia seems to be moving fast and is open source, so it's only a matter of time until this improvements are made by the core team or contributed from outside.

  • GitHub repo Seaweed File System

    SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, local tiering, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, Hadoop, WebDAV, encryption, Erasure Coding.

    Project mention: SeaweedFS uses skip list on Redis to store millions of files in one directory. | reddit.com/r/filesystems | 2021-10-11
  • GitHub repo micro

    Micro is a distributed cloud operating system (by micro)

    Project mention: Migrating from PHP to Go | reddit.com/r/golang | 2021-09-30

    https://micro.mu/ - a framework based approach that selects the core stuff you need.

  • GitHub repo Nomad

    Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.

    Project mention: How are you guys deploying containers? | reddit.com/r/homelab | 2021-10-03

    its an orchestration tool similar to something like K8S https://www.nomadproject.io/

  • Nanos

    Run Linux Software Faster and Safer than Linux with Unikernels.

  • GitHub repo NATS

    High-Performance server for NATS.io, the cloud and edge native messaging system.

    Project mention: Moving from REST API to pub/sub | reddit.com/r/dotnet | 2021-10-05

    We're looking into https://nats.io/ for a cloud/on-prem hybrid. Seems relatively easy to set up and can talk mqtt, websockets and SDKs for most languages. Perhaps it suits your needs!

  • GitHub repo rqlite

    The lightweight, distributed relational database built on SQLite

    Project mention: What is the best approach in order to send the database state to a server periodically? | reddit.com/r/learnprogramming | 2021-09-17

    Disclaimer: I have never tried this, but a quick search found this: https://github.com/rqlite/rqlite

  • GitHub repo rpcx

    Best microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! 𝐉𝐚𝐯𝐚有𝐝𝐮𝐛𝐛𝐨, 𝐆𝐨𝐥𝐚𝐧𝐠有𝐫𝐩𝐜𝐱!

  • GitHub repo cadence

    Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.

    Project mention: For those running Go in production at scale, what do you use for distributed task queues? | reddit.com/r/golang | 2021-06-04

    We use cadence https://github.com/uber/cadence

  • GitHub repo git-bug

    Distributed, offline-first bug tracker embedded in git, with bridges

    Project mention: Git as a Storage | news.ycombinator.com | 2021-10-08

    I've been doing something a bit similar with https://github.com/MichaelMure/git-bug/tree/master/entity/da... if you care about digging.

    It's a generic distrubuted data structure in git, with identities and signature, and conflict merge. At the moment it's used to store bugs, more to come later.

  • GitHub repo pachyderm

    Reproducible Data Science at Scale!

    Project mention: Dud: a tool for versioning data alongside source code, written in Go | reddit.com/r/golang | 2021-06-21
  • GitHub repo raft

    Golang implementation of the Raft consensus protocol

  • GitHub repo KrakenD

    Ultra performant API Gateway with middlewares. A project hosted at The Linux Foundation

    Project mention: Authelia is an open-source authentication/authorization server with 2FA/SSO | news.ycombinator.com | 2021-03-10

    Thanks for the pointer to Express API Gateway.

    A took a look at Krakend a while back, and it didn't seem to support it either [1]

    [1] https://github.com/devopsfaith/krakend/issues/274

  • GitHub repo Jocko

    Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

    Project mention: ZooKeeper-free Kafka is out. First Demo | news.ycombinator.com | 2021-03-30

    Reminded me of this project: https://github.com/travisjeffery/jocko

    Kafka implemented in Go without needing Zookeeper.

  • GitHub repo tendermint

    ⟁ Tendermint Core (BFT Consensus) in Go

    Project mention: Mises — Decentralized Personal Accounts and Social Relationships | reddit.com/r/Rad_Decentralization | 2021-10-18

    [4] Tendermint https://tendermint.com/

  • GitHub repo torrent

    Full-featured BitTorrent client package and utilities

    Project mention: exatorrent - Self-hostable , easy-to-use , lightweight and feature-rich torrent client written in Go | reddit.com/r/seedboxes | 2021-08-27
  • GitHub repo dragonboat

    A feature complete and high performance multi-group Raft library in Go.

  • GitHub repo juicefs

    JuiceFS is a distributed POSIX file system built on top of Redis and S3.

    Project mention: JuiceFS v0.15: Backup, Restore and Migration | news.ycombinator.com | 2021-07-07
  • GitHub repo emitter-io

    High performance, distributed and low latency publish-subscribe platform.

  • GitHub repo glow

    Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant. (by chrislusf)

  • GitHub repo gleam

    Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly. (by chrislusf)

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-10-18.

Index

What are some of the best open-source Distributed System projects in Go? This list will help you:

Project Stars
1 etcd 37,595
2 nsq 20,366
3 go-micro 16,975
4 grpc-go 14,705
5 Seaweed File System 12,995
6 micro 10,569
7 Nomad 10,112
8 NATS 10,006
9 rqlite 8,972
10 rpcx 6,098
11 cadence 5,365
12 git-bug 5,314
13 pachyderm 5,252
14 raft 5,162
15 KrakenD 4,602
16 Jocko 4,393
17 tendermint 4,376
18 torrent 4,066
19 dragonboat 3,880
20 juicefs 3,809
21 emitter-io 3,070
22 glow 2,994
23 gleam 2,906
Find remote jobs at our new job board 99remotejobs.com. There are 37 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com