Go Distributed Systems

Open-source Go projects categorized as Distributed Systems
Edit details
Topics: Golang Go Kubernetes Microservices Database
The modern identity platform for B2B SaaS
The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
workos.com
Sponsored

Top 23 Go Distributed System Projects

  • etcd

    Distributed reliable key-value store for the most critical data of a distributed system

    • Project mention: Oracle Linux 8.8'de PostgreSQL 13 Yedekli Yapı Nasıl Kurulur? - Patroni, ETCD, HAProxy | dev.to | 2023-12-07

    sudo dnf -y install curl wget vim ETCD_RELEASE=$(curl -s https://api.github.com/repos/etcd-io/etcd/releases/latest|grep tag_name | cut -d '"' -f 4) echo $ETCD_RELEASE wget https://github.com/etcd-io/etcd/releases/download/${ETCD_RELEASE}/etcd-${ETCD_RELEASE}-linux-amd64.tar.gz tar xvf etcd-${ETCD_RELEASE}-linux-amd64.tar.gz cd etcd-${ETCD_RELEASE}-linux-amd64 sudo mv etcd* /usr/local/bin ls /usr/local/bin /usr/local/bin/etcd --version

  • nsq

    A realtime distributed messaging platform

    • Project mention: NSQ: Open-source realtime distributed messaging, billions of messages / day | news.ycombinator.com | 2024-01-09

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • go-micro

    A Go microservices framework

  • seaweedfs

    SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

    • Project mention: DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System | news.ycombinator.com | 2024-04-11

    Whoops: WebDAV:

    https://news.ycombinator.com/item?id=39417503

    SeaweedFS supports WebDAV. https://github.com/seaweedfs/seaweedfs/wiki/WebDAV

    I'm not able to find if both/restic supports mounting backups as WebDAV, but in theory there's nothing stopping you.

    It's 100% user space (expose a rest service) and supported by a bunch of file-browsers with a bit of a network aware component to it as well.

  • grpc-go

    The Go language implementation of gRPC. HTTP/2 based RPC

    • Project mention: Reverse Engineering Protobuf Definitions from Compiled Binaries | news.ycombinator.com | 2024-03-09

    The reflection service is open-sourced (at least for some sdks):

    * https://github.com/grpc/grpc-go/blob/master/Documentation/se...

    * https://chromium.googlesource.com/external/github.com/grpc/g...

  • rqlite

    The lightweight, distributed relational database built on SQLite.

    • Project mention: The lightweight, easy-to-use, distributed relational database built on SQLite | news.ycombinator.com | 2024-02-23

  • NATS

    High-Performance server for NATS.io, the cloud and edge native messaging system.

    • Project mention: Implementing OTel Trace Context Propagation Through Message Brokers with Go | dev.to | 2024-04-04

    Several message brokers, such as NATS and database queues, are not supported by OpenTelemetry (OTel) SDKs. This article will guide you on how to use context propagation explicitly with these message queues.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • Nomad

    Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.

    • Project mention: IBM Planning to Acquire HashiCorp | news.ycombinator.com | 2024-04-23

    I don't have any further insight, but looking at <https://github.com/hashicorp/nomad/forks?include=active&page...> coughed up https://github.com/atlassian/nomad/branches although confusingly it says "updated last week" but browsing any one of the branches seems to be stupid old so I got nothing

    Finding conceptual forks, e.g. $(git push --mirror ...) would be trickier but I bet sourcegraph could do it

    Ultimately, the question boils down to: what risk are you driving down: hitching your wagon to a dead stack, not getting security updates, not getting PRs merged, $other?

  • micro

    API first development platform (by micro)

    • Project mention: Micro – A Go service development platform | news.ycombinator.com | 2024-04-24

  • temporal

    Temporal service

    • Project mention: Rethinking Serverless with Flame | news.ycombinator.com | 2023-12-06

    I don't know if I agree with the argument regarding durability vs elastic execution. If I can get both (with a nice API/DX) via something like Temporal (https://github.com/temporalio/temporal), what's the drawback here?

  • juicefs

    JuiceFS is a distributed POSIX file system built on top of Redis and S3.

    • Project mention: South Korea's No.1 Search Engine Chose JuiceFS over Alluxio for AI Storage | dev.to | 2024-01-18

    Support for Kerberos keytab files

  • git-bug

    Distributed, offline-first bug tracker embedded in git, with bridges

    • Project mention: Radicle: Peer-to-Peer Collaboration with Git | news.ycombinator.com | 2024-03-30

    Unfortunately github appears to be actively breaking the ability to use git-bug on large repositories (like nixpkgs):

    https://github.com/MichaelMure/git-bug/issues/749#issuecomme...

  • rpcx

    Best microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! 𝐉𝐚𝐯𝐚有𝐝𝐮𝐛𝐛𝐨, 𝐆𝐨𝐥𝐚𝐧𝐠有𝐫𝐩𝐜𝐱! build for cloud!

  • raft

    Golang implementation of the Raft consensus protocol

    • Project mention: Leader election library | /r/golang | 2023-07-07

    Depending on your exact needs, you could try HashiCorp's Raft implementation: https://github.com/hashicorp/raft

  • cadence

    Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.

    • Project mention: Show HN: Hatchet – Open-source distributed task queue | news.ycombinator.com | 2024-03-08

  • kitex

    Go RPC framework with high-performance and strong-extensibility for building micro-services.

    • Project mention: How to Visualize and Analyze Data in Open Source Communities | dev.to | 2024-04-21

    After successful deployment, the data display panel looks as follows. Here, we present statistics and displays of the open-source community CloudWeGo, a project by ByteDance. The panels include:

  • pachyderm

    Data-Centric Pipelines and Data Versioning

    • Project mention: Open Source Advent Fun Wraps Up! | dev.to | 2024-01-05

    20. Pachyderm | Github | tutorial

  • KrakenD

    Ultra performant API Gateway with middlewares. A project hosted at The Linux Foundation

  • easegress

    A Cloud Native traffic orchestration system

    • Project mention: Easegress: Cloud Native traffic orchestration system | news.ycombinator.com | 2024-01-17

  • tendermint

    ⟁ Tendermint Core (BFT Consensus) in Go

    • Project mention: On Implementation of Distributed Protocols | dev.to | 2024-04-05

    Tendermint Core / CometBFT — a state machine replication engine (written in Go);

  • torrent

    Full-featured BitTorrent client package and utilities

    • Project mention: BTFS (BitTorrent Filesystem) | news.ycombinator.com | 2024-04-15

    https://github.com/anacrolix/torrent has a fuse driver since 2013. I'm in the early stages of removing it. There are WebDAV, 3rd party FUSE, and HTTP wrappers of the client all doing similar things: serving magnet links, infohashes, and torrent files like an immutable filesystem. BitTorrent v2 support is currently in master.

  • gaia

    Build powerful pipelines in any programming language.

  • dragonboat

    A feature complete and high performance multi-group Raft library in Go.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Go Distributed Systems related posts

Index

What are some of the best open-source Distributed System projects in Go? This list will help you:

Project Stars
1 etcd 46,345
2 nsq 24,561
3 go-micro 21,393
4 seaweedfs 21,013
5 grpc-go 19,836
6 rqlite 14,862
7 NATS 14,720
8 Nomad 14,398
9 micro 11,999
10 temporal 9,806
11 juicefs 9,791
12 git-bug 7,998
13 rpcx 7,946
14 raft 7,845
15 cadence 7,795
16 kitex 6,665
17 pachyderm 6,074
18 KrakenD 6,053
19 easegress 5,691
20 tendermint 5,649
21 torrent 5,313
22 gaia 5,156
23 dragonboat 4,936

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com