On Implementation of Distributed Protocols

• replica-io

  1 27 4.5

  Compose practical distributed replication mechanisms

  

Exploring distributed protocol implementations was the first phase of the initial state-of-the-art exploration. The next step is to select and examine some existing frameworks for developing distributed protocols in order to find out how they attempt to approach the problem and, perhaps, also discover some interesting techniques or ideas employed there. Then there are some potentially related concepts, approaches, and techniques worth looking into. The exploration tasks are tracked in the scope of this issue on GitHub.

• tendermint

  33 5,649 0.0 Go

  ⟁ Tendermint Core (BFT Consensus) in Go

  

Tendermint Core / CometBFT — a state machine replication engine (written in Go);

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• tokio

  196 24,677 9.5 Rust

  A runtime for writing reliable asynchronous applications with Rust. Provides I/O, networking, scheduling, timers, ...

  

Being able to control nondeterminism is particularly useful for testing and debugging. This allows creating reproducible test environments, as well as discrete-event simulation for faster-than-real-time simulation of time delays. For example, Cardano uses a simulation environment for the IO monad that closely follows core Haskell packages; Sui has a simulator based on madsim that provides an API-compatible replacement for the Tokio runtime and intercepts various POSIX API calls in order to enforce determinism. Both allow running the same code in production as in the simulator for testing.

• cometbft

  2 536 9.8 Go

  CometBFT (fork of Tendermint Core): A distributed, Byzantine fault-tolerant, deterministic state machine replication engine

  

Tendermint Core / CometBFT — a state machine replication engine (written in Go);

• raft

  4 532 8.8 Go

  Raft library for maintaining a replicated state machine (by etcd-io)

  

etcd Raft — a library for maintaining replicated state machines (written in Go);

• prometheus

  381 52,748 9.9 Go

  The Prometheus monitoring system and time series database.

  

Distributed system administrators need mechanisms and tools for monitoring individual nodes in order to analyze the system and promptly detect anomalies. Developers also need effective mechanisms for analyzing, diagnosing issues, and identifying bugs in protocol implementations. Logging, tracing, and collecting metrics are common observability techniques to allow monitoring and obtaining diagnostic information from the system; most of the explored code bases use these techniques. OpenTelemetry and Prometheus are popular open-source monitoring solutions, which are used in many of the explored code bases.

• aptos-core

  19 5,784 10.0 Rust

  Aptos is a layer 1 blockchain built to support the widespread use of blockchain through better technology and user experience.

  

AptosBFT — a consensus component supporting state machine replication in the Aptos blockchain (written in Rust);

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• library

  2 428 6.7 Java

  BFT-SMaRt's project home page (by bft-smart)

  

BFT-SMaRt — a library implementing BFT-SMaRt, a state machine replication system (written in Java);

• SmartBFT

  1 89 7.1 Go

  Implementation of the SmartBFT consensus library (https://arxiv.org/abs/2107.06922)

  

SmartBFT-Go — a library implementing state machine replication inspired by BFT-SMaRt (written in Go);

• substrate

  83 8,348 8.1 Rust

  Discontinued Substrate: The platform for blockchain innovators

  

Substrate — a framework for building application-specific blockchains (written in Rust);

• opentelemetry-go

  127 4,783 9.7 Go

  OpenTelemetry Go API and SDK

  

Distributed system administrators need mechanisms and tools for monitoring individual nodes in order to analyze the system and promptly detect anomalies. Developers also need effective mechanisms for analyzing, diagnosing issues, and identifying bugs in protocol implementations. Logging, tracing, and collecting metrics are common observability techniques to allow monitoring and obtaining diagnostic information from the system; most of the explored code bases use these techniques. OpenTelemetry and Prometheus are popular open-source monitoring solutions, which are used in many of the explored code bases.

• lighthouse

  124 2,738 9.7 Rust

  Ethereum consensus client in Rust (by sigp)

  

Lighthouse — an Ethereum consensus client (written in Rust);

• go-algorand

  215 1,324 9.4 Go

  Algorand's official implementation in Go.

  

Algorand — a blockchain based on the Algorand consensus protocol (written in Go);

• avalanchego

  39 2,065 9.8 Go

  Go implementation of an Avalanche node.

  

Avalanche — a blockchain platform based on the Avalanche consensus protocol (written in Go);

• rust-libp2p

  31 4,152 9.8 Rust

  The Rust Implementation of the libp2p networking stack.

  

Substrate and Lighthouse use libp2p as a networking stack for communication between nodes. The libp2p framework is a versatile modular peer-to-peer networking stack. It provides a collections of abstractions, mechanisms, and protocols for facilitating communication in P2P systems. In particular, libp2p supports multiple transport mechanisms (TCP, QUIC, WebSocket, WebTransport, etc.), encryption schemes (TLS and Noise), and stream multiplexing. Higher-level protocols in libp2p are implemented on top of reliable, ordered, bidirectional binary streams, which are transparently encrypted and multiplexed by the framework.

• ic

  42 1,469 10.0 Rust

  Internet Computer blockchain source: the client/replica software run by nodes

  

Internet Computer blockchain (ICP) — a general-purpose blockchain system developed by the DFINITY Foundation (written in Rust);

• sui

  12 5,724 10.0 Rust

  Sui, a next-generation smart contract platform with high throughput, low latency, and an asset-oriented programming model powered by the Move programming language

  

Sui — a smart contract platform based on Narwhal and Bullshark protocols (written in Rust);

• Apache ZooKeeper

  36 11,925 8.3 Java

  Apache ZooKeeper

  

Apache ZooKeeper — a distributed coordination, synchronization, and configuration service (written in Java);

• Apache Kafka

  26 27,335 9.9 Java

  Mirror of Apache Kafka

  

Apache Kafka — a distributed event streaming platform implementing a variant of the Raft consensus protocol (written in Java, integrated with Scala);

• cardano-node

  1 3,018 - Haskell

  Discontinued The core component that is used to participate in a Cardano decentralised blockchain. [Moved to: https://github.com/IntersectMBO/cardano-node] (by input-output-hk)

  

Cardano — a blockchain platform based on the Ouroboros family of consensus protocols (written in Haskell).

• anemo

  1 41 5.3 Rust
  

Communication layer in Sui is based on anemo, a peer-to-peer networking library built on top of QUIC. QUIC is a modern higher-level network transport protocol layered over UDP. It has built-in support for encryption and multiplexing. Similar to TCP connections, QUIC streams are reliable, ordered, bidirectional, providing flow control (backpressure), but they are cheap and almost instantaneous to open once an initial connection is established. The anemo library takes advantage of the efficient stream-multiplexing capability of QUIC; libp2p also uses the built-in capabilities of QUIC when it is used as a transport mechanism.

• madsim

  5 582 7.5 Rust

  Magical Deterministic Simulator for distributed systems in Rust.

  

Being able to control nondeterminism is particularly useful for testing and debugging. This allows creating reproducible test environments, as well as discrete-event simulation for faster-than-real-time simulation of time delays. For example, Cardano uses a simulation environment for the IO monad that closely follows core Haskell packages; Sui has a simulator based on madsim that provides an API-compatible replacement for the Tokio runtime and intercepts various POSIX API calls in order to enforce determinism. Both allow running the same code in production as in the simulator for testing.

• typed-protocols

  2 8 7.7 Haskell

  Session types framework with support of protocol pipelining.

  

Certain correctness properties of code can be ensured statically, i.e. at compile time. Those checks rely on the programming language's type system. Software engineers can take advantage of type safety features to implement components in a way that makes them safe by construction. For example, Cardano uses the typed-protocols package, a generic framework for implementing application-level protocols, which is based on a simple form of session typing.6 Within this framework, protocols are described as state machines encoded into Haskell types. The allowed transitions between states correspond to messages exchanged between the peers, so the protocol state determines which messages are allowed to be sent or must be accepted when received, at type level. This simplifies protocol implementation, allows early detection of protocol violations, and makes the protocols themselves deadlock-free by construction. More advanced type-level programming techniques may allow achieving impressive levels of type safety; however, such code may be significantly harder to implement, understand, and maintain.

• SaaSHub

  www.saashub.com sponsored

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project