On Implementation of Distributed Protocols

This page summarizes the projects mentioned and recommended in the original post on dev.to

CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai
featured
Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers
Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.
www.nutrient.io
featured
  1. replica-io

    Compose practical distributed replication mechanisms

    Exploring distributed protocol implementations was the first phase of the initial state-of-the-art exploration. The next step is to select and examine some existing frameworks for developing distributed protocols in order to find out how they attempt to approach the problem and, perhaps, also discover some interesting techniques or ideas employed there. Then there are some potentially related concepts, approaches, and techniques worth looking into. The exploration tasks are tracked in the scope of this issue on GitHub.

  2. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  3. tendermint

    ⟁ Tendermint Core (BFT Consensus) in Go

    Tendermint Core / CometBFT — a state machine replication engine (written in Go);

  4. tokio

    A runtime for writing reliable asynchronous applications with Rust. Provides I/O, networking, scheduling, timers, ...

    Being able to control nondeterminism is particularly useful for testing and debugging. This allows creating reproducible test environments, as well as discrete-event simulation for faster-than-real-time simulation of time delays. For example, Cardano uses a simulation environment for the IO monad that closely follows core Haskell packages; Sui has a simulator based on madsim that provides an API-compatible replacement for the Tokio runtime and intercepts various POSIX API calls in order to enforce determinism. Both allow running the same code in production as in the simulator for testing.

  5. cometbft

    CometBFT: A distributed, Byzantine fault-tolerant, deterministic state machine replication engine. A fork and successor to Tendermint Core.

    Tendermint Core / CometBFT — a state machine replication engine (written in Go);

  6. raft

    Raft library for maintaining a replicated state machine (by etcd-io)

    etcd Raft — a library for maintaining replicated state machines (written in Go);

  7. prometheus

    The Prometheus monitoring system and time series database.

    Distributed system administrators need mechanisms and tools for monitoring individual nodes in order to analyze the system and promptly detect anomalies. Developers also need effective mechanisms for analyzing, diagnosing issues, and identifying bugs in protocol implementations. Logging, tracing, and collecting metrics are common observability techniques to allow monitoring and obtaining diagnostic information from the system; most of the explored code bases use these techniques. OpenTelemetry and Prometheus are popular open-source monitoring solutions, which are used in many of the explored code bases.

  8. aptos-core

    Aptos is a layer 1 blockchain built to support the widespread use of blockchain through better technology and user experience.

    AptosBFT — a consensus component supporting state machine replication in the Aptos blockchain (written in Rust);

  9. Nutrient

    Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.

    Nutrient logo
  10. library

    BFT-SMaRt's project home page (by bft-smart)

    BFT-SMaRt — a library implementing BFT-SMaRt, a state machine replication system (written in Java);

  11. SmartBFT

    Implementation of the SmartBFT consensus library (https://arxiv.org/abs/2107.06922)

    SmartBFT-Go — a library implementing state machine replication inspired by BFT-SMaRt (written in Go);

  12. substrate

    Discontinued Substrate: The platform for blockchain innovators

    Substrate — a framework for building application-specific blockchains (written in Rust);

  13. opentelemetry-go

    OpenTelemetry Go API and SDK

    Distributed system administrators need mechanisms and tools for monitoring individual nodes in order to analyze the system and promptly detect anomalies. Developers also need effective mechanisms for analyzing, diagnosing issues, and identifying bugs in protocol implementations. Logging, tracing, and collecting metrics are common observability techniques to allow monitoring and obtaining diagnostic information from the system; most of the explored code bases use these techniques. OpenTelemetry and Prometheus are popular open-source monitoring solutions, which are used in many of the explored code bases.

  14. lighthouse

    Ethereum consensus client in Rust (by sigp)

    Lighthouse — an Ethereum consensus client (written in Rust);

  15. go-algorand

    Algorand's official implementation in Go.

    Algorand — a blockchain based on the Algorand consensus protocol (written in Go);

  16. avalanchego

    Go implementation of an Avalanche node.

    Avalanche — a blockchain platform based on the Avalanche consensus protocol (written in Go);

  17. rust-libp2p

    The Rust Implementation of the libp2p networking stack.

    Substrate and Lighthouse use libp2p as a networking stack for communication between nodes. The libp2p framework is a versatile modular peer-to-peer networking stack. It provides a collections of abstractions, mechanisms, and protocols for facilitating communication in P2P systems. In particular, libp2p supports multiple transport mechanisms (TCP, QUIC, WebSocket, WebTransport, etc.), encryption schemes (TLS and Noise), and stream multiplexing. Higher-level protocols in libp2p are implemented on top of reliable, ordered, bidirectional binary streams, which are transparently encrypted and multiplexed by the framework.

  18. ic

    Internet Computer blockchain source: the client/replica software run by nodes

    Internet Computer blockchain (ICP) — a general-purpose blockchain system developed by the DFINITY Foundation (written in Rust);

  19. sui

    Sui, a next-generation smart contract platform with high throughput, low latency, and an asset-oriented programming model powered by the Move programming language

    Sui — a smart contract platform based on Narwhal and Bullshark protocols (written in Rust);

  20. Apache ZooKeeper

    Apache ZooKeeper

    Apache ZooKeeper — a distributed coordination, synchronization, and configuration service (written in Java);

  21. Apache Kafka

    Mirror of Apache Kafka

    Apache Kafka — a distributed event streaming platform implementing a variant of the Raft consensus protocol (written in Java, integrated with Scala);

  22. cardano-node

    Discontinued The core component that is used to participate in a Cardano decentralised blockchain. [Moved to: https://github.com/IntersectMBO/cardano-node] (by input-output-hk)

    Cardano — a blockchain platform based on the Ouroboros family of consensus protocols (written in Haskell).

  23. anemo

    Communication layer in Sui is based on anemo, a peer-to-peer networking library built on top of QUIC. QUIC is a modern higher-level network transport protocol layered over UDP. It has built-in support for encryption and multiplexing. Similar to TCP connections, QUIC streams are reliable, ordered, bidirectional, providing flow control (backpressure), but they are cheap and almost instantaneous to open once an initial connection is established. The anemo library takes advantage of the efficient stream-multiplexing capability of QUIC; libp2p also uses the built-in capabilities of QUIC when it is used as a transport mechanism.

  24. madsim

    Magical Deterministic Simulator for distributed systems in Rust.

    Being able to control nondeterminism is particularly useful for testing and debugging. This allows creating reproducible test environments, as well as discrete-event simulation for faster-than-real-time simulation of time delays. For example, Cardano uses a simulation environment for the IO monad that closely follows core Haskell packages; Sui has a simulator based on madsim that provides an API-compatible replacement for the Tokio runtime and intercepts various POSIX API calls in order to enforce determinism. Both allow running the same code in production as in the simulator for testing.

  25. typed-protocols

    Session types framework with support of protocol pipelining.

    Certain correctness properties of code can be ensured statically, i.e. at compile time. Those checks rely on the programming language's type system. Software engineers can take advantage of type safety features to implement components in a way that makes them safe by construction. For example, Cardano uses the typed-protocols package, a generic framework for implementing application-level protocols, which is based on a simple form of session typing.6 Within this framework, protocols are described as state machines encoded into Haskell types. The allowed transitions between states correspond to messages exchanged between the peers, so the protocol state determines which messages are allowed to be sent or must be accepted when received, at type level. This simplifies protocol implementation, allows early detection of protocol violations, and makes the protocols themselves deadlock-free by construction. More advanced type-level programming techniques may allow achieving impressive levels of type safety; however, such code may be significantly harder to implement, understand, and maintain.

  26. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • CommetBFT 1.0: Byzantine Fault Tolerant middleware for replicated state machines

    1 project | news.ycombinator.com | 19 Dec 2024
  • CometBFT 'v1 Alpha Release'

    1 project | /r/cosmosnetwork | 6 Dec 2023
  • Fault Tolerance in Distributed Systems: Strategies and Case Studies

    4 projects | dev.to | 18 Oct 2023
  • Implementing a distributed key-value store on top of implementing Raft in Go

    5 projects | news.ycombinator.com | 25 May 2023
  • Introduction To Cosmos Network And Cosmos Hub

    3 projects | dev.to | 24 Jan 2023