Rust Distributed Systems

Open-source Rust projects categorized as Distributed Systems

Top 23 Rust Distributed System Projects

  • spacedrive

    Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.

    Project mention: (Ab)using a server library as a GUI - bad idea or only sort of bad idea? | reddit.com/r/rust | 2022-12-28

    In Tauri (or Axum) the app compiles to a single binary. rspc is the key to this because it allows for multiple transports with the frontend. It supports both Tauri IPC, HTTP or websockets. Our core crate (at ./core) exports an rspc router that is transport agnostic then within the apps (at ./apps/desktop or ./app/server) we expose it with a transport. We use Tauri IPC for desktop and websockets for Axum because we use subscriptions. Then in the wrapper React project (at ./apps/desktop/src/App.tsx) we create the rspc client with the Tauri link, mount its React context and then mount the app package (). You can give the codebase a look if you want cause it’s all open source https://github.com/spacedriveapp/spacedrive

  • talent-plan

    open source training courses about distributed database and distributed systems

    Project mention: How does serde_json work? | reddit.com/r/rust | 2022-11-18

    You should try this lab by PingCAP (write a serde implementation for redis serialization protocol) ; you can find answers/seek help in github if you really don't know to figure out it.

  • InfluxDB

    Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.

  • materialize

    Materialize is a fast, distributed SQL database built on streaming internals. (by MaterializeInc)

    Project mention: What makes a time series oriented database (ex: QuestDB) more efficient for OLAP on time series than an OLAP "only" oriented database (ex: DuckDB) technically? | reddit.com/r/dataengineering | 2023-01-23

    AFAIK there is a lot of overlap between OLAP databases and time series databases. Timescale](https://legacy-docs.timescale.com/v1.7/introduction/architecture) gains a lot of its performance via the "Hypertable" abstraction which is fairly similar to something like Parquet partitioning/bucketing. In terms of performance I don't know if there is a huge gap either for non optimized use cases. The [Clickhouse] team for example feels confident that Clickhouse can be used as a time series database. There are also [independent benchmarks showing the performance is comparable[(https://pradeepchhetri.xyz/clickhousevstimescaledb/). I think where time series specific databases excel are in their tooling for time series specific queries. Things like continuous aggregates or efficient gap filling. But non time series databases are catching up on that front. Clickhouse has live views and Materialize is also playing in that space

  • quickwit

    Cloud-native search engine for log management & analytics

    Project mention: Elastic, Loki and SigNoz – A Perf Benchmark of Open-Source Logging Platforms | news.ycombinator.com | 2023-01-24

    How is it compared to https://quickwit.io ?

  • Ockam

    Orchestrate end-to-end encryption, mutual authentication, key management, credential management & authorization policy enforcement — at scale.

    Project mention: Hiring - Ockam (Series A SaaS) | reddit.com/r/devopsjobs | 2023-01-12
  • bastion

    Highly-available Distributed Fault-tolerant Runtime

    Project mention: Bastion – Highly-Available Distributed Fault-Tolerant Runtime for Rust | news.ycombinator.com | 2022-10-27
  • rustpad

    Efficient and minimal collaborative code editor, self-hosted, no database required

    Project mention: Could someone suggest a editable, no-account, password protected paste-bin? | reddit.com/r/PrivacyGuides | 2022-12-14

    You could try rustpad

  • SonarLint

    Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.

  • habitat

    Modern applications with built-in automation

    Project mention: 42 Companies using Rust in production | dev.to | 2022-10-12

    And many more such as Scaleway, Oxide, Fuchsia, MeiliSearch, Vector, embark, Chef, BBC...

  • raft-rs

    Raft distributed consensus algorithm implemented in Rust.

  • stateright

    A model checker for implementing distributed systems.

    Project mention: Distributed Async Executors? | reddit.com/r/rust | 2022-12-05
  • rust-rdkafka

    A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka

    Project mention: A Rust client library for interacting with Microsoft Airsim https://github.com/Sollimann/airsim-client | reddit.com/r/robotics | 2023-01-22

    kafka

  • reth

    Modular, contributor-friendly and blazing-fast implementation of the Ethereum protocol, in Rust

    Project mention: [AMA] We are EF Research (Pt. 9: 11 January, 2023) | reddit.com/r/ethereum | 2023-01-10

    It wasn't explicitly mentioned as the reason, but I'm fairly certain it's associated with Paradigm's new client, Reth. This controversy raised some concerns for me. What is the team's opinion on the ability for large companies to "buy a seat at the table" by either building their own client or purchasing an existing client (for example, Offchain Labs recent acquisition of Prysmatic Labs)? Is this good to bad for the ecosystem? If bad, is there anything we can do to disincentivize this?

  • rust-crdt

    a collection of well-tested, serializable CRDTs for Rust

    Project mention: Rust-CRDT: Foundational CRDTs implemented in Rust | reddit.com/r/CRDTs | 2022-04-23
  • zenoh

    zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

    Project mention: This week in Rust 479 - Eclipse Zenoh release | reddit.com/r/rust | 2023-01-26

    The new Eclipse Zenoh 0.7.0 release, codename Charmander, brings to the table many features requested by the community on the Zenoh’s Discord server.

  • delicate

    A lightweight and distributed task scheduling platform written in rust. (一个轻量的分布式的任务调度平台通过rust编写)

  • Ruma

    A set of Rust crates for interacting with the Matrix chat network.

    Project mention: Looking for Axum app boilerplate | reddit.com/r/rust | 2023-01-19
  • little-raft

    The lightest distributed consensus library. Run your own replicated state machine! ❤️

    Project mention: Show /r/rust: Viewstamped Replication for Rust | reddit.com/r/rust | 2022-10-30

    My perspective is that VSR seems much more easier to grasp than Raft based on reading and contributing to Ilya Andreev's Little Raft. The selection of primary replica ("leader election" in Raft), for example, is dead-simple in VSR (you maintain a deterministic order of nodes and round-robin). Also, VSR seems to do a better job at decoupling the different scenarios at protocol level (backup fell behind, crashed, etc.) whereas in Raft the AppendEntries is a big hammer that does everything. I have very little experience on Paxos so can't really compare it to VSR.

  • anoma

    Reference implementation of the Anoma protocols in Rust.

    Project mention: Official /r/rust "Who's Hiring" thread for job-seekers and job-offerers [Rust 1.60] | reddit.com/r/rust | 2022-04-07

    COMPANY: Heliax (Project: Anoma)

  • tifs

    A distributed POSIX filesystem based on TiKV, with partition tolerance and strict consistency.

  • riteraft

    RiteRaft - A raft framework, for regular people

  • madsim

    Magical Deterministic Simulator for distributed systems in Rust.

    Project mention: Announcing Turmoil, a framework for testing distributed systems | reddit.com/r/rust | 2023-01-11

    How Turmoil different from madsim? Is Turmoil a successor of madsim?

  • veracruz

    Main repository for the Veracruz privacy-preserving compute project, an adopted project of the Confidential Compute Consortium (CCC).

    Project mention: Any active open source repos built using Rust that need development ? | reddit.com/r/rust | 2022-06-16

    Veracruz (https://github.com/veracruz-project/veracruz) always welcomes new contributors. It uses trusted hardware, remote attestation, and Wasm as a way of performing privacy-preserving collaborative computations, like privacy-preserving ML, or secret surveys, elections, and auctions.

  • nodejs-snowflake

    Generate time sortable 64 bits unique ids for distributed systems (inspired from twitter snowflake)

    Project mention: Python! Rust! What next?! | reddit.com/r/learnprogramming | 2022-05-19
  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-01-26.

Rust Distributed Systems related posts

Index

What are some of the best open-source Distributed System projects in Rust? This list will help you:

Project Stars
1 spacedrive 16,407
2 talent-plan 8,398
3 materialize 4,771
4 quickwit 2,954
5 Ockam 2,845
6 bastion 2,591
7 rustpad 2,532
8 habitat 2,432
9 raft-rs 2,322
10 stateright 1,307
11 rust-rdkafka 1,117
12 reth 1,062
13 rust-crdt 1,043
14 zenoh 673
15 delicate 540
16 Ruma 442
17 little-raft 367
18 anoma 320
19 tifs 314
20 riteraft 255
21 madsim 252
22 veracruz 161
23 nodejs-snowflake 114
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com