Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 23 Rust Distributed System Projects
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Ockam
Orchestrate end-to-end encryption, cryptographic identities, mutual authentication, and authorization policies between distributed applications – at massive scale.
-
reth
Modular, contributor-friendly and blazing-fast implementation of the Ethereum protocol, in Rust
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
-
-
zenoh
zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.
-
-
delicate
A lightweight and distributed task scheduling platform written in rust. （一个轻量的分布式的任务调度平台通过rust编写）
-
-
-
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Need insights to build a distributed key value store from scratch. | /r/DistributedComputing | 2023-12-08Pls check this course: https://github.com/pingcap/talent-plan . It includes how to implement sql and key value store. It’s an awesome course
Project mention: Ask HN: How Can I Make My Front End React to Database Changes in Real-Time? | news.ycombinator.com | 2024-04-17[2] https://materialize.com/
disclosure: I work at Ockam.
The Portals for Mac app is an example of the type of thing you could build using the open source stack of protocols. The README (linked by parent) links out to all of the relevant parts of the protocol documentation to explain how these work together. The NAT Traversal (https://github.com/build-trust/ockam/blob/develop/examples/a...) part of the README is probably the best explanation of why the free relay you get via Ockam Orchestrator is a useful part of this demo.
As for why would anyone trust this: The protocols are designed so you absolutely don't have to trust the relay. Trust is pushed out to the edges that you control and so you're not susceptible to a MITM attack if something like a relay is compromised. The protocol design for all of this is open and documented, and was independently audited by (IMO) some of the best in the business, Trail of Bits: https://docs.ockam.io/reference/protocols.
Reth is Paradigm's new modular and blazing-fast implementation of Ethereum. The Reth team has already been using Kurtosis to spin up interop devnets and find bugs (source and bug fix).
Project mention: Stashpad launches Google Docs alternative you can use without any login | news.ycombinator.com | 2024-03-15Take a look at https://github.com/ekzhang/rustpad
Her recent project - https://github.com/aurae-runtime/aurae
Project mention: Rust Cpp Interop via Cxx, Autocxx / any best practices out there | /r/rust | 2023-06-19I use this library a lot and it's got some nice touches for how to handle wrapping a C library: https://github.com/fede1024/rust-rdkafka
Kubernetes (sorta) https://github.com/nxthat/nanocl It's still in early stages rn but always looking for more contributors!
Project mention: Local-first software: You own your data, in spite of the cloud (2019) | news.ycombinator.com | 2023-10-02Couchdb/pouchdb remains one of the best: it's super easy to setup and is production-ready, but it's gonna be json docs with no transactions, so it can be limiting.
Y.js and automerge emerged as solutions combining CRDTs and content transfer, they look really promising. There is a Y.rs version if that's better for you.
I've always dreamt of building something on top of Syncthing, ie something that would use file synchronization. It's more versatile and will definitely last longer than anything else, and it has some built-in capabilities for having a third party helping transport but not being allowed to read content.
I recently came across https://github.com/superfly/corrosion , a service discovery and state management tool that is working completely p2p. CR-SQLite, in particular, allows multiple tables from multiple databases to be merged thanks to CRDTs. I'm sure there's a lot to build on top of it.
I feel like you're not really interested in full p2p but want some centralization point to manage some auth stuff, so I'd investigate couchdb/pouchdb first.
Being able to control nondeterminism is particularly useful for testing and debugging. This allows creating reproducible test environments, as well as discrete-event simulation for faster-than-real-time simulation of time delays. For example, Cardano uses a simulation environment for the IO monad that closely follows core Haskell packages; Sui has a simulator based on madsim that provides an API-compatible replacement for the Tokio runtime and intercepts various POSIX API calls in order to enforce determinism. Both allow running the same code in production as in the simulator for testing.
Project mention: Building an open source vector database. Looking for advice. | /r/rust | 2023-12-07If you want some code examples, I wrote a db in rust that might help you: https://github.com/tontinton/dbeel
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Rust Distributed Systems related posts
- Interview with Mo Rajabi, co-founder and CEO of Noor
- On Implementation of Distributed Protocols
- A Rust Framework for Cloud Development: Volo
- Need insights to build a distributed key value store from scratch.
- Implementing Practical Byzantine Fault Tolerance - part 2
- Spacedrive Alpha 0.1.0
- Spacedrive is an open source cross-platform file explorer written in Rust
-
A note from our sponsor - InfluxDB
www.influxdata.com | 27 Apr 2024
Index
What are some of the best open-source Distributed System projects in Rust? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | talent-plan | 9,813 |
| 2 | materialize | 5,567 |
| 3 | Ockam | 4,347 |
| 4 | reth | 3,165 |
| 5 | rustpad | 3,054 |
| 6 | raft-rs | 2,795 |
| 7 | bastion | 2,759 |
| 8 | fluvio | 2,638 |
| 9 | habitat | 2,565 |
| 10 | aurae | 1,823 |
| 11 | stateright | 1,516 |
| 12 | rust-rdkafka | 1,477 |
| 13 | rust-crdt | 1,282 |
| 14 | zenoh | 1,243 |
| 15 | openraft | 1,205 |
| 16 | Coerce-rs | 670 |
| 17 | delicate | 651 |
| 18 | nanocl | 605 |
| 19 | corrosion | 586 |
| 20 | madsim | 582 |
| 21 | Ruma | 550 |
| 22 | Xline | 515 |
| 23 | dbeel | 459 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com