SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Distributed System Open-Source Projects
-
advanced-java
😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲:涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
spacedrive
Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.
-
xgboost
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
-
Nomad
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
-
NebulaGraph Database
A distributed, fast open-source graph database featuring horizontal scalability and high availability (by vesoft-inc)
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Scalability – Patterns of Scalable and Reliable Large-Scale Systems | news.ycombinator.com | 2024-05-07
Project mention: Oracle Linux 8.8'de PostgreSQL 13 Yedekli Yapı Nasıl Kurulur? - Patroni, ETCD, HAProxy | dev.to | 2023-12-07sudo dnf -y install curl wget vim ETCD_RELEASE=$(curl -s https://api.github.com/repos/etcd-io/etcd/releases/latest|grep tag_name | cut -d '"' -f 4) echo $ETCD_RELEASE wget https://github.com/etcd-io/etcd/releases/download/${ETCD_RELEASE}/etcd-${ETCD_RELEASE}-linux-amd64.tar.gz tar xvf etcd-${ETCD_RELEASE}-linux-amd64.tar.gz cd etcd-${ETCD_RELEASE}-linux-amd64 sudo mv etcd* /usr/local/bin ls /usr/local/bin /usr/local/bin/etcd --version
Project mention: Learn how to design systems at scale and prepare for system design interviews | news.ycombinator.com | 2023-07-05There are things in that doc that are completly wrong and ridiculous such as:
https://github.com/karanpratapsingh/system-design#streaming-...
In the video, Mo talked about a few packages like Cidre and StrOm, and we referred to SpaceDrive.
Project mention: stackgbm VS xgboost - a user suggested alternative | libhunt.com/r/stackgbm | 2024-05-05
Project mention: NSQ: Open-source realtime distributed messaging, billions of messages / day | news.ycombinator.com | 2024-01-09
Project mention: DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System | news.ycombinator.com | 2024-04-11Whoops: WebDAV:
https://news.ycombinator.com/item?id=39417503
SeaweedFS supports WebDAV. https://github.com/seaweedfs/seaweedfs/wiki/WebDAV
I'm not able to find if both/restic supports mounting backups as WebDAV, but in theory there's nothing stopping you.
It's 100% user space (expose a rest service) and supported by a bunch of file-browsers with a bit of a network aware component to it as well.
grpc-go: go get -u google.golang.org/grpc
Project mention: The lightweight, easy-to-use, distributed relational database built on SQLite | news.ycombinator.com | 2024-02-23
Project mention: Implementing OTel Trace Context Propagation Through Message Brokers with Go | dev.to | 2024-04-04Several message brokers, such as NATS and database queues, are not supported by OpenTelemetry (OTel) SDKs. This article will guide you on how to use context propagation explicitly with these message queues.
Project mention: Ask HN: Are there any open source forks of nomad smd consul? | news.ycombinator.com | 2024-05-02Doesn't look like it.
* https://github.com/hashicorp/nomad/forks
* https://github.com/search?q=nomad%20fork&type=repositories
* https://www.google.com/search?q=hashicorp+nomad+forks
There are products that do similar things of course.
Kotlin also has a construct for asynchronous collections/streams. Kotlin's version of AsyncSequence is called a Flow. Just as Swift's AsyncSequence builds upon prior experience with RxSwift and Combine, Kotlin's Flow APIs build upon earlier stream/collection APIs in the JVM ecosystem: Java's RxJava, Java8 Streams, Project Reactor, and Scala's Akka.
Apache ZooKeeper — a distributed coordination, synchronization, and configuration service (written in Java);
Project mention: Show HN: Hatchet – Open-source distributed task queue | news.ycombinator.com | 2024-03-08
Project mention: Ask HN: How do I find technical papers (for distributed systems)? | news.ycombinator.com | 2023-07-03Distributed systems is a very broad research topic nowadays, so it might make sense to check conferences covering a narrower topic, e.g. Cloud computing. Google Scholar might give some useful results for more specific search terms.
The reading lists for distributed systems university courses might also be interesting as well as this "awesome" link list on github:
https://github.com/theanalyst/awesome-distributed-systems
I don't know if I agree with the argument regarding durability vs elastic execution. If I can get both (with a nice API/DX) via something like Temporal (https://github.com/temporalio/temporal), what's the drawback here?
We should chat more about Reminders v2 - I want to learn more about what you'd like it to provide. Can you comment on the issue (https://github.com/dotnet/orleans/issues/7573)? If you'd like to see some of what is in the works, please message me on Discord and we can chat more about it (https://aka.ms/orleans/discord)
Project mention: JuiceFS 1.2 Beta 1: Gateway Upgrade, Enhanced Multi-User Permission Management | dev.to | 2024-05-05Feel free to download and try JuiceFS 1.2-beta1 here. If you have any questions, join JuiceFS discussions on GitHub and our community on Slack.
Distributed Systems related posts
-
Consistent Hashing: An Overview and Implementation in Golang
-
Configure Renovate to update preview versions of NuGet packages
-
Micro – A Go service development platform
-
IBM Planning to Acquire HashiCorp
-
SwiftPaxos: Fast Geo-Replicated State Machines
-
How to Visualize and Analyze Data in Open Source Communities
-
SwiftPaxos: Fast Geo-Replicated State Machines
-
A note from our sponsor - SaaSHub
www.saashub.com | 8 May 2024
Index
What are some of the best open-source Distributed System projects? This list will help you:
Project | Stars | |
---|---|---|
1 | advanced-java | 74,356 |
2 | awesome-scalability | 53,397 |
3 | etcd | 46,412 |
4 | Dubbo | 40,048 |
5 | system-design | 29,004 |
6 | spacedrive | 28,893 |
7 | xgboost | 25,595 |
8 | nsq | 24,589 |
9 | go-micro | 21,411 |
10 | seaweedfs | 21,123 |
11 | grpc-go | 19,899 |
12 | rqlite | 14,898 |
13 | NATS | 14,816 |
14 | Nomad | 14,450 |
15 | Akka | 12,931 |
16 | micro | 12,026 |
17 | Apache ZooKeeper | 11,937 |
18 | conductor | 10,688 |
19 | awesome-distributed-systems | 10,197 |
20 | NebulaGraph Database | 10,166 |
21 | temporal | 9,940 |
22 | Orleans | 9,841 |
23 | juicefs | 9,836 |
Sponsored