etcd VS minio

GitHub Repository. etcd-io/etcd

etcd (https://etcd.io/) is an open-source leader-based distributed key-value datastore designed by a vibrant team of engineers at CoreOS in 2013 and donated to Cloud Native Computing Foundation (CNCF) in 2018. Since then, etcd has grown to be adopted as a datastore in major projects like Kubernetes, CoreDNS, OpenStack, and other relevant tools. etcd is built to be simple, secure, reliable, and fast (benchmarked 10,000 writes/sec), it is written in Go and uses the Raft consensus algorithm to manage a highly-available replicated log. etcd is strongly consistent because it has strict serializability (https://jepsen.io/consistency/models/strict-serializable), which means a consistent global ordering of events, to be practical, no client subscribed to an etcd database will ever see a stale database (this isn't the case for NoSQl databases the eventual consistency of NoSQL databases ). Also unlike traditional SQL databases, etcd is distributed in nature, allowing high availability without sacrificing consistency.

etcd is an open-source leader-based distributed key-value datastore designed by a vibrant team of engineers at CoreOS in 2013 and donated to Cloud Native Computing Foundation (CNCF) in 2018. Since then, etcd has grown to be adopted as a datastore in major projects like Kubernetes, CoreDNS, OpenStack, and other relevant tools. etcd is built to be simple, secure, reliable, and fast (benchmarked 10,000 writes/sec), it is written in Go and uses the Raft consensus algorithm to manage a highly-available replicated log. etcd is strongly consistent because it has strict serializability, which means a consistent global ordering of events, to be practical, no client subscribed to an etcd database will ever see a stale database (this isn't the case for NoSQl databases the eventual consistency of NoSQL databases ). Also unlike traditional SQL databases, etcd is distributed in nature, allowing high availability without sacrificing consistency.

Unfortunately there is a big problem with custom controllers, they can't handle huge amount of data for several reasons. Kubernetes relies on ETCD for all data storage, which limits scalability, flexibility, and performance for complex or high-volume workloads. What kind of problems I'm talking about?

In Amazon EKS, AWS Config helps you track key components such as: EKS control plane logging, VPC settings and network exposure, encryption status for logs and secrets and IAM roles used by worker node groups. It can detect misconfigurations like: 🚫 Publicly accessible EKS clusters ⚠️ Disabled encryption for secrets stored in Kubernetes ETCD ⚠️ Ensures EKS clusters are running on currently supported versions,

curl -LO https://github.com/etcd-io/etcd/releases/download/v3.5.21/etcd-v3.5.21-linux-amd64.tar.gz tar xzf etcd-v3.5.21-linux-amd64.tar.gz

Etcd is a distributed key-value store, somewhat like Redis, but it operates quite differently under the hood (more on this later). It's implemented in Golang and is fully open-source.

> https://github.com/etcd-io/etcd/issues/9771

> stale bot marked this as completed (by fucking closing it)

Ah, yes, what would a Kubernetes-adjacent project be without a fucking stale bot to close issues willy nilly

Using a service discovery mechanism: A service discovery mechanism, such as etcd or ZooKeeper, can help to manage the complexity of microservices by providing a centralized registry of available services and their instances.

ETCD_VERSION='3.5.4' wget https://github.com/etcd-io/etcd/releases/download/v${ETCD_VERSION}/etcd-v${ETCD_VERSION}-linux-amd64.tar.gz tar -xvf etcd-v${ETCD_VERSION}-linux-amd64.tar.gz && cd etcd-v${ETCD_VERSION}-linux-amd64 cp -a etcd etcdctl /usr/bin/ nohup etcd >/tmp/etcd.log 2>&1 & etcd

MinIO also switched to AGPLv3 a while back, and they stated that “the AGPL license requires that all software connecting with MinIO be 100% open source for you/your users not to be in violation of the license.”[^1] Since Redis and MinIO are somewhat similar, (Both can be used to store and retrieve data. One uses a custom protocol. The other one uses an S3-compatible API.) Should I assume that this statement also applies to Redis?

[^1]: https://github.com/minio/minio/issues/13308#issuecomment-929...

Congratulations on your launch, the animation makes it seem like a neat product!

I don't think I've ever seen a "coming soon" pricing page before <https://colanode.com/pricing/>

For my curiosity, your readme mentions Valkey but the docker compose uses Redis - is that on purpose? https://github.com/colanode/colanode/blob/v0.1.3/docker-comp...

You will also almost certainly want to either use the Apache 2 version of Minio[1] or label that dependency as AGPLv3 to ensure folks are aware. I would also recommend always pinning image versions, because you don't control what that project does or doesn't do in releases

1: https://github.com/minio/minio/blob/RELEASE.2021-04-22T15-44...

I havent tried yet, but I think we can use https://github.com/minio/minio for this.

To provide a comprehensive understanding, I will delve into the technical specifics of the issue as documented on GitHub (Issue #20559).

Maybe the license is pretty clear, but interpretations differ. E.g. Minio provides a very aggressive interpretation of AGPL, equating to "if you use it in closed-source commercial product, it's a violation of AGPL": [0], [1], [2]. For me the whole problem with AGPL is that it's so subjective.

[0] https://github.com/minio/minio/issues/13308#issuecomment-929...

[1] https://github.com/minio/minio/issues/12829#issuecomment-889...

[2] https://github.com/minio/minio/discussions/13571#discussionc...

I'm not sure that interpretation is correct. I don't read the intention that way (but I'd have to reread the licenses on detail to refresh myself before speaking more definitively).

Having said that, the fact that minio's documentation answers the question with a shrug and says "lawyer up" (see https://github.com/minio/minio/blob/master/COMPLIANCE.md) seems quite telling.

- MinIO uses 2-block https://github.com/minio/minio/blob/master/internal/grid/con...

In all of those cases if you make a change and just run 'go fmt' it very well could inject any new imports in the first block, which would be wrong and you wouldn't know until project CI picks it up.

> Just have them upload blobs to S3 or Azure Storage.

Not everyone can store their data in cloud services, most likely.

That said, S3 compatible solutions like MinIO might be a good choice: https://github.com/minio/minio or maybe SeaweedFS for something with a more permissive license: https://github.com/seaweedfs/seaweedfs