etcd VS traefik

GitHub Repository. etcd-io/etcd

etcd (https://etcd.io/) is an open-source leader-based distributed key-value datastore designed by a vibrant team of engineers at CoreOS in 2013 and donated to Cloud Native Computing Foundation (CNCF) in 2018. Since then, etcd has grown to be adopted as a datastore in major projects like Kubernetes, CoreDNS, OpenStack, and other relevant tools. etcd is built to be simple, secure, reliable, and fast (benchmarked 10,000 writes/sec), it is written in Go and uses the Raft consensus algorithm to manage a highly-available replicated log. etcd is strongly consistent because it has strict serializability (https://jepsen.io/consistency/models/strict-serializable), which means a consistent global ordering of events, to be practical, no client subscribed to an etcd database will ever see a stale database (this isn't the case for NoSQl databases the eventual consistency of NoSQL databases ). Also unlike traditional SQL databases, etcd is distributed in nature, allowing high availability without sacrificing consistency.

etcd is an open-source leader-based distributed key-value datastore designed by a vibrant team of engineers at CoreOS in 2013 and donated to Cloud Native Computing Foundation (CNCF) in 2018. Since then, etcd has grown to be adopted as a datastore in major projects like Kubernetes, CoreDNS, OpenStack, and other relevant tools. etcd is built to be simple, secure, reliable, and fast (benchmarked 10,000 writes/sec), it is written in Go and uses the Raft consensus algorithm to manage a highly-available replicated log. etcd is strongly consistent because it has strict serializability, which means a consistent global ordering of events, to be practical, no client subscribed to an etcd database will ever see a stale database (this isn't the case for NoSQl databases the eventual consistency of NoSQL databases ). Also unlike traditional SQL databases, etcd is distributed in nature, allowing high availability without sacrificing consistency.

Unfortunately there is a big problem with custom controllers, they can't handle huge amount of data for several reasons. Kubernetes relies on ETCD for all data storage, which limits scalability, flexibility, and performance for complex or high-volume workloads. What kind of problems I'm talking about?

In Amazon EKS, AWS Config helps you track key components such as: EKS control plane logging, VPC settings and network exposure, encryption status for logs and secrets and IAM roles used by worker node groups. It can detect misconfigurations like: 🚫 Publicly accessible EKS clusters ⚠️ Disabled encryption for secrets stored in Kubernetes ETCD ⚠️ Ensures EKS clusters are running on currently supported versions,

curl -LO https://github.com/etcd-io/etcd/releases/download/v3.5.21/etcd-v3.5.21-linux-amd64.tar.gz tar xzf etcd-v3.5.21-linux-amd64.tar.gz

Etcd is a distributed key-value store, somewhat like Redis, but it operates quite differently under the hood (more on this later). It's implemented in Golang and is fully open-source.

> https://github.com/etcd-io/etcd/issues/9771

> stale bot marked this as completed (by fucking closing it)

Ah, yes, what would a Kubernetes-adjacent project be without a fucking stale bot to close issues willy nilly

Using a service discovery mechanism: A service discovery mechanism, such as etcd or ZooKeeper, can help to manage the complexity of microservices by providing a centralized registry of available services and their instances.

ETCD_VERSION='3.5.4' wget https://github.com/etcd-io/etcd/releases/download/v${ETCD_VERSION}/etcd-v${ETCD_VERSION}-linux-amd64.tar.gz tar -xvf etcd-v${ETCD_VERSION}-linux-amd64.tar.gz && cd etcd-v${ETCD_VERSION}-linux-amd64 cp -a etcd etcdctl /usr/bin/ nohup etcd >/tmp/etcd.log 2>&1 & etcd

Hetzner, Docker, Traefik as proxy and Apache / httpd inside the containers

With some more work, you could configure proxies that work lower on the infrastructure level, like NGINX, Envoy Proxy, HAProxy, or Traefik Proxy to give you the classic API gateway functionality. There's also KrakenD as another open-source option.

Traefik - Modern reverse proxy and load balancer

I wanted to try another one on Kubernetes. I chose Traefik because my searches mentioned it was the easiest to use in Kubernetes. As I mentioned, Traefik provides a Helm Chart, which makes it easy to install. Additionally, it integrates with OpenTelemetry.

Another problem is that the default certificates issued by Traefik, are not trusted by other systems or browsers. So we frequently need to bypass security warnings, which by itself is indicative of a problem and encourages bad habits. Furthermore, even if we manage to configure our setup to use the ingress service from within the cluster, it depends on the backend application if it allows bypassing TLS host checking.

Sidecar containers: Google Cloud Run has a cool feature where you can run multiple containers next to each other. So for example, if you want to run Caddy or Traefik as a reverse proxy for your ingress container and then have both your web frontend container & backend api container co-located in the same service, you can do that & have everything be super low latency.

traefik: link --> re-uses go/http: 1MB for headers

The main goal of this guide is to establish a streamlined process for deploying web applications with minimal effort. Using Amazon EC2 with Docker and Traefik as a reverse proxy, we will create a flexible server environment that supports multiple web applications and services, including databases like PostgreSQL, on different ports. This setup will ensure smooth deployment workflows, easy vertical scaling, and adaptable management of routing for various services, allowing for efficient expansion and integration of additional components as needed.