kine VS flux2

-- Kine (Kine is not etcd)

Looking at [1], is there any reason why this couldn’t run in something like DynamoDB? I feel like the operations required would fit

https://github.com/k3s-io/kine/blob/master/pkg/drivers/gener...

I don't like some of the decisions that k3s makes which is why I stay away from it, but it's a great project. I'm not using kine yet, but I fantasize about using it exclusively every time I deal with etcd-induced slowness/random CPU spikes on my group of controller nodes. k0s supports kine too of course but I just haven't switched it on yet.

Well in some situations you can also get away with CRDTs or Gossip (SWIM), but yeah in the general sense Paxos (Raft is a Paxos-family algo). There are lots of paxos family algos to choose from[0].

> I’m also currently building a distributed search engine using etcd for the service registry, broker peer announcements, and worker queue and it’s been a good experience so far.

As a random stranger on the internet, please build an abstraction layer around etcd. Even if there's only one implementation, I've found that so many distributed projects that just never reach the scale that etcd is built for would benefit from the option of writing their own drivers (you don't need a full plugin system just a regular abstraction layer).

One example is Kine[0] for k8s -- if k8s had a built in option for writing/reading from something like Postgres from the beginning it would have been a better project for it, IMO.

If you do build a plugin system though, you can pass off the work of maintaining the other implementations!

[0]: https://vadosware.io/post/paxosmon-gotta-concensus-them-all

[1]: https://github.com/k3s-io/kine

I could spent >0s looking into this, but I've been curious how kline implements Etcd's Watch API. For the matter, I don't have much idea in general how kline handles data. Kline would be a great candidate for a blog post or introductory technical article on.

I do quickly see one issue- evidently watches presently use polling. The issue requests use of postgres triggers: https://github.com/k3s-io/kine/issues/20

check out https://github.com/k3s-io/kine

Not to be mistaken for high-availability Dqlite[1], which is one of the options one can run the k3s kubernetes distribution on (instead of etcd), via the Kine etcd shim[2]. Ultimately though the K3s team replaced Dqlite with an embedded etcd to get high-availability[3].

[1] https://dqlite.io/

[2] https://github.com/k3s-io/kine

[3] https://rancher.com/docs/k3s/latest/en/installation/ha-embed...

Given the team had already adopted GitOps and were familiar with deployments powered by Helm Releases and Flux, we wanted to move the provisioning of the infrastructure to be part of the same process of creating the service and its continuous deployment.

Your GitHub action can trigger a helm chart, or series thereof, or other infra tools. Declarative specifications, triggered procedurally with the context of the branch’s latest build. We use this pattern quite extensively for preview app workflows.

As of a year ago this is possible in a fully declarative way with Flux 2, but there’s a lot more moving parts and security footguns - and the idea that the maintenance of this project has lost one of its primary sponsors is worrying at best.

https://github.com/fluxcd/flux2/discussions/831

https://blog.kluctl.io/introducing-the-template-controller-a...

FluxCD - FluxCD is another popular GitOps tool that allows developers to use a Git repository as the sole source of configuration. Flux automatically ensures that the state of the Kubernetes cluster is synchronized with the configuration in the Git repository. It supports automatic updates, meaning Flux can monitor Docker image repositories for new images and push updates to the cluster.

#!/bin/bash aws eks update-kubeconfig --name $CLUSTER_NAME --region $AWS_REGION flux_installed=$(kubectl api-resources | grep flux) if [ -z "$flux_installed" ]; then echo "flux is not installed" curl -s https://fluxcd.io/install.sh | sudo bash flux bootstrap github \ --owner=$GH_USER_NAME \ --repository=$FLUX_REPO_NAME \ --path="clusters/$ENVIRONMENT/$CLUSTER_NAME/bootstrap" \ --branch=main \ --personal else echo "flux is installed" fi

Flux CD enables continuous deployment to Kubernetes through GitOps by syncing Git repositories with Kubernetes clusters. Flux CD enables GitOps for Kubernetes through source control integration. It manages Kubernetes manifests as code and syncs git repo changes to clusters. Flux automates checks, deployments, and updates within clusters.

FluxCD is a GitOps tool developed by Weaveworks that allows you to implement continuous and progressive delivery of your applications on Kubernetes. It is a CNCF graduated project that offers a set of controllers to monitor Git repositories and reconciles the cluster's actual state with the desired state defined by manifests committed in the repo.

From here, we can explore other developments and tutorials on Kubernetes, such as o11y or observability (PLG, ELK, ELF, TICK, Jaeger, Pyroscope), service mesh (Linkerd, Istio, NSM, Consul Connect, Cillium), and progressive delivery (ArgoCD, FluxCD, Spinnaker).

Instead, we will create a single long-lived cluster, and deploy our application in different namespaces. There are a bunch of ways to do that - see ArgoCD, Flux, custom internal tooling, or other solutions (we use our own product). That way, we:

One is to make sure configurations in both clusters is same. And for that there are many tools like fluxcd or projectsveltos