kine VS tilt-extensions

-- Kine (Kine is not etcd)

Looking at [1], is there any reason why this couldn’t run in something like DynamoDB? I feel like the operations required would fit

https://github.com/k3s-io/kine/blob/master/pkg/drivers/gener...

I don't like some of the decisions that k3s makes which is why I stay away from it, but it's a great project. I'm not using kine yet, but I fantasize about using it exclusively every time I deal with etcd-induced slowness/random CPU spikes on my group of controller nodes. k0s supports kine too of course but I just haven't switched it on yet.

Well in some situations you can also get away with CRDTs or Gossip (SWIM), but yeah in the general sense Paxos (Raft is a Paxos-family algo). There are lots of paxos family algos to choose from[0].

> I’m also currently building a distributed search engine using etcd for the service registry, broker peer announcements, and worker queue and it’s been a good experience so far.

As a random stranger on the internet, please build an abstraction layer around etcd. Even if there's only one implementation, I've found that so many distributed projects that just never reach the scale that etcd is built for would benefit from the option of writing their own drivers (you don't need a full plugin system just a regular abstraction layer).

One example is Kine[0] for k8s -- if k8s had a built in option for writing/reading from something like Postgres from the beginning it would have been a better project for it, IMO.

If you do build a plugin system though, you can pass off the work of maintaining the other implementations!

[0]: https://vadosware.io/post/paxosmon-gotta-concensus-them-all

[1]: https://github.com/k3s-io/kine

I could spent >0s looking into this, but I've been curious how kline implements Etcd's Watch API. For the matter, I don't have much idea in general how kline handles data. Kline would be a great candidate for a blog post or introductory technical article on.

I do quickly see one issue- evidently watches presently use polling. The issue requests use of postgres triggers: https://github.com/k3s-io/kine/issues/20

check out https://github.com/k3s-io/kine

Not to be mistaken for high-availability Dqlite[1], which is one of the options one can run the k3s kubernetes distribution on (instead of etcd), via the Kine etcd shim[2]. Ultimately though the K3s team replaced Dqlite with an embedded etcd to get high-availability[3].

[1] https://dqlite.io/

[2] https://github.com/k3s-io/kine

[3] https://rancher.com/docs/k3s/latest/en/installation/ha-embed...

The first is the load function which loads Tilt extensions. It's a way to expand the tool's features, and several are available. Here we are using docker_build_with_restart, which will update the container running inside our Kubernetes cluster.

This is the central viewport into manual resource control and environment enhancement through the open-source extensions for Tilt.

argocd for cd Tilt

Recently, I found Tilt [0] to be a good partner of mine to run "all services locally". It can be compared to "webpack (live-reloading, a lot of configuration possibilities) for backend". You want to run a bunch of services directly? Use local_resource()/local(). You have Procfile? There is procfile() function. You have docker-compose.yml with databases? You can run it too with docker_compose(). You want have Tiltfile and include them all-together? There is load(). You need some web-ui for frontend devs and a nice log browser? It is there too. You need to do some extra steps before running a service? You want to update your local cluster with newly built image on file save? No problem, tilt will do that with k8s_yaml() function. Tilt uses Titlfiles for configuration, which are written Pythonish Starlark language and you use them to run any specific logic there.

Also, I am not very lucky in having resemble 1:1 k8s cluster locally. You could be close but as long as you don't run already in cloud you will have different configuration (additional annotations, various quirks that do not exist in kind/k3s but they are on GCP). However, making dedicated dev environments in the cloud might be very costly and incur a lot of additional tinkering.

[0]: https://tilt.dev/

If `host` means your production/staging hosts or whatever, you should get out of that habit now. Look into something like tilt.dev or telepresence.io or any number of other solutions that help solve this issue. Doing it directly on any host is just a recipe for bad habits and disaster.

The article doesn't mention k3d (https://k3d.io/) which is a variant of k3s that runs in docker (rather than a VM) - very nice for k8s dev/test on developer workstations.

It integrates very nicely with https://tilt.dev/ also (another very useful tool for k8s related dev/test).

If this is an issue during development then you can use something like tilt with docker-compose to directly copy modified source inside container and incrementally build it. https://tilt.dev/

I'd add Tilt

Tilt

Along similar lines, how about comparing Devspace to [Tilt](https://tilt.dev/)?