rules_docker VS kubefed

My company uses Bazel's rules docker to build our images: https://github.com/bazelbuild/rules_docker

They're pretty great and have a lot of the caching and parallelism benefits mentioned in the post for free out of the box, along with determinism (which Docker files don't have because you can run arbitrary shell commands). Our backend stack is also built with Bazel so we get a nice tight integration to build our images that is pretty straightforward.

We've also built some nice tooling around this to automatically put our maven dependencies into different layers using Bazel query and buildozer. Since maven deps don't change often we get a lot of nice caching advantages.

I've seen rules_docker is looking for maintainers here ; Does this mean it doesn't use it that much internally? If so, how do they go about using other services e.g docker-compose for running external services e.g database?

Did you mean this one? https://github.com/bazelbuild/rules_docker

I was very interested in this Bazel-based way of building containers but its README page says "it is on minimal life support," which does not inspire confidence. How's your experience using it?

As others have said docker in docker or a separate build server are your best options using docker. You can also use Bazel (which doesn't require the docker daemon) to build docker images which will build deterministic images every time due to not incorporating the timestamp: https://github.com/bazelbuild/rules_docker

There's some BazelCon talks about people doing similar stuff but not actually open sourcing their code.

P.S. if you use rules_docker please feel free to open a PR to add your company to our README: https://github.com/bazelbuild/rules_docker/#adopters

The docker utility isn't the only way to build and run containers. There's also cri-o, podman, and crun among others for running containers. For building there is podman again, Jib for Java applications, and bazel plus many others. The docker approach of using a client to connect to a daemon required to run as root has turned out to be slow and insecure.

During the last 3 years I've had the pleasure of using Bazel's rules_docker to generate all my container images (https://github.com/bazelbuild/rules_docker).

In a nutshell, rules_docker is a set of build rules for the Bazel build system (https://bazel.build). What's pretty nice about these rules is that they don't rely on a Docker daemon. They are rules that directly construct image tarballs that you can either load into your local Docker daemon or push to a registry.

What's nice about this approach is that image generation works on any operating system. For example, even on a Mac or Windows system that doesn't have Docker installed, you're able to build Linux containers. They are also fully reproducible, meaning that you often don't need to upload layers when pushing (either because they haven't changed, or because some colleague/CI job already pushed those layers).

I guess rules_docker works fine for a variety of programming languages. I've mainly used it with Go, though.

The project is similar (in spirit) to kubefed.

What Is KubeFed? KubeFed (Kubernetes Cluster Federation) allows you to use a single Kubernetes cluster to coordinate multiple Kubernetes clusters. It can deploy multiple-cluster applications in different regions and design for disaster recovery. To learn more about KubeFed: https://github.com/kubernetes-sigs/kubefed

There's active work on a standard called kubefed [0] that is being worked on.

> I want a scale-to-zero node-pool in every region, and one kube master api for the world.

Personally, I'd generalize this to: "I want to describe the reliability requirements and configuration for my software and have an automated system solve for where, how many, when, and how to route to it"

I want to have something where I can say "I need to have high availability, lowest latency, and X GB of RAM and Y cores" and have a system automatically schedule me wherever compute is cheapest while also intelligently routing traffic to my servers based on client origins.

[0] - https://github.com/kubernetes-sigs/kubefed

Two of the more common approaches to deploying Kubernetes in hybrid environments are from cloud-to-cloud and cloud to on-prem. Whether this is from using a single control plane like Rancher, Platform9, or Gardener to create multiple clusters that are managed from a single location, or utilizing Kubernetes federation to create a cluster that spans different regions, this model has become a key feature offered by Kubernetes that has helped drive adoption.

This is made possible by the very nature of Kubernetes being a standard portable platform across cloud providers, ability to manage infrastructure as code, ability to setup networking between them whenever needed with the help of multi-cluster service meshes and also due to the ability to orchestrate the deployments using Kubefed and Crossplane.

And the interesting thing about networking in cloud is that it need not be just be limited to the cloud provider within your region but can span across multiple providers across multiple regions as needed and this is where projects like Kubefed, Crossplane definitely does help.

Projects like Kubefed and Crossplane are especially useful here since they help you to manage and orchestrate clusters and the requests you send across different cloud providers even if its going to be across regions.