terraform-kubestack VS Grafana

I've recently released a major update for Kubestack, the Terraform framework for Kubernetes platform engineering teams. This update moves all functionality previously provided by Kubestack Cloud into the kbst CLI.

We've got some overlap on the stack part. But I am more focused on the infra side of platform engineering (including bootstrapping clusters), seems for you it's more app dependencies. Check https://www.kubestack.com and ping me if you're interested in chatting.

With Kubestack, the open-source Terraform framework I maintain, I'm on a mission to provide the best developer experience for teams working with Terraform and Kubernetes. And unified provisioning of all platform components, from cluster infrastructure to cluster services, is something I consider crucial in my relentless pursuit of said developer experience.

kops - Production Grade K8s Installation, Upgrades, and Management silver-surfer - Check ApiVersion compatibility and provide Migration path for Kubernetes objects when upgrading Kubernetes to latest versions Kube-ops-view - Kubernetes Operational View - read-only system dashboard for multiple K8s clusters kubeprompt - Kubernetes prompt info Metalk8s - An opinionated Kubernetes distribution with a focus on long-term on-prem deployments kind - Kubernetes IN Docker - local clusters for testing Kubernetes Clusterman - Cluster Autoscaler for Kubernetes and Mesos Cert-manager - Automatically provision and manage TLS certificates Goldilocks - Get your resource requests "Just Right" katafygio - Dump, or continuously backup Kubernetes objets as yaml files in git Rancher - Complete container management platform Sealed Secrets - A Kubernetes controller and tool for one-way encrypted Secrets OpenKruise/Kruise - Automate application workloads management on Kubernetes https://openkruise.io kubectl snapshot - Take Cluster Snapshots kapp - simple deployment tool focused on the concept of "Kubernetes application" — a set of resources with the same label https://get-kapp.io keda - Event-driven autoscaler for Kubernetes Octant - To better understand the complexity of Kubernetes clusters Portainer - Portainer inside a Kubernetes environment Gardener - Deliver fully-managed clusters at scale everywhere with your own Kubernetes-as-a-Service Kubed - Kubernetes Cluster Operator Daemon Kubestack - Kubestack is the free and open-source GitOps framework to codify your custom platform stack using Terraform.

In this post I hope to provide a few reasons why frameworks make sense for infrastructure as code, Having tried out Kubestack this got me thinking, Should more of these frameworks exist? And do they provide the same value as a traditional web framework like Flask, Express.js, or Ruby on Rails. Now you might be thinking don't tools like Terraform, Ansible, Chef already exists?

Unsurprisingly, my biased proposal is to use Kubestack to provision the GKE and EKS clusters leveraging the Kubestack framework's unified GKE and EKS modules, and to write a custom module to connect the resulting clusters to Anthos. The bespoke module would integrate the IAM, Anthos and Kubernetes resources required fully into the Terraform state and lifecycle instead of calling kubectl and gcloud like the official Google modules do.

Really cool project. I'll put it on the list of things to integrate with my project Kubestack https://github.com/kbst/terraform-kubestack

To help us visualize these scenarios, we'll build a Grafana Dashboard so we can follow along.

Visualization and Analysis: Choose a tool with intuitive and customizable dashboards, charts, and visualizations. A question to ask is, "Are the visualization features of this tool user-friendly and adaptable to our team's specific needs?" Tools like Grafana and Kibana provide powerful visualization capabilities.

Prometheus: Open-source monitoring system. Often used together with Grafana.

Grafana

Yes I'm aware that Grafana is open source but the method I used to find the API endpoints is far quicker than digging through hundreds of files in a codebase I'm not familiar with.

So, you will add one last container to allow us to visualize this data: Grafana, an open-source analytics and visualization platform that allows us to see traces and metrics simply. You can set Grafana to read data from both Tempo and Prometheus by setting them as datastores with the following grafana.datasource.yaml config file:

In example above, we use 2 additional parameters: code (HTTP response code) and page (page identifier), which provide detailed statistics. For example, you can build such graphs in Grafana:

Robin switched to the Grafana dashboard tab, and sure enough, the 5xx volume on web service was rising. It had not hit the critical alert thresholds yet, but customers had already started noticing.

I completely agree but do feel it needs qualifying. The problems beginners run into aren't usually the same as the problems experienced devs run into when adopting a language new to them, but where I see the two overlap I know something is a serious hazard in a language.

Java as a first language: won't like the boilerplate but won't have any point of comparison anyway, will get a few NPEs, might use threads and get data races but won't experience memory unsafety.

Go as a first language: much less boilerplate, but will still get nil panics, will be encouraged to use goroutines because every tutorial shows off how "easy" they are, will get data races with full blown memory unsafety immediately.

Rust as a first language: `None` // no examples found

I think Go as a beginner language would be better if people were discouraged from using goroutines instead of actively encouraged (the myth of "CSP solves everything"), otherwise I think it needs much better tooling to save people from walking off a cliff with their goroutines. And no, -race clearly isn't it, especially not for a beginner.

And in one respect I've found Go more of a hazard for experienced devs than beginners: the function signature of append() gives you the intuition of a functional programming append that never modifies the original slice. This has literally resulted in CVEs[1] even by experienced devs, especially combined with goroutines. Beginners won't have an intuition for this and will hopefully check the documentation instead of assuming.

[1] https://github.com/grafana/grafana/security/advisories/GHSA-...