tofu-controller VS backstage

We stumbled upon a project for maintaining Terraform with CRDs that we could deploy with Helm. That project is now called Tofu-Controller - another WeaveWorks project, so it integrated great with our existing Flux setup.

https://github.com/weaveworks/tf-controller/issues/1166#issu...

Weave's TF-Controller, which also has fewer bugs, much better adoption, and it looks like it's actually being developed by someone. But requires a weird argocd <-> flux interop boilerplate. It's a "controller for flux" and not a Kubernetes controller, and I don't really get such ambiguous targeting , but meh...

Wrt tools, I wanted to integrate terraform with Flux thanks to their tf-controller. Conciling the core gitops features with terraform would be great imho.

I'm using Flux instead of Argo which has support for running terraform from a given Git Repo or OCI artifact so essentially I still fall back on Terraform when needed and it's applied via GitOps.

tf-controller which is integrated with Flux GitOps and reconciles Terraform files in a control loop

While searching for alternatives for running Terraform using Kubernetes, I found several controllers and operators, but none that I felt had as much potential as the tf-controller from Weaveworks. We are already using Flux as our GitOps tool, and the tf-controller works by utilizing some of the core functionality from Flux, and has a custom resource for Terraform deployments. The source controller takes care of fetching our modules, the kustomize controllers apply the Terraform resources, and then the controller spin up static pods (called runners) that runs your Terraform commands.

For us, GitOps is a vital part of how we operate, and it is the magic sauce that fuels our platform offering. Not only do we use it for application deployments, but by utilizing the Weaveworks tf-controller, we can create services using Terraform to automate our infrastructure deployments.

Some further research brought me to Weaveworks' TF-Controller which appears to be able to do what I want at least for the initial deployment step. Flux CD (also by Weaveworks) integrates with KEDA now, so it would be great if it could also integrate with KEDA to manage terraform-deployed Azure resources.

apiVersion: scaffolder.backstage.io/v1beta3 # https://backstage.io/docs/features/software-catalog/descriptor-format#kind-template kind: Template metadata: name: sample-btpsubaccount-remote-template title: Remote Template for SAP BTP Subaccount Setup description: A remote template that creates a basic SAP BTP Subaccount setup tags: - sap - btp - basic - javascript spec: owner: user:guest type: service

Backstage is an open-source platform developed by Spotify for managing the entire lifecycle of developer infrastructure, including services, APIs, documentation, and more. Backstage streamlines the development process through its centralized and customizable platform, offering a unified dashboard that consolidates information on projects, services, and infrastructure. Acting as a service catalog enhances transparency by allowing teams to document and discover internal services easily. Backstage's extensible architecture supports a robust plugin ecosystem, enabling teams to tailor the platform to their specific workflows and preferences. The platform promotes collaboration, accelerates onboarding through standardized documentation, and integrates seamlessly with various DevOps tools.

It’s also well understood that having a k8s cluster is not enough to make developers able to host their services - you need a devops team to work with them, using tools like delivery pipelines, Helm, kustomize, infra as code, service mesh, ingress, secrets management, key management - the list goes on! Developer Portals like Backstage, Port and Cortex have started to emerge to help manage some of this complexity.

We use Confluence and markdown files in GitHub. I think we are moving a lot of our docs to Backstage [0] soon.

One process that ends up being really valuable for documentation purposes is our "Architecture Review Documents". This is a standard document that team leads fill out before starting work on a new Saga/Epic/Feature/whatever. It includes the scope and business value of a new feature or large block of work, high level technical architecture of implementation, the impact on existing database schemas and service APIs, etc. This document is presented in a meeting with technical leadership in our organization who deep dive on the topic and explore potential pitfalls in the plan.

The document and recording of that meeting live on forever, and this information is very useful when getting acquainted with a certain part of our product/codebase. You are able to read and hear clearly the intention of a certain service or module, and you can identify several relevant points of contact to ask questions to.

[0] https://backstage.io/

Check the Backstage repo on GitHub

Backstage - An open platform for building developer portals. [Internal Developer Portal]

So you've just bought a new platform tool? Maybe it's Hashicorp Vault? Snyk? Backstage? You’re excited about all of the developer experience, security and other benefits you're about to unleash on your company—right? But wait…

The idea is to have a central point where people can find standards, documentation, and designs. The team can do this with a specialized tool like Backstage, Confluence, Github, Google Docs, or some internal implementation. The software is not the most important thing here, but having an easy way to find what is needed for the person to be more productive.