actions-runner-controller VS cert-manager

To set-up the self-hosted runner, an Action Runner Controller (ARC) and Runner scale sets application will be installed via helm. This post will be using Azure Kubernetes Service and ARC that is officialy maintained by Github. There is another ARC that is maintained by the community. You can follow the discussion where github adopted the ARC project into a full Github product here

Before this we were using https://github.com/actions/actions-runner-controller but that's running on K8s instead of VMs. So along with common limitations of running CI jos in K8s/container, it cannot have exactly the same environment as the official GitHub runners. Maintaining a K8s cluster was also very difficult.

ARC is great for running GitHub Actions on Kubernetes:

https://github.com/actions/actions-runner-controller

Almost all of our cicd, builds run on GitHub. I'm talking cypress tests, deployments via terraform and helm to over 25 environments, all backend tests, daily test runs etc. Overall we were racking up a cost of almost 20k on GitHub. With the ARC deployed and using spot instances I think our total infrastructure costs went up about 4-5k even though we added more actions. If we switched back to their runners we'd probably be around 25k at this point.

What else needs to be moved to my artifactory (charts - https://github.com/actions/actions-runner-controller/tree/master/charts ) - if so tar or entire folder or anything else ? ) What should the above steps correspond to?

You need to use the steps in the repo instead of the steps on the docs if you're using enterprise server.

Honestly not a fan of Github docs.....I feel like the ones in the repo are much clearer and easier to understand/read.

Its pretty easy to set up honestly. Deploy this on your k8s cluster https://github.com/actions/actions-runner-controller and a runnerDeployment and youre good to go.

Trying to use the Actions Runner Controller (https://github.com/actions/actions-runner-controller) to utilize self-hosted runners. I keep getting this error on the controller.

I'm convinced one of (or a combination) of things is happening here in regards to authentication. This GH enterprise account is configured with SAML. I feel like that is a valid data point. I'm using https://github.com/actions/actions-runner-controller as a reference guide for what I should be doing. I suspect whoever is Owner of this organization has modified what I can do as a user. The steps in the doc where I can actually Install the Application isn't available to me. When configuring the GitHub App I'm given two options. I select the option for "this account only" knowing the documentation says it is possible to use this Github App with a repo in the Organization as long as I have Admin privileges or I'm the owner.

cert-manager

The second one is a combination of tools: External DNS, cert-manager, and NGINX ingress. Using these as a stack, you can quickly deploy an application, making it available through a DNS with a TLS without much effort via simple annotations. When I first discovered External DNS, I was amazed at its quality.

On top of its core components, SpinKube depends on cert-manager. cert-Manager is responsible for provisioning and managing TLS certificates that are used by the admission webhook system of the Spin Operator. Let’s install cert-manager and KWasm using the commands shown here:

terraform { required_providers { kubectl = { source = "gavinbunney/kubectl" version = "1.14.0" } } } # The reference to the current project or a AWS project data "google_client_config" "provider" {} # The reference to the current cluster or EKS data "google_container_cluster" "my_cluster" { name = var.cluster_name location = var.cluster_location } # We configure the kubectl provider to use those values for authenticating provider "kubectl" { host = data.google_container_cluster.my_cluster.endpoint token = data.google_client_config.provider.access_token cluster_ca_certificate = base64decode(data.google_container_cluster.my_cluster.master_auth[0].cluster_ca_certificate) } #Download the multiple manifests file. data "http" "cert_manager_crds" { url = "https://github.com/cert-manager/cert-manager/releases/download/v${var.cert_manager_version}/cert-manager.crds.yaml" } data "kubectl_file_documents" "cert_manager_crds" { content = data.http.cert_manager_crds.response_body lifecycle { precondition { condition = 200 == data.http.cert_manager_crds.status_code error_message = "Status code invalid" } } } # We use the for_each or else this kubectl_manifest will only import the first manifest in the file. resource "kubectl_manifest" "cert_manager_crds" { for_each = data.kubectl_file_documents.cert_manager_crds.manifests yaml_body = each.value }

SSL certificates thanks to Cloudflare and cert-manager

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/${CERT_MANAGER_VERSION}/cert-manager.crds.yaml

put the Sub-CA inside hashicorp vault to be used for automatic signing of services like https://cert-manager.io/ inside our k8s clusters.

install-cert-manager: desc: Install cert-manager deps: - init-cluster cmds: - kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/{{.CERT_MANAGER_VERSION}}/cert-manager.yaml - echo "Waiting for cert-manager to be ready" && sleep 25 status: - kubectl -n cert-manager get pods | grep Running | wc -l | grep -q 3

I've been pretty frustrated with how private CAs are supported. Your private root CA can be maliciously used to MITM every domain on the Internet, even though you intend to use it for only a couple domain names. Most people forget to set Name Constraints when they create these and many helper tools lack support [1][2]. Worse, browser support for Name Constraints has been slow [3] and support isn't well tracked [4]. Public CAs give you certificate transparency and you can subscribe to events to detect mis-issuance. Some hosted private CAs like AWS's offer logs [5], but DIY setups don't.

Even still, there are a lot of folks happily using private CAs, they aren't the target audience for this initial release.

[1] https://github.com/FiloSottile/mkcert/issues/302

[2] https://github.com/cert-manager/cert-manager/issues/3655

[3] https://alexsci.com/blog/name-non-constraint/

[4] https://github.com/Netflix/bettertls/issues/19

[5] https://docs.aws.amazon.com/privateca/latest/userguide/secur...

the Cert Manager