github-workflows-kt VS cache

We use Kotlin to generate the yaml for our github actions: https://github.com/typesafegithub/github-workflows-kt

Nothing like a good old type safe compiled language to cut down on the verbosity, copy paste usage, silly syntax errors, weird undocumented you just have to know the magical incantations, etc. Kotlin or similar languages are the way to go. Much safer, more compact, easier to cut down on the copy paste reuse (which is just miserable drudgery), easy to introduce some sane abstractions where that makes sense. You get auto completion. And if it compiles, it's likely to just work.

People keep on moving around the deck chairs on the proverbial Titanic when it comes to configuration languages. Substituting yaml for json or toml just moves the problems. And substituting those with XML just introduces other issues and only marginally improves things. Well formed xml is nice. But so is well formed json. Schemas help, if the urls don't 404 and you have tools that can actually do something with them. Which, as it turns out is mostly not a thing in practice. And without that, it's just repetitive bloat. XML with schemas becomes very hard to read quickly.

There's a reason, people started ignoring XML once json became popular: json does most of the essential stuff well enough that XML just isn't worth the effort. And if you have something where you'd actually need the complexity of XML, it's likely to be some really ugly bloated kind of thing where the last thing you'd want to do is edit it manually.

I've dealt with cloudformation in XML form at some point in my life. It sucks. Not just a little bit. It's an absolute piss poor format for a thing like that. Since such a thing was lacking at the time, we ended up actually building our own little tools to generate that xml. Hand editing it was just too painful. One mistake could corrupt your entire stack. And it takes ages to find out if you actually got it right. In Json form it's hardly any better. It's just one of those convoluted over-engineered things. Anyway, Json support for cloudformation was not there at the time and the difference is like asking whether you'd preferred to be shot or stabbed. It's going to suck either way.

github-workflows-kt is a tool for creating GitHub Actions workflows in a type-safe script, helping you to build robust workflows for your GitHub projects without mistakes, with pleasure, in Kotlin.

The github-workflows-kt project uses Kotlin scripting, and it recommends doing everything using main.kts, because it's easier.

I use matrix tests with github actions to test my kt-search client with different versions of elastisearch and opensearch. Pretty easy to set up: https://github.com/jillesvangurp/kt-search/blob/master/.gith...

Basically it fires up elasticsearch using docker-compose and then the integration tests run against that. You could use a similar strategy to test different feature flag combinations.

For some of our private projects, we use kts to generate the github action yaml files using this: https://github.com/krzema12/github-workflows-kt

Well worth checking out if you have more complex workflows. Yaml is just horrible in terms of copy paste reuse. Also nice to get some compile time safety and auto complete with our action files.

One of my team members, Nikky, got annoyed with the verbosity and insane amount of copy-paste reuse needed to drive Github Actions. And true to her nature, promptly fixed it by using and contributing to GitHub Actions Kotlin DSL

GitHub: https://github.com/krzema12/github-actions-kotlin-dsl

> with no persistent storage

There's https://github.com/actions/cache though?

Use Cache Actions: GitHub Actions provides cache actions that simplify caching implementation. The @actions/cache JavaScript library is a popular choice for managing caching in workflows. It offers flexible options for storing and retrieving cache artifacts based on keys, scopes, and paths.

GitHub maintain a set of repos called actions. One of which is called cache.

By utilizing the actions/cache action action, we implemented a strategy to store and retrieve dependencies, preventing redundant installations.

I do think it's good practice to enable caching, such that your script doesn't hit RubyGems / pip / npm / etc every time it runs.

That way at least the automation activity stays entirely within the GitHub / Azure network.

It looks like you can do that for Ruby by adding this:

https://github.com/actions/cache/blob/master/examples.md#rub...

    - uses: ruby/setup-ruby@v1

➜ getting-started-with-act git:(master) act -j build WARN ⚠ You are using Apple M1 chip and you have not specified container architecture, you might encounter issues while running act. If so, try running it with '--container-architecture linux/amd64'. ⚠ [Node.js CI/build] 🚀 Start image=node:16-buster-slim [Node.js CI/build] 🐳 docker pull image=node:16-buster-slim platform= username= forcePull=false [Node.js CI/build] 🐳 docker create image=node:16-buster-slim platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] [Node.js CI/build] 🐳 docker run image=node:16-buster-slim platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] [Node.js CI/build] ☁ git clone 'https://github.com/actions/setup-node' # ref=v3 [Node.js CI/build] ☁ git clone 'https://github.com/actions/cache' # ref=v3 [Node.js CI/build] ☁ git clone 'https://github.com/actions/upload-artifact' # ref=v3 [Node.js CI/build] ⭐ Run Main actions/checkout@v3 [Node.js CI/build] 🐳 docker cp src=/Users/andrewevans/Documents/projects/getting-started-with-act/. dst=/Users/andrewevans/Documents/projects/getting-started-with-act [Node.js CI/build] ✅ Success - Main actions/checkout@v3 [Node.js CI/build] ⭐ Run Main Use Node.js 16.x [Node.js CI/build] 🐳 docker cp src=/Users/andrewevans/.cache/act/actions-setup-node@v3/ dst=/var/run/act/actions/actions-setup-node@v3/ [Node.js CI/build] 🐳 docker exec cmd=[node /var/run/act/actions/actions-setup-node@v3/dist/setup/index.js] user= workdir= [Node.js CI/build] 💬 ::debug::isExplicit: [Node.js CI/build] 💬 ::debug::explicit? false

You could try to find some point in the process where you can set up Actions caches with actions/cache, otherwise Container customization for Self-Hosted Runners is currently in Beta.

I would configure Github Actions cache so Docker doesn't have to compile all layers from scratch every time

In the copied example, npm caching is done via actions/cache@v2 action. But we can simplify our workflow by dropping this step and using built-in functionality for caching