Our great sponsors
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
noyaml
A silly emotional rant about the state of devops tooling/the infrastructure sector in 2018. #noyaml.com
-
-
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
-
-
-
-
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Indeed why? However the conclusion I have is not to use JSON but to use a type safe configuration language that can express my intent much better making illegal states impossible. One example of such lang is Dhall.
https://dhall-lang.org/
I would recommend implementing a similar API to Grafana Tanka: https://tanka.dev
When you "synthesise", the returned value should be an array or an object.
1. If it's an object, check if it has an `apiVersion` and `kind` key. If it does, yield that as a kubernetes object and do not recurse.
Relevant: https://noyaml.com/
YAML and its ecosystem is full of footguns and ergonomics problems, especially when the length of the document extends beyond the height of a user's editor or viewport. Loss of context with indentation, non-compliant or unsafe parsers, and strange boolean handling to name a few.
It becomes even worse when people decide that static YAML data files should have variable substitution or control flow via templating. "Stringly-typed programming" if you will. If we all started writing JSON text templates I think a lot of people would rightly argue we should write small stdlib-only programs in Python, Typescript, or Ruby to emit this JSON instead of using templated text files. Then it becomes apparent that the YAML template isn't a static data file at all, but part of a program which emits YAML as output. We're already exposing people to basic programming if we're using YAML templates. People brew a special kind of YAML-templated devops hell using tools like Kustomize and Helm, each of which are "just YAML" but are full of idiosyncracies and tool-specific behaviour which make the use of YAML almost coincidental rather than a necessity.
Yes, sometimes people would prefer to look at YAML instead of JSON, in which case I suggest you use a YAML serialization library, or pipe output into a tool like `yq` so you can view the pretty output. In a pinch you could even output JSON and then feed it through a YAML formatter.
The Kubernetes community seems to have this penetrating "oh, it's just YAML" philosophy which means we get mediocre DSLs in "just YAML" which actually encode a lot of nuanced and unintuitive behaviour which varies from tool to tool.
Look at kyverno, for examle: it uses _parentheses_ in YAML key names to change the semantics of security policies! https://kyverno.io/docs/writing-policies/validate/ . This is different to what I think is the (much better ideas of) something like kubewarden, gatekeeper, or jspolicy, which allow engineers to write their policies in anything that compiles to WASM, OPA, and Typescript/Javascript respectively.
We engineers, as a discipline, have decades of know-how building and using general purpose programming languages with type checkers, linters, packaging systems, and other tools, but we throw them all away as soon as YAML comes along. It's time to put the stringified YAML templates away and engage in the ecosystem of mature tools we already to use to perform one simple task they are already good at: dumping JSON on stdout.
Let's move the control flow back into the tool and out of the YAML.
Is the list on their website not good? https://github.com/json5/json5/wiki/In-the-Wild
And it shouldn't take much to modify an existing JSON parser.
Take a look at cdk8s from Amazon.
https://github.com/cdk8s-team/cdk8s-examples/tree/main/types...
This is where I usually pitch in with "Have your heard of CUELang, our lord and savior?": https://cuelang.org/
- Not turing complete
Personally I prefer INI over nearly all configuration formats.
https://github.com/madmurphy/libconfini/wiki/An-INI-critique...
- Portable - It runs pretty much anywhere
Our runtime takes the Starlark and creates environments in both Docker and Kubernetes; from one definition
Our CTO wrote this - https://docs.kurtosis.com/advanced-concepts/why-kurtosis-sta...
Here is a popular environment definition - https://github.com/kurtosis-tech/ethereum-package that protocol developers use to setup custom Ethereum test environments
> I don't think even though TOML has some official spec
Read it on https://toml.io/ (Full spec on upper-right… with its evolutions up to final 1.00 version).
For anyone struggling with Helm YAML syntax errors in their day job, I shamelessly advertise my browser-based debug tool Helm Playground: https://helm-playground.com/ - https://github.com/shipmight/helm-playground
HJson https://hjson.github.io seems a nice 'in-between' between YAML and JSON without the indentation-based syntax, so closer to the JSON side but with comments and less quotes.
What I don't really get is why the cloud providers / tooling implementors have never drafted up a "YAML-light" that just throws out the rarely-used headache-inducing syntax elements.
Ansible convinced me that doing programming tasks in YAML is insanity, so I started an experiment: What would Ansible be like if it's syntax were more like Python than YAML. https://github.com/linsomniac/uplaybook
I spent around 3 months over the holidays exploring that by implementing a "micro Ansible", I have a pretty solid tool that implements it, but haven't had much "seat time" with it: working on it rather than in it. But what I've done has convinced me that there are some benefits.
oh yeah; operators are great and sometimes they are necessary.
On the other hand, most operators I've seen are just k8s manifest templates implemented in Go.
I often end up preferring using Jsonnet to deal with that instead of doing the same stuff in Go.
Jsonnet is much more close to the underlying datamodel (the k8s manifest Json/Yaml document) and comes with some useful functionality out of the box, such "overlays".
It has downsides too! It's untyped, debugging tools are lacking, people are unfamiliar with it and don't care to learn it. So I totally get why one would entertain the possibility of writing your "templates" using a better language.
However, an operator is often too much freedom. It's not just using Go or Rust or Typescript to "generate" some Json manifests, but it also contains the code to interact with the API server, setup watches, and reactions etc.
I often wish there was a better way to separate those two concerns
I'm a fan of metacontroller [1], which is a tool that allows you to write operators without actually writing a lot of imperative code that interacts with the k8s API, but instead just provide a general JSON->JSON transformer, which you could write in any langue (Go, Python, Rust, Javascript, .... and also Jsonnet if you want).
I recently implemented something similar but much tailored to just "installing" stuff, called Kubit. An OCI artifact contains some abitrary tarball (generally containing some template sources) and a reference to a docker image containing an "engine" and runs the engine with your provided tarball + some parameters passed in a CRD. The OCI artifact could contain a helm chart and the template engine could contain the helm binary, or the template engine could be kubecfg and the OCI artifact could contain a bunch of jsonnet files. Or you could write your own stuff in python or typescript. The kubit operator then just runs your code, gathers the output and applies with with kubectl apply-set.
1. https://metacontroller.github.io/metacontroller/intro.html
2. https://github.com/kubecfg/kubit
In the case of GitHub Actions, it's made more painful by the lack of support for YAML anchors, which provide a bare minimum of composability.
https://github.com/actions/runner/issues/1182
> These same feelings extend to other proprietary config languages like HCL for Terraform, ASL for AWS Step Functions, etc. It's fine that you want a declarative API, but let me generate my declaration programatically.
Yeah, I've had the same sort of opinion since the bad old AWS CloudFormation days. I wrote an experimental CloudFormation generator 4 years ago where all of the resources and Python type hints were generated from a JSON file that AWS published and it worked really well (https://github.com/weberc2/nimbus/blob/master/examples/src/n...).
> Config declared in and generated by code has been a superior experience. It's one of the things that AWS CDK got absolutely right.
Is that how CDK works? I've only dabbled with it, but it was pretty far from the "generate cloudformation" experience that I had built; I guess I never "saw the light" for CDK. It felt like trading YAML/templating problems for inheritance/magic problems. I'd really like to hear from more people who have used AWS CDK, Terraform's CDK, and/or Pulumi.
https://github.com/nickelpack/nck
I wanted to use Nickel, but it turns out that it can't do everything you'd need it to do to completely replace NixLang. So right now I'm bikeshedding on what to use instead (and desperately trying not to invent something), in other words it's definitely being renamed. Either way there's a bash script in the `test` dir that shows the general concept.
I'm designing a simple dev environment from scratch.
My solution for this is a sandboxed lua for programatic configuration:
https://github.com/civboot/civlua/tree/main/lib/luck
I can't stand JSON (for many reasons) so I created a serialization format that combines it and CSV for nested objects
https://github.com/civboot/civlua/tree/main/lib/tso
I wish the industry would standardize on a solution like this. IMO you shouldn't use a "real" language unless you can lock it down to be determinisitic. JSON is supposed to be human readable but fails for lots of real-world data like multi-line strings or lists of records.
CSV is more readable but doesn't supported nested objects.
The mutations and side-effects only last until synthesis. You can imagine a CDK app as a pure function that runs a bunch of mutations on an App object and then serializes the state of that object in the end to static assets that can be deployed. The internals of it all are messy, but at a conceptual level, it's easy to think about.
CDKTF is really promising, IMO. When I last looked, it was still pretty new, but it's maturing, I think. One downside compared to regular AWS CDK is that the higher level constructs from the official AWS CDK can't be used in CDKTF. There is an adapter that exists, but it's one more layer between you and knowing what's going on: https://github.com/hashicorp/cdktf-aws-cdk