noyaml VS dhall-lang

Most commonly written in YAML, these files are large and complex to read and understand. And being written in YAML comes with its challenges (and quirks) since it is an additional programming language that devs need to learn.

YAML is kind of like C++:

> You like C++ because you're only using 20% of it. And that's fine, everyone only uses 20% of C++, the problem is that everyone uses a different 20% :)

https://eli.thegreenplace.net/2009/10/17/the-c-bashing-seaso...

The YAML footguns are too numerous to reproduce here, so here are some sources:

https://stackoverflow.com/questions/3790454/how-do-i-break-a...

https://www.arp242.net/yaml-config.html

https://noyaml.com/

Relevant: https://noyaml.com/

YAML and its ecosystem is full of footguns and ergonomics problems, especially when the length of the document extends beyond the height of a user's editor or viewport. Loss of context with indentation, non-compliant or unsafe parsers, and strange boolean handling to name a few.

It becomes even worse when people decide that static YAML data files should have variable substitution or control flow via templating. "Stringly-typed programming" if you will. If we all started writing JSON text templates I think a lot of people would rightly argue we should write small stdlib-only programs in Python, Typescript, or Ruby to emit this JSON instead of using templated text files. Then it becomes apparent that the YAML template isn't a static data file at all, but part of a program which emits YAML as output. We're already exposing people to basic programming if we're using YAML templates. People brew a special kind of YAML-templated devops hell using tools like Kustomize and Helm, each of which are "just YAML" but are full of idiosyncracies and tool-specific behaviour which make the use of YAML almost coincidental rather than a necessity.

Yes, sometimes people would prefer to look at YAML instead of JSON, in which case I suggest you use a YAML serialization library, or pipe output into a tool like `yq` so you can view the pretty output. In a pinch you could even output JSON and then feed it through a YAML formatter.

The Kubernetes community seems to have this penetrating "oh, it's just YAML" philosophy which means we get mediocre DSLs in "just YAML" which actually encode a lot of nuanced and unintuitive behaviour which varies from tool to tool.

Look at kyverno, for examle: it uses _parentheses_ in YAML key names to change the semantics of security policies! https://kyverno.io/docs/writing-policies/validate/ . This is different to what I think is the (much better ideas of) something like kubewarden, gatekeeper, or jspolicy, which allow engineers to write their policies in anything that compiles to WASM, OPA, and Typescript/Javascript respectively.

We engineers, as a discipline, have decades of know-how building and using general purpose programming languages with type checkers, linters, packaging systems, and other tools, but we throw them all away as soon as YAML comes along. It's time to put the stringified YAML templates away and engage in the ecosystem of mature tools we already to use to perform one simple task they are already good at: dumping JSON on stdout.

Let's move the control flow back into the tool and out of the YAML.

The webpage documenting some of the sharp edges of yaml is also displayed as an editable yaml document

https://noyaml.com/

It’s the least secure config format, even worse than XML IMO since it’s unsafe even with trusted inputs. https://noyaml.com/

Fail to see how this is any different than Dhall (https://dhall-lang.org/) other than it produces plists too.

Kubernetes config is a decent example. I had ChatGPT generate a representative silly example -- the content doesn't matter so much as the structure:

https://gist.github.com/cstrahan/528b00cd5c3a22e3d8f057bb1a7...

Now consider 100s (if not 1000s) of such files.

I haven't given Pkl an in depth look yet, but I can say that the Industry Standard™ of "simple YAML" + string substitution (with delicate, error prone indentation -- since YAML is indentation sensitive) is easily beat by any of:

- https://jsonnet.org/

- https://nickel-lang.org/

- https://nixos.org/manual/nix/stable/language/index.html

- https://dhall-lang.org/

- (insert many more here, probably including Pkl)

There are underpowered languages / tools, that can only solve a problem for which they are intended poorly. But not all limited tools are like that.

Say, eBPF is prominently not Turing-complete, which allows to guarantee that a eBPF program terminates, and even how soon. Still eBPF is hugely useful in its area.

Or, say, regular expressions are limited to regular languages; in particular, they famously [1] cannot process recursive structures, like trees. Still tools like grep / ag / rg are mightily useful.

Yes, I agree that YAML is underpowered for proper k8s configuration! But it's also too powerful for its own good in other aspects [2]. I wish Google used Dhall [3] or their own purely functional config language (FCL? I already forgot the name) instead of YAML; sadly, they did not.

[1]: https://stackoverflow.com/a/1732454/223424

[2]: https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-fr...

[3]: https://dhall-lang.org/

Dhall: Dhall is a programmable configuration language that combines features like JSON, functions, types, and import capabilities. Its style leans towards functional programming, so if you're familiar with functional-style languages such as Haskell, you might find Dhall to be quite intuitive.

I've been thinking along these lines but more 'strongly validated' than statically typed in the sense that you'd be better off being able to load the entire config and then produce a list of problems (and should be able to offer good editor support if done correctly).

Though https://dhall-lang.org/ demonstrates that you can statically type quite a lot of configuration to great advantage, which appears to be programmatically embeddable in multiple languages per https://docs.dhall-lang.org/howtos/How-to-integrate-Dhall.ht...

> Where practical is in the sense of an engineer (or in their terms, a CS practitioner),

Configuration processing. E.g. I'd like my yamls to be decidable, though I'd settle for guaranteed to halt[1].

[1] https://dhall-lang.org/

Maybe you'd like jsonnet: https://jsonnet.org/

I find it particularly useful for configurations that often have repeated boilerplate, like ansible playbooks or deploying a bunch of "similar-but" services to kubernetes (with https://tanka.dev).

Dhall is also quite interesting, with some tradeoffs: https://dhall-lang.org/

A few years ago I did a small comparison by re-implementing one of my simpler ansible playbooks: https://github.com/retzkek/ansible-dhall-jsonnet

And to checkout Dhall: https://dhall-lang.org/