skycfg VS dhall-lang

I can definitely sympathize here - in every context, just straight JSON/YAML configuration seems never expressive enough, but the tooling created in response always seems to come with sharp edges.

Here are some of the things I appreciate about Jsonnet:

- It evals to JSON, so even though the semantics of the language are confusing, it is reasonably easy to eval and iterate on some Jsonnet until it emits what one is expecting - and after that, it's easy to create some validation tests so that regressions don't occur.

- It takes advantage of the fact that JSON is a lowest-common-denominator for many data serialization formats. YAML is technically a superset of JSON, so valid JSON is also valid YAML. Proto3 messages have a canonical JSON representation, so JSON can also adhere to protobuf schemas. This covers most "serialized data structure" use-cases I typically encounter (TOML and HCL are outliers, but many tools that accept those also accept equivalent JSON). This means that with a little bit of build-tool duct-taping, Jsonnet can be used to generate configurations for a wide variety of tooling.

- Jsonnet is itself a superset of JSON - so those more willing to write verbose JSON than learn Jsonnet can still write JSON that someone else can import/use elsewhere. Using Jsonnet does not preclude falling back to JSON.

- The tooling works well - installing the Jsonnet VSCode plugin brings in a code formatter that does an excellent job, and rules_jsonnet[0] provides good bazel integration, if that's your thing.

I'm excited about Jsonnet because now as long as other tool authors decide to consume JSON, I can more easily abstract away their verbosity without writing a purpose-built tool (looking at you, Kubernetes) without resorting to text templating (ahem Helm). Jsonnet might just be my "one JSON-generation language to rule them all"!

---

Though if Starlark is your thing, do checkout out skycfg[1]

[0] - https://github.com/bazelbuild/rules_jsonnet

[1] - https://github.com/stripe/skycfg

Can you say more about what GCL does better than all of the open source ones?

Anecdotally, I've heard a lot of GCL horror stories, and many Xooglers have chosen to create things like Jsonnet or Skycfg (https://github.com/stripe/skycfg) instead.

If you’re looking at other solutions check out https://github.com/stripe/skycfg It works with Envoy and lots of other things that support protobuf configs

This is good and there are several other options like https://github.com/stripe/skycfg#why-use-skycfg to add full language support (using Go or python for ex) to configurations.

Fail to see how this is any different than Dhall (https://dhall-lang.org/) other than it produces plists too.

Kubernetes config is a decent example. I had ChatGPT generate a representative silly example -- the content doesn't matter so much as the structure:

https://gist.github.com/cstrahan/528b00cd5c3a22e3d8f057bb1a7...

Now consider 100s (if not 1000s) of such files.

I haven't given Pkl an in depth look yet, but I can say that the Industry Standard™ of "simple YAML" + string substitution (with delicate, error prone indentation -- since YAML is indentation sensitive) is easily beat by any of:

- https://jsonnet.org/

- https://nickel-lang.org/

- https://nixos.org/manual/nix/stable/language/index.html

- https://dhall-lang.org/

- (insert many more here, probably including Pkl)

There are underpowered languages / tools, that can only solve a problem for which they are intended poorly. But not all limited tools are like that.

Say, eBPF is prominently not Turing-complete, which allows to guarantee that a eBPF program terminates, and even how soon. Still eBPF is hugely useful in its area.

Or, say, regular expressions are limited to regular languages; in particular, they famously [1] cannot process recursive structures, like trees. Still tools like grep / ag / rg are mightily useful.

Yes, I agree that YAML is underpowered for proper k8s configuration! But it's also too powerful for its own good in other aspects [2]. I wish Google used Dhall [3] or their own purely functional config language (FCL? I already forgot the name) instead of YAML; sadly, they did not.

[1]: https://stackoverflow.com/a/1732454/223424

[2]: https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-fr...

[3]: https://dhall-lang.org/

Dhall: Dhall is a programmable configuration language that combines features like JSON, functions, types, and import capabilities. Its style leans towards functional programming, so if you're familiar with functional-style languages such as Haskell, you might find Dhall to be quite intuitive.

I've been thinking along these lines but more 'strongly validated' than statically typed in the sense that you'd be better off being able to load the entire config and then produce a list of problems (and should be able to offer good editor support if done correctly).

Though https://dhall-lang.org/ demonstrates that you can statically type quite a lot of configuration to great advantage, which appears to be programmatically embeddable in multiple languages per https://docs.dhall-lang.org/howtos/How-to-integrate-Dhall.ht...

> Where practical is in the sense of an engineer (or in their terms, a CS practitioner),

Configuration processing. E.g. I'd like my yamls to be decidable, though I'd settle for guaranteed to halt[1].

[1] https://dhall-lang.org/

Maybe you'd like jsonnet: https://jsonnet.org/

I find it particularly useful for configurations that often have repeated boilerplate, like ansible playbooks or deploying a bunch of "similar-but" services to kubernetes (with https://tanka.dev).

Dhall is also quite interesting, with some tradeoffs: https://dhall-lang.org/

A few years ago I did a small comparison by re-implementing one of my simpler ansible playbooks: https://github.com/retzkek/ansible-dhall-jsonnet

And to checkout Dhall: https://dhall-lang.org/