mazzle VS cue

I've been experimenting with alternative pipeline syntaxes.

One example is using Graphviz graph files as an execution plan for software deployments. You can see a graph here:

https://github.com/samsquire/mazzle/blob/master/docs/archite...

I wrote code with networkx to parse this graph dot file and then follow the plan topologically, spinning up packer, terraform, bash scripts, compilation and so on. It also executes graph nodes in parallel where it can.

I talk about it here https://devops-pipeline.com/

My problem with bash pipelines is passing data along and referring to previous steps' data. You kind of need to enrich the data that goes along the pipeline.

I use Graphviz dot syntax to configure entire cloud computing environments in the order they must be built. This is pipelines as code and infrastructure as code

It's very powerful! Not every organisation can bring up an entire environment with one command.

Here is a executable diagram of a build worker cloud nodes, Kubernetes, consul, vault, Debian package server, Java app, SSH bastion, Prometheus, grafana, DNS and security groups.

https://github.com/samsquire/mazzle/blob/master/docs/archite...

The tool also parallelises the build based on the graph so packer builds can run in parallel.

https://devops-pipeline.com/

It also has a GUI but it's not ready for other people to use.

I use a tool I wrote to layer my infrastructure with layers called components and I configure it with a Graphviz file.

My tool, called mazzle (previously devops-pipeline) would run parts of the graph that can run in parallel in parallel. It can also run parts of the build on SSH workers. You bring up the workers at the beginning of the build.

Here's an example of a graph generated from graphviz file: https://github.com/samsquire/mazzle/blob/master/docs/archite...

This graph brings up a hashicorp vault server, Java application, bastion proxy etc.

here's the graphviz file: https://github.com/samsquire/mazzle-starter/blob/master/arch...

It describes the ordering of the infrastructure, the invocation of Ansible, packer, shell scripts to set up vault etc.

The idea is to be able to bring up a new environment by changing one parameter. There's a React GUI too.

https://devops-pipeline.com

I'm continuing to work on https://concise-encoding.org which is a new security-conscious ad-hoc encoding format to replace JSON/XML and friends. I've been at it for 3 years so far and am close to a release.

In a nutshell:

- Edit in text, transmit in binary. One can be seamlessly converted to the other, but binary is far more efficient for processing, storage and transmission, while text is better for humans to read and edit (which happens far less often than the other things).

- Secure by design: Everything is tightly specced and accounted for so that there aren't differences between implementations that can be exploited to compromise your system. https://github.com/kstenerud/concise-encoding/blob/master/ce...

- Real type support because coercing everything into strings sucks (and is another security risk and source of incompatibilities).

XML had a good run but was replaced by JSON which was a big improvement. JSON also had a good run but it's time for it to retire now that the landscape has changed even further: Security and efficiency are the desires of today, and JSON provides neither.

I've got the spec nailed down and can finally see the light at the end of the tunnel for the reference implementation in golang. I still need to come up with a system for schemas, but I'm hoping that https://cuelang.org will fit the bill.

Has anyone taken a look at Cue who can share any experiences?

https://cuelang.org/

It's mentioned on the site as an alternative to Yaml. Recently watched (~half of) this intro to it: https://youtu.be/fR_yApIf6jU

the most interesting summary explanation of cue lang and its differences is from a bug filing - https://github.com/cuelang/cue/issues/33

>CUE is a bit different from the languages used in linguistics and more tailored to the general configuration issue as we've seen it at Google. But under the hood it adheres strictly to the concepts and principles of these approaches and we have been careful not to make the same mistakes made in BCL (which then were copied in all its offshoots). It also means that CUE can benefit from 30 years of research on this topic. For instance, under the hood, CUE uses a first-order unification algorithm, allowing us to build template extractors based on anti-unification (see issue #7 and #15), something that is not very meaningful or even possible with languages like BCL and Jsonnet.

I agree with you. Personally, I think Cue is much better than either YAML, TOML or JSON because it adds the concept of types to the idea of describing configuration.

true, but the tooling and workflow remains the same.

Not sure of any tool that could abstract the details sufficiently to be widely adopted. There is just too much nuance in cloud config.

I'm exploring using CUE (https://cuelang.org) to define TF resources, exporting as JSON for TF. So far it's much nicer