magic-modules VS cue

Given: https://registry.terraform.io/providers/hashicorp/google/5.3...

how would any reasonable person know what https://registry.terraform.io/providers/hashicorp/google/5.3... to enable without (a) trying it and squinting at the error message (b) clicking on the <> then realizing it, also, does not mention run.googleapis.com, click on "supported service endpoints" <https://cloud.google.com/run/docs/reference/rest#rest_endpoi...> and only then learning about https://cloud.google.com/run/docs/reference/rest#service:-ru...

Repeat for https://registry.terraform.io/providers/hashicorp/google/5.3... although in both cases I guess the astute reader may have spotted the run.googleapis.com in the forbidden service labels and cloudidentity.googleapis.com in the example

Since, to the best of my knowledge those bindings are auto generated <https://github.com/GoogleCloudPlatform/magic-modules#magic-m...>, I would hypothesize it is not insurmountable drop in the seemingly existing declaration of APIs required: https://github.com/GoogleCloudPlatform/magic-modules/blob/7d... https://github.com/GoogleCloudPlatform/magic-modules/blob/7d...

In the case of the Google Cloud Platform provider, folks at Google built magic modules with the explicit goal of being able to generate schemas and behaviors for a Terraform provider and for other systems with similar needs. Since the vendor was explicitly aiming to support Terraform, this was the most ideal case where the schema could be designed to contain all of the information needed to generate a functional, usable provider.

I think the "Developing the provider" instructions in this provider's repository are rather stale, because they still talk about GOPATH even though that's been obsolete for several Go versions now. Note also that much of the code in that repository is auto-generated from an upstream repository googleCloudPlatform/magic-modules, and so for some changes it may be better to contribute there once you've tested the modifications more directly inside the provider repository.

For GCP, both ansible modules and terraform modules are actually generated from https://github.com/GoogleCloudPlatform/magic-modules, so their "production readiness" are the same.

I understand that mitchellh himself personally created a bunch of cloud modules for terraform at the beginning, and those were likely of higher quality than whatever created by some internal developers assigned by Google/Microsoft, and might be slightly better than the AWS modules maintained by community.

Anyway, when it comes to ansible versus terraform, we shall move the discourse to states management instead. With ansible, you don't have to deal with states, but will need to clean up the cloud resources separately. With terraform, you can use the tool to clean up the cloud resources easily, but then you also have the headache of managing states. Plus, whenever you change something, there is always the nagging feeling that it will do a destroy/recreate instead of an in-place update.

The Terraform provider for Google Cloud uses partial autogeneration, here is the repo that does the autogeneration for multiple automation tools:

https://github.com/GoogleCloudPlatform/magic-modules

I'm continuing to work on https://concise-encoding.org which is a new security-conscious ad-hoc encoding format to replace JSON/XML and friends. I've been at it for 3 years so far and am close to a release.

In a nutshell:

- Edit in text, transmit in binary. One can be seamlessly converted to the other, but binary is far more efficient for processing, storage and transmission, while text is better for humans to read and edit (which happens far less often than the other things).

- Secure by design: Everything is tightly specced and accounted for so that there aren't differences between implementations that can be exploited to compromise your system. https://github.com/kstenerud/concise-encoding/blob/master/ce...

- Real type support because coercing everything into strings sucks (and is another security risk and source of incompatibilities).

XML had a good run but was replaced by JSON which was a big improvement. JSON also had a good run but it's time for it to retire now that the landscape has changed even further: Security and efficiency are the desires of today, and JSON provides neither.

I've got the spec nailed down and can finally see the light at the end of the tunnel for the reference implementation in golang. I still need to come up with a system for schemas, but I'm hoping that https://cuelang.org will fit the bill.

Has anyone taken a look at Cue who can share any experiences?

https://cuelang.org/

It's mentioned on the site as an alternative to Yaml. Recently watched (~half of) this intro to it: https://youtu.be/fR_yApIf6jU

the most interesting summary explanation of cue lang and its differences is from a bug filing - https://github.com/cuelang/cue/issues/33

>CUE is a bit different from the languages used in linguistics and more tailored to the general configuration issue as we've seen it at Google. But under the hood it adheres strictly to the concepts and principles of these approaches and we have been careful not to make the same mistakes made in BCL (which then were copied in all its offshoots). It also means that CUE can benefit from 30 years of research on this topic. For instance, under the hood, CUE uses a first-order unification algorithm, allowing us to build template extractors based on anti-unification (see issue #7 and #15), something that is not very meaningful or even possible with languages like BCL and Jsonnet.

I agree with you. Personally, I think Cue is much better than either YAML, TOML or JSON because it adds the concept of types to the idea of describing configuration.

true, but the tooling and workflow remains the same.

Not sure of any tool that could abstract the details sufficiently to be widely adopted. There is just too much nuance in cloud config.

I'm exploring using CUE (https://cuelang.org) to define TF resources, exporting as JSON for TF. So far it's much nicer