constructs VS Dagger.jl

To "meet developers where they are" is a beautiful tenet of AWS, and of the CDK, and inspired us to create awesome technology such as JSII and constructs.

I agree with constructs, I recommend using both (article mentions). Create your template, you might have various CDK constructs (community, AWS, personal), the template will produce a full project how you / your org want it. This might include your github actions for linting, cdk synth, testing, deployment etc. Application source (Lambda, API, container etc). Anyone in your team or external should be able to take that template and produce a project that is the same.

Have you heard of or explored https://github.com/aws/constructs (related: https://github.com/aws/jsii and https://github.com/aws/aws-cdk)?

This is what CDK uses for declarative modeling, but gives the opportunity to use languages/tooling that most devs are already familiar with. CDK8s already uses it as a replacement for yaml (technically, the yaml becomes an implementation detail rather than actually replaced)

All CDKs are based on Amazon's Constructs, (which also uses projen). They come with a CLI and Development Kit (API). In short, you set up an Object in code and then synthesize the representation to disk. This opens up the full power of programming languages. If you are sick of the issues with terraform, CloudFormation Templates, troposphere, Azure Blue Prints, or the like then this is for you.

Firstly, I'll say that we already have work started to implement out-of-core directly in Dagger: https://github.com/JuliaParallel/Dagger.jl/pull/289.

With that PR in place, it should be possible to define a "storage device" which is backed by a database. I haven't had a chance to actually try this, since the PR still needs quite some work and testing, but it's definitely something on my radar!

I'm not sure that's necessarily the domain of a low-level package like CUDA.jl though (which I assume you're referring to). That kind of interface is more the domain of higher-level packages like https://github.com/JuliaParallel/Dagger.jl/ and to a lesser extent https://juliagpu.github.io/KernelAbstractions.jl/stable/. Moreover, the jury is still out on whether the built-in Distributed module is an ideal abstraction for every use-case (clusters, heterogeneous compute, etc.)

WRT Nx, my biggest question is how they'll crack the problem of still needing big balls of C++ and the shims everywhere to get acceleration. Creating a compiler that generates efficient GPU or other accelerator code is a massive research project with no clear winners, never mind the challenge of reconciling the very mutation-heavy needs of GPU compute with a mostly immutable language model.