faust VS Enzyme

Glicol looks very cool! Also check out Faust if you haven't (https://faust.grame.fr), another FP sound programming language.

The linked (https://github.com/grame-cncm/faust) looks reasonable to me.

Chata probably needs to work out roughly what the semantics of the language should be. Its good to know what the library support is intended to be as that informs language design (assuming the library is to be implemented in chata anyway). Quite a lot of this page is about syntax.

There are some design decisions that have deep impact on programming languages. Reflection, mutation, memory management, control flow, concurrency. There are some implementation choices that end up constraining the language spec - python seems full of these.

Echoing p4bl0, implementing the language will change the spec. Writing a spec up front might be an interesting exercise anyway. I'd encourage doing both at the same time - sometimes describe what a feature should be and then implement it, sometimes implement something as best you can and then describe what you've got.

Implementation language will affect how long it takes to get something working, how good the thing will be and what you'll think about along the way. The usual guidance is to write in something familiar to you, ideally with pattern matching as compilers do a lot of DAG transforms.

- I'd say that writing a language in C took me ages and forced me to really carefully think through the data representation.

- Writing one in lua took very little time but the implementation was shaky, probably because it let me handwave a lot of the details.

- Writing a language in itself, from a baseline of not really having anything working, makes for very confusing debugging and (eventually) a totally clear understanding of the language semantics.

Good luck with the project.

Faust integration would be awesome: https://faust.grame.fr Then again we have MaxMSP, so in the end it feels kind of redundant

Csound is extremely powerful, but my favorite thing in this vein these days is Faust:

https://faust.grame.fr/

It's a functional language with a nice way of generating diagrams of DSP algorithms, but its big killer feature for me is its language bindings, which include C, C++, Cmajor, Codebox, CSharp, DLang, Java, JAX, Julia, JSFX, "old" C++, Rust, VHDL, and WebAssembly (wast/wasm) out of the box.

Have a look at FAUST as well: https://faust.grame.fr/

Once you have an idea of basic programming practice, you need to learn some DSP programming. One of the better tools for this is Faust https://faust.grame.fr/ , bear in mind this is a functional programming language, and has very different syntax to C++, but the same principles apply.

Automatic differentiation is done using https://enzyme.mit.edu/

lattner left google and was the primary reason they chose swift, so they lost interest.

if you're asking from an ML perspective, i believe the original motivation was to incorporate automatic differentiation in the swift compiler. i believe enzyme is the spiritual successor.

https://github.com/EnzymeAD/Enzyme

https://ispc.github.io/ispc.html

For the auto-differentiation when I need performance or memory, I currently use tapenade ( http://tapenade.inria.fr:8080/tapenade/index.jsp ) and/or manually written gradient when I need to fuse some kernel, but Enzyme ( https://enzyme.mit.edu/ ) is also very promising.

MPI for parallelization across machines.

For autodiff, we are currently working again on publishing a new Enzyme (https://enzyme.mit.edu) Frontend for Rust which can also handle pure Rust types, first version should be done in ~ a week.

Have you taken a look at enzymeAD? There is a group porting it to rust.

Enzyme dev here, so take everything I say as being a bit biased:

While, by design Enzyme is able to run very fast by operating within the compiler (see https://proceedings.neurips.cc/paper/2020/file/9332c513ef44b... for details) -- it aggressively prioritizes correctness. Of course that doesn't mean that there aren't bugs (we're only human and its a large codebase [https://github.com/EnzymeAD/Enzyme], especially if you're trying out newly-added features).

Notably, this is where the current rough edges for Julia users are -- Enzyme will throw an error saying it couldn't prove correctness, rather than running (there is a flag for "making a best guess, but that's off by default"). The exception to this is garbage collection, for which you can either run a static analysis, or stick to the "officially supported" subset of Julia that Enzyme specifies.

Incidentally, this is also where being a cross-language tool is really nice -- namely we can see edge cases/bug reports from any LLVM-based language (C/C++, Fortran, Swift, Rust, Python, Julia, etc). So far the biggest code we've handled (and verified correctness for) was O(1million) lines of LLVM from some C++ template hell.

I will also add that while I absolutely love (and will do everything I can to support) Enzyme being used throughout arbitrary Julia code: in addition to exposing a nice user-facing interface for custom rules in the Enzyme Julia bindings like Chris mentioned, some Julia-specific features (such as full garbage collection support) also need handling in Enzyme.jl, before Enzyme can be considered an "all Julia AD" framework. We are of course working on all of these things (and the more the merrier), but there's only a finite amount of time in the day. [^]

[^] Incidentally, this is in contrast to say C++/Fortran/Swift/etc, where Enzyme has much closer to whole-language coverage than Julia -- this isn't anything against GC/Julia/etc, but we just have things on our todo list.

Idk, Enzyme is pretty next gen, all the way down to LLVM code.

https://github.com/EnzymeAD/Enzyme

I'm working on merging my build-tool for (oxide)-enzyme into Enzyme itself. Also looking into improving the documentation.

that seems one of the points of enzyme[1], which was mentioned in the article.

[1] - https://enzyme.mit.edu/

being able in effect do interprocedural cross language analysis seems awesome.