Catalyst.jl VS MuladdMacro.jl

ModelingToolkit.jl adds a different spin on this by noting what makes a good modeling system isn't top down but a system that allows for bottom up contributions. ModelingToolkit is built on Symbolics.jl which uses OSCAR.jl etc., so every time the symbolics community gets better ModelingToolkit.jl gets better. It connects to the whole SciML ecosystem, so any improvement to any of the SciML interface packages is directly an improvement to ModelingToolkit.jl. ModelingToolkit is made to be a set of composable compiler abstractions called transformations, so anyone can add new packages that do new transformations that improve the ecosystem. One that I really like is MomentClosure.jl which symbolically transforms stochastic ModelingToolkit models (ReactionSystem) to approximate symbolic ODESystem models of the moments. And there's domain-specific langauges like Catalyst.jl being built on the interface to give more ways to build models, which is spawning the biocommunity to make model importers into the symbolic forms, when then feeds more ODE models into the same compiler. JuliaSim is then building on this ecosystem, adding cloud infrastructure that is special-purpose made for doing parallel computations of these models, automatic symbolic model discovery from data, automatic generation of approximate models with machine learning, and tying the Julia Computing compiler team into the web that is building this ecosystem.

Totally agreed. In Julia we use https://github.com/SciML/MuladdMacro.jl all over the place so that way it's contextual and does not bleed into other functions. fast-math changing everything is just... dangerous.

But if what you want is automatic FMA, then why carry along every other possible behavior with it? Just because you want FMA, suddenly NaNs are turned into Infs, subnormal numbers go to zero, handling of sin(x) at small values is inaccurate, etc? To me that's painting numerical handling in way too broad of strokes. FMA also only increases numerical accuracy, it doesn't decrease numerical accuracy, so bundling it with unsafe transformations makes one uncertain now whether it has improved or decreased accuracy.

For reference, to handle this well we use MuladdMacro.jl which is a semantic transformation that turns x*y+z into muladd expressions, and it does not recurse into functions so it does not change the definitions of the callers inside of the macro scope.

https://github.com/SciML/MuladdMacro.jl

This is something that will always increase performance and accuracy (performance because muladd in Julia is an FMA that is only applied if hardware FMA exists, effectively never resorting to a software FMA emulation) because it's targeted to do only a transformation that has that property.