PySR VS ModelingToolkitStandardLibrary.jl

> Yes, julia can be called from other languages rather easily

This seems false to me. StaticCompiler.jl [1] puts in their limitations that "GC-tracked allocations and global variables do not work with compile_executable or compile_shlib. This has some interesting consequences, including that all functions within the function you want to compile must either be inlined or return only native types (otherwise Julia would have to allocate a place to put the results, which will fail)." PackageCompiler.jl [2] has the same limitations if I'm not mistaken. So then you have to fall back to distributing the Julia "binary" with a full Julia runtime, which is pretty heavy. There are some packages which do this. For example, PySR [3] does this.

There is some word going around though that there is an even better static compiler in the making, but as long as that one is not publicly available I'd say that Julia cannot easily be called from other languages.

[1]: https://github.com/tshort/StaticCompiler.jl

[2]: https://github.com/JuliaLang/PackageCompiler.jl

[3]: https://github.com/MilesCranmer/PySR

I first learned about it with PySR https://github.com/MilesCranmer/PySR, they have an arxiv paper with some use cases as well.

I encourage everyone to read this paper. It's well written and easy to follow along. To the uninitiated, SR is the problem of finding a mathematical (symbolic) expression that most accurately describes a dataset of input-output examples (regression). The most naive implementation of SR is basically a breath first search starting from the simplest program tree: x -> sin(x) -> cos(x) ... sin(cos(tan(x))) until timeout. However, we can prune out equivalent expressions and, in general, the problem is embarrassingly parallel which alludes to some hope that we can solve this pretty fast (check out PySR[1] for a modern implementation). I find SR fascinating because it can be used for model distillation: learn a DNN approximation and "distill" it to a symbolic program.

Note that the paper talks about the decision version of the SR problem. ie: can we discover the global optimum expression. I think this proof is important for the SR community but not particularly surprising (to me). However, I'm excited by the potential future work for this paper! A couple of discussion points:

* First, SR is technically a bottom up program synthesis problem where the DSL (math) has an equivalence operator. Can we use this proof to impose stronger guarantees on the "hyperparameters" for bottom up synthesis. Conversely, does the theoretical foundation of the inductive synthesis literature [2] help us define tighter bounds?

* Second, while SR itself is NP hard, can we say anything about the approximate algorithms (eg: distilling a deep neural network to find a solution[3])? Specifically, what proof tell us about the PAC learnability of SR?

Anyhow, pretty cool seeing such work getting more attention!

[1] https://github.com/MilesCranmer/PySR

[2] https://susmitjha.github.io/papers/togis17.pdf

[3] https://astroautomata.com/paper/symbolic-neural-nets/

I found it curious that one of the implementations of symbolic regression (the "machine scientist" referenced in the article) is a Python wrapper on Julia: https://github.com/MilesCranmer/PySR

I don't think I've seen a Python wrapper on Julia code before.

This is really neat! Since you're interested in this subject, you may also appreciate PySR and the corresponding paper which uses Graph Neural Networks to perform symbolic regression.

I see, yes, RigidBodySim is in a bit of a bad place since Twan and Robin spend all of their time doing "real robotics projects" now (they are both at Boston Dynamics). I think it's fine though, since I don't think that that is the right implementation anymore anyways. Robotics simulators like Drake, (Diff)Taichi, MuJoCo, etc. end up numerically unstable when trying to model real physics, which is why still the big industrial simulations use Dymola. This is why these days it's all going the route of ModelingToolkit. MTK plus a differentiable simulator (DifferentialEquations.jl) already runs circles around MuJoCo and DiffTaichi, it just needs to complete its library to make building rigid body simulations a lot simpler. Once the mechanical components portion of the ModelingToolkit Standard Library is completed, we plan to demonstrate some things like control of UAVs and such. That's all slated for this year (and is connected to some things going on in JuliaSim), in which case I think we'll be in a much better state for this domain.

The thing to watch in the space of Simulink/Modelica is https://github.com/SciML/ModelingToolkit.jl . It's an acausal modeling system similar to Modelica (though extended to things like SDEs, PDEs, and nonlinear optimization), and has a standard library (https://github.com/SciML/ModelingToolkitStandardLibrary.jl) similar to the MSL. There's still a lot to do, but it's pretty functional at this point. The two other projects to watch are FunctionalModels.jl (https://github.com/tshort/FunctionalModels.jl, which is the renamed Sims.jl), which is built using ModelingToolkit.jl and puts a more functional interface on it. Then there's Modia.jl (https://github.com/ModiaSim/Modia.jl) which had a complete rewrite not too long ago, and in its new form it's fairly similar to ModelingToolkit.jl and the differences are more in the details. For causal modeling similar to Simulink, there's Causal.jl (https://github.com/zekeriyasari/Causal.jl) which is fairly feature-complete, though I think a lot of people these days are going towards acausal modeling instead so flipping Simulink -> acausal, and in that transition picking up Julia, is what I think is the most likely direction (and given MTK has gotten 40,000 downloads in the last year, I think there's good data backing it up).

And quick mention to bring it back to the main thread here, the DataDrivenDiffEq symbolic regression API gives back Symbolics.jl/ModelingToolkit.jl objects, meaning that the learned equations can be put directly into the simulation tools or composed with other physical models. We're really trying to marry this process modeling and engineering world with these "newer" AI tools.