Catlab.jl VS Dagger.jl

Related to this, AlgebraicJulia has been doing a lot with applying concepts from algebra and category theory to data analysis and modelling.

https://www.algebraicjulia.org/

There's some blog posts that are also interesting:

https://blog.algebraicjulia.org/

> Maybe I’m just too bitter about academia in this point in my career but it seems like we’ve run out of things to study and/or have too many people doing it.

We have certainly not run out of things to study, but I think we've hit the limit on what can effectively be communicated through traditional science journals [1], and we need to address the reproducibility crisis through open source science and reconsider the incentive structures around academia [2]. We need to oppose initiatives from people like Bill Gates who wish to privatize science through his various non-profits, as knowledge works better as as commons (we were unable to deal with the pandemic partly because Bill Gates prevented Oxford from open sourcing their work on COVID [3]). We need software that can compose scientific models [4], and organizations that can facilitate greater coordination among scientists. Science will become all the more important in an increasingly uncertain world, but are we up to the task?

[1] https://www.science.org/content/article/frustrated-science-s...

[1] https://numfocus.org/open-source-science-initiative-ossci

[2] https://www.wired.com/story/opinion-the-world-loses-under-bi...

[3] https://www.algebraicjulia.org/

Thank you for replying, but what prevents you from releasing your code? Dr Rydeheard has shared the StandardML version from his book (and the book). Of course if you don't want to share your code that is your prerogative and that is fine, but I am just trying to understand the issue that is preventing you a little more clearly. My interest in your implementation is strictly one of personal education. With applied category theory becoming more popular and computing implementations often used for teaching purposes (e.g. this book ) I would like to see a lisp implementation. It is built into Haskell, mostly, and people are developing libraries for Idris and Julia. I would find it instructive to see the implementation in common-lisp. Thank you for taking the time to respond to my original question.

The biggest group outside of numerical computing in Julia land are the PL and systems people though? This includes type theorists [1], database folks [2], distributed systems people ([3] to name just one). There are also a fair number of compiler nuts, hence the existence of multiple projects [4][5] in this space. And this is before getting into things that bridge more than one of the domains above, e.g. [7] or [8].

FTR, I think it's fair to question whether numerical computing should have an outsized influence on the direction of the language. I also think it's a pretty fair comparison to point out how standardized and consistent the Rust governance process is compared to Julia's (the Rust RFC system is an exemplar here). That doesn't mean there is a dearth of PL and systems knowledge in the Julia community though.

[1] https://github.com/AlgebraicJulia/Catlab.jl

Firstly, I'll say that we already have work started to implement out-of-core directly in Dagger: https://github.com/JuliaParallel/Dagger.jl/pull/289.

With that PR in place, it should be possible to define a "storage device" which is backed by a database. I haven't had a chance to actually try this, since the PR still needs quite some work and testing, but it's definitely something on my radar!

I'm not sure that's necessarily the domain of a low-level package like CUDA.jl though (which I assume you're referring to). That kind of interface is more the domain of higher-level packages like https://github.com/JuliaParallel/Dagger.jl/ and to a lesser extent https://juliagpu.github.io/KernelAbstractions.jl/stable/. Moreover, the jury is still out on whether the built-in Distributed module is an ideal abstraction for every use-case (clusters, heterogeneous compute, etc.)

WRT Nx, my biggest question is how they'll crack the problem of still needing big balls of C++ and the shims everywhere to get acceleration. Creating a compiler that generates efficient GPU or other accelerator code is a massive research project with no clear winners, never mind the challenge of reconciling the very mutation-heavy needs of GPU compute with a mostly immutable language model.