dtype-next VS neanderthal

Related (?): https://github.com/cnuernber/dtype-next/blob/master/test/tec...

Tablecloth by generateme is a friendly & expressive table-processing library built on top of tech.ml.dataset & dtype-next, Chris Nuernberger's high-performance data libraries.

There is one other detail here that I found out w/r/t arrays - Clojure's aset implementation returns the previous value; it isn't a faithful wrapper of the JVM's array set value instruction. Due to this if you are using aset on primitive arrays you end up boxing every value you are setting which at least in my tests leads to a performance disadvantage when compared to a tight loop using Java. This is why I have a specialized class implementing an aset that returns void.

If this is something common and the work is numeric, I would highly recommend exploring dtype-next buffer abstraction and tensors. The tensor api supports a nice APL-like substrate for working in index space without having to have the underlying storage "be" a boxed datastructure. You also get the option of off-heap / native tensors that can be zero-copy shuttled between other runtimes.

dtype-next - Major discoverability upgrades for the tech.v3.datatype and tech.v3.datatype.functional namespaces. Similarly to tmd, Cursive and Calva users now get full intellisense help with these main namespaces. Furthermore the FFI bindings now support linting with clj-kondo.

One thing about the readme that is incorrect - [dtype-next](https://github.com/cnuernber/dtype-next)'s ffi does in fact support callbacks :-). It is used as the backend to [libpython-clj](https://github.com/clj-python/libpython-clj) where you certainly can call clojure functions from python.

There are some interesting efforts concurrent with scicloj work by Chris Nuernberger specifically dtype-next, and the earlier tech-jna stuff. It's the same stuff underlying libpython-clj and libjulia-clj. recent talk.

This is great work. One of the things that has been on my mind working through our numerics stack is how to extend the number tower to complex numbers or more generally to arbitrary algebras. This project seems to me to be sort of a type-system-in-a-box that we can use to add arbitrary typing to Clojure where necessary/ideal. Thanks for sharing.

There is the official Hello World https://github.com/uncomplicate/neanderthal/tree/master/examples/hello-world

Did you have any occasion to evaluate neanderthal during your research? People seem to prefer it over core.matrix because it focus on primitive speed and sticking to BLAS idioms (as well as offering a decent api for working with GPU backends via cuda and opencl). I am curious to see if you did and found anything lacking there. I have a project on the backburner to try and target neanderthal for local search stuff, expressing problems in a high-level API that can then be baked into some numerically-friendly representation for efficient execution. It's often easier (trivial) to express solution representations, neighborhood functions, and objectives/constraints in a general purpose language, of which none of the things we like (sparse data structures, dynamically allocated stuff) are amenable to the contiguous memory, primitive numeric model that the hardware wants.

Do clojure as a side gig or in free time. Let day job pay the bills. If you can, maybe incorporate clojure into work job to solve small problems (https://github.com/clj-python/libpython-clj and https://github.com/scicloj/clojisr provide bridges to/from python and r). There is a lot of effort going into the data science side as well; the scicloj effort has resulted in a lot of growth over the last 2 years. tech.ml.dataset, tech.ml (now scicloj.ml). Dragan has a bunch of excellent stuff in neanderthal and deep diamond. There are also bindings to other jvm libraries from multiple languages.