polymorphic-functions VS dense-arrays

If performance is a concern, then you would want to stick to CLHS provided simple-array and create appropriate types using deftype, and then dispatch on the types either by yourself, or by using something like polymorphic-functions and polymorph.maths.

If you want to dispatch on vectors, you can try out polymorphic-functions which was made for the express purpose of dispatching on specialized arrays aka types rather than classes.

Since the past year or two, I have been working on numericals that aims to provide the speed of NumPy with the goodness of Common Lisp. In particular, this includes the use of dynamic variables, restarts, and compiler-notes wherever appropriate. It uses CLTL2 API (and may be slightly more) under the hood to provide AOT dispatch, but nothing stops you from combining it with JAOT dispatch provided by numcl/specialized-function. This also spawned a number of projects most notably polymorphic-functions to dispatch on types instead of classes and extensible-compound-types that allows one to define user defined compound types (beyond just the type-aliases enabled by deftype. Fortunately enough, interoperation between magicl, numcl and numericals/dense-numericals actually looks plausible!

Before reading this, please go and check out https://github.com/digikar99/polymorphic-functions which this project is fully based on. It's great.

What I am calling parametric polymorphism is this test:

*I have not; I have a couple of WIP/alpha-stage libraries like dense-arrays and numericals that could be useful; once I find the time, I want to think about if these or its dependencies can be integrated into the existing libraries including antik mentioned by awesome-cl.

Personally, I've been relying on the stream-based method using py4cl/2, mostly because I did not - and perhaps do not - have the knowledge and time to dig into the CFFI based method. The limitation is that this would get you less than 10000 python interactions per second. That is sufficient if you will be running a long running python task - and I have successfully run trivial ML programs using it, but any intensive array processing gets in the way. For this later task, there are a few emerging libraries like numcl and array-operations without SIMD (yet), and numericals using SIMD. For reasons mentioned on the readme, I recently cooked up dense-arrays. This has interchangeable backends and can also use cl-cuda. But barring that, the developer overhead of actually setting up native-CFFI ecosystem is still too high, and I'm back to py4cl/2 for tasks beyond array processing.

Currently I have put successfully this to use at dense-numericals - which I created over dense-arrays after finding CL arrays to be not that suitable, as compared to numpy or julia. Now, dense-numericals relies on passing the array pointer to C functions. However, IIUC, this runs into issues for what if the GC moves the arrays while the computation is still not done; is this worry valid? I think I ran into this while running multithreaded tests on CCL, ending up in segfaults.

Shameless unstable plug: I think it should be possible to provide type checking with a different backend that does not upgrade the types at https://github.com/digikar99/dense-arrays - the backend things are themselves unstable though.

In semi-production, ideally the problems are best represented using state diagrams, but I don't see a way to comfortably represent graphs in textual formats. The best I see is list of lists, which doesn't feel significantly better than the spaghetti code it currently is (for instance this and this - but these are just about one function each in a larger system, so not totally worth a DSL, unless there existed a defacto state-diagram DSL which everyone could be expected to know.