dense-arrays VS py4cl

*I have not; I have a couple of WIP/alpha-stage libraries like dense-arrays and numericals that could be useful; once I find the time, I want to think about if these or its dependencies can be integrated into the existing libraries including antik mentioned by awesome-cl.

Personally, I've been relying on the stream-based method using py4cl/2, mostly because I did not - and perhaps do not - have the knowledge and time to dig into the CFFI based method. The limitation is that this would get you less than 10000 python interactions per second. That is sufficient if you will be running a long running python task - and I have successfully run trivial ML programs using it, but any intensive array processing gets in the way. For this later task, there are a few emerging libraries like numcl and array-operations without SIMD (yet), and numericals using SIMD. For reasons mentioned on the readme, I recently cooked up dense-arrays. This has interchangeable backends and can also use cl-cuda. But barring that, the developer overhead of actually setting up native-CFFI ecosystem is still too high, and I'm back to py4cl/2 for tasks beyond array processing.

Currently I have put successfully this to use at dense-numericals - which I created over dense-arrays after finding CL arrays to be not that suitable, as compared to numpy or julia. Now, dense-numericals relies on passing the array pointer to C functions. However, IIUC, this runs into issues for what if the GC moves the arrays while the computation is still not done; is this worry valid? I think I ran into this while running multithreaded tests on CCL, ending up in segfaults.

Shameless unstable plug: I think it should be possible to provide type checking with a different backend that does not upgrade the types at https://github.com/digikar99/dense-arrays - the backend things are themselves unstable though.

In semi-production, ideally the problems are best represented using state diagrams, but I don't see a way to comfortably represent graphs in textual formats. The best I see is list of lists, which doesn't feel significantly better than the spaghetti code it currently is (for instance this and this - but these are just about one function each in a larger system, so not totally worth a DSL, unless there existed a defacto state-diagram DSL which everyone could be expected to know.

py4cl and cl4py rely on uiop:launch-program and python's subprocess respectively. These are portable to the extent uiop and subprocess are portable and do not require any additional installation.

If you want to use Python libs from CL, see py4cl: https://github.com/bendudson/py4cl the other way around, calling your efficient CL library from Python: https://github.com/marcoheisig/cl4py/ There might be more CL libraries than you think! https://github.com/CodyReichert/awesome-cl (or at least a project sufficiently advanced on your field to join forces ;) )

FYI you can call Python from CL: https://github.com/bendudson/py4cl and CL from Python: https://github.com/marcoheisig/cl4py/

If you don't know Emacs, see other editors: https://lispcookbook.github.io/cl-cookbook/editor-support.ht... If you want the more Smalltalk-like experience I'd go with the free LispWorks version: it has many GUI panes that allow to watch and discover the state of the program.

I personally couldn't stay long with Hylang. You won't get CL niceties: more language features, performance, standalone binaries, interactive debugger (all the niceties of an image-based development)…

I ended up using a fair bit of matplotlib through college and with colleagues. I too don't want to use python, but I also don't like throwing away its libraries, and I'm too lazy to invest in other* plotting ecosystems. In effect, I use up using matplotlib through py4cl/2.

Note that it is not my aim to replace the python ecosystem; I think that is far too lofy a goal to be of any good. My original intention was to interoperate with python through py4cl/2 or the likes, but felt that one needs a Common Lisp library for "small" operations, while "large" operations can be offloaded to python libraries through py4cl/2.

If performance is absolutely not a concern, then third option is using python libraries through py4cl/2. To put it differently, if calling python from lisp is not the bottleneck, then this is a feasible option.

I encourage people to try out Common Lisp because, unlike with Hy, you will get: speed, ability to build binaries, truly interactive image-based development (yes, more interactive than ipython), more static type checks, more language features (no closures in Hy last time I checked), language stability… To reach to Python libs, you have https://github.com/bendudson/py4cl My comparison of Python and CL: https://lisp-journey.gitlab.io/pythonvslisp/

> Not sure if such a thing already exists for CL

couple of solutions exist for this

https://github.com/bendudson/py4cl

https://github.com/pinterface/burgled-batteries

Ahh, makes sense. Well, if you ever wanna steal some of python's thunder, libpython-clj worked great for me lol. Supposedly py4cl fills a similar role in Common Lisp.