libpython-clj VS spark-nlp

It’s also worth noting that Clojure has libpython-clj (https://github.com/clj-python/libpython-clj) which offers an interface with Python from another lisp. Here are some advanced ML and dataviz examples using that lib: https://github.com/gigasquid/libpython-clj-examples.

Basically, you take a programming language and make it work on a platform that meant to be programmed using a different PL. Clojure is hosted by design - it's not Java, but can be used to program for JVM. It ain't Javascript, but can be used to target nodejs and browser; not an [official] CLR language, but you can write .Net programs. You can use Clojure to make Flutter apps with ClojureDart. You can integrate Python into Clojure with libpython-clj. Or write Clojure to target Erlang/OTP; or Rust; or R; There's even a clojure-like language for Lua - Fennel.

There's something about Clojure people like so much, they want it to work atop any platform.

https://github.com/Tensegritics/ClojureDart

https://github.com/clj-python/libpython-clj

https://github.com/clojerl/clojerl

https://github.com/clojure-rs/ClojureRS

https://github.com/scicloj/clojisr

https://fennel-lang.org

There's this project that's used a lot for taking advantage of Pythons ecosystem through Clojure JVM. https://github.com/clj-python/libpython-clj

Ahh, makes sense. Well, if you ever wanna steal some of python's thunder, libpython-clj worked great for me lol. Supposedly py4cl fills a similar role in Common Lisp.

High quality interop with any python library via libpython-clj including, but not limited to, keras, numpy, matplotlib, and pandas. This includes zero copy paths from many of those.

libpython-clj now has Java bindings...go tell all your Java friends!

I recently had the opportunity to run a workshop at ODSC East, focusing on using PySpark for Natural Language Processing (NLP). Had a great time explaining PySpark's fundamentals and exploring the Spark NLP library.

I'd like to use this transformer model in rust (because it's on the backend, because I can use data munging and it will be faster, and for other reasons). It looks like a good model! But, it doesn't compile on Apple Silicon for wierd linking issues that aren't apparent - https://github.com/guillaume-be/rust-bert/issues/338. I've spent a large part of today and yesterday attempting to find out why. The only other library that I've found for doing this kind of thing programmatically (particularly sentiment analysis) is this (https://github.com/JohnSnowLabs/spark-nlp). Some of the models look a little older, which is OK, but it does mean that I'd have to do this in another language.

Does anyone know of any sentiment analysis software that can be tuned (other than VADER - I'm looking for more along the lines of a transformer model) - like BERT, but is pretrained and can be used in Rust or Python? Otherwise I'll probably using spark-nlp and having to spin another process.

Thanks.