clj-djl VS spark-nlp

You may wish to add clj-djl to your list.

Regarding Tensorflow: As far as I understand, it is accessible through DJL, which has a Clojure wrapper (work in progress): clj-djl. (But I haven't tried it.)

We have currently three 4 plugins into existing ml libraries: https://github.com/scicloj/scicloj.ml.smile https://github.com/scicloj/scicloj.ml.xgboost https://github.com/scicloj/sklearn-clj https://github.com/scicloj/clj-djl

I recently had the opportunity to run a workshop at ODSC East, focusing on using PySpark for Natural Language Processing (NLP). Had a great time explaining PySpark's fundamentals and exploring the Spark NLP library.

I'd like to use this transformer model in rust (because it's on the backend, because I can use data munging and it will be faster, and for other reasons). It looks like a good model! But, it doesn't compile on Apple Silicon for wierd linking issues that aren't apparent - https://github.com/guillaume-be/rust-bert/issues/338. I've spent a large part of today and yesterday attempting to find out why. The only other library that I've found for doing this kind of thing programmatically (particularly sentiment analysis) is this (https://github.com/JohnSnowLabs/spark-nlp). Some of the models look a little older, which is OK, but it does mean that I'd have to do this in another language.

Does anyone know of any sentiment analysis software that can be tuned (other than VADER - I'm looking for more along the lines of a transformer model) - like BERT, but is pretrained and can be used in Rust or Python? Otherwise I'll probably using spark-nlp and having to spin another process.

Thanks.