CoreNLP VS Deep Java Library (DJL)

The Stanford CoreNLP Model

If your looking at spacy have a look at Apache OpenNLP and Core NLP.

Tokenize with NLTK, SpaCy or CoreNLP

It should not take nearly that long. However, again I must recommend you take this conversation to github

Stanford has a great program and the Stanford NLP Group maintains CoreNLP which I have used before.

So why not use the Stanford CoreNLP library?

If you need NER, there’s no need to implement it yourself. There are several popular libraries that can do this for you nowadays. Five of these libraries, Stanford CoreNLP, NLTK, OpenNLP, SpaCy, and GATE, were already mentioned in the title.

Check something like this out to start: https://stanfordnlp.github.io/CoreNLP/

You don't actually even need to learn how to do deep learning if you're doing something fairly basic, which it sounds like you are. There are a lot of good tools you can use basically straight out of the box for something like this. Check out https://huggingface.co/course/chapter1, https://course.spacy.io/en/, https://guide.allennlp.org/ and https://www.nltk.org/book/. If java's more your thing, add https://stanfordnlp.github.io/CoreNLP/ to the list.

To give a contrasting perspective, I think the Java ecosystem is much better suited for many data science tasks, and has a growing and well-maintained set of libraries for general purpose machine learning. I won't list them all, but TF-Java, DJL et al. have implementations of many modern architectures and there are a number of excellent libraries (CoreNLP, Lucene et al.) for working with text.

It seems to have been picked up by Eclipse and there is also Oracle Labs' Tribuo and Deep Java Library. All seem active, but I don't know much about any of them. I agree it's probably best to follow the community and use a more popular tool like PyTorch.

Although it may be a bit more work, you can do both machine learning and AI in Java. If you are doing deep learning, you can use DeepJavaLibrary (I do work on this one at Amazon). If you are looking for other ML algorithms, I have seen Smile, Tribuo, or some around Spark.

Image preprocessing I know less about, but tokenization is something I've dealt with a bunch. There are a few options, either push the tokenizer into the ONNX model and use MS's ONNX Runtime extensions (we've used this when working with sentencepiece tokenizers), port the tokenizer entirely to Java (we did this for BERT), or use a sentencepiece or HF tokenizers wrapper directly (e.g. Amazon's DJL did this - HF, sentencepiece).

https://djl.ai/ seems very promising. I've played around with it quite a bit, not in real production though. It's a very well documented (https://d2l.djl.ai/) and active project, with Amazon working on it.

Here is link number 1 - Previous text "DJL"

To give a contrasting perspective, I think the Java ecosystem is much better suited for many data science tasks, and has a growing and well-maintained set of libraries for general purpose machine learning. I won't list them all, but TF-Java, DJL et al. have implementations of many modern architectures and there are a number of excellent libraries (CoreNLP, Lucene et al.) for working with text.

I think DJL also use use it for their tutorials - https://docs.djl.ai/jupyter/tutorial/01_create_your_first_network.html.

AWS Deep Learning more deep learning.