Deeplearning4j VS Deep Java Library (DJL)

Have you considered migrating off of Python to just using JVM ML libraries then? I hear good things about Deeplearning4j, but there's quite a few.

I've gone to the linux workflow as directed in the docs and reconstructed the maven command line:

DL4J

Hey folks, just letting you know we see this thread and I appreciate you guys running these benchmarks. I'm not seeing any of your posts on our forums. I think I saw a notification from our examples but we do not actually monitor that. Please use: https://community.konduit.ai/ or at least the main repo dl4j issues: https://github.com/eclipse/deeplearning4j/issues and you'll get a lot more visibility. Thanks!

Also, the website is now redirected to: https://deeplearning4j.konduit.ai/

On top of this several popular projects have been built. This includes tensorflow-java and our project eclipse deeplearning4j: https://github.com/eclipse/deeplearning4j

Nd4j is actively developed. The latest commit was 6 hours ago. Nd4j is part of deeplearning4j which is now owned by eclipse (but the main contributors are from a company) https://github.com/eclipse/deeplearning4j/tree/master/nd4j

It seems to have been picked up by Eclipse and there is also Oracle Labs' Tribuo and Deep Java Library. All seem active, but I don't know much about any of them. I agree it's probably best to follow the community and use a more popular tool like PyTorch.

Although it may be a bit more work, you can do both machine learning and AI in Java. If you are doing deep learning, you can use DeepJavaLibrary (I do work on this one at Amazon). If you are looking for other ML algorithms, I have seen Smile, Tribuo, or some around Spark.

Image preprocessing I know less about, but tokenization is something I've dealt with a bunch. There are a few options, either push the tokenizer into the ONNX model and use MS's ONNX Runtime extensions (we've used this when working with sentencepiece tokenizers), port the tokenizer entirely to Java (we did this for BERT), or use a sentencepiece or HF tokenizers wrapper directly (e.g. Amazon's DJL did this - HF, sentencepiece).

https://djl.ai/ seems very promising. I've played around with it quite a bit, not in real production though. It's a very well documented (https://d2l.djl.ai/) and active project, with Amazon working on it.

Here is link number 1 - Previous text "DJL"

To give a contrasting perspective, I think the Java ecosystem is much better suited for many data science tasks, and has a growing and well-maintained set of libraries for general purpose machine learning. I won't list them all, but TF-Java, DJL et al. have implementations of many modern architectures and there are a number of excellent libraries (CoreNLP, Lucene et al.) for working with text.

I think DJL also use use it for their tutorials - https://docs.djl.ai/jupyter/tutorial/01_create_your_first_network.html.

AWS Deep Learning more deep learning.