java
nitro
Our great sponsors
java | nitro | |
---|---|---|
6 | 7 | |
761 | 1,572 | |
5.0% | 19.1% | |
8.7 | 9.8 | |
13 days ago | 6 days ago | |
Java | C++ | |
Apache License 2.0 | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
java
- FLaNK Weekly 08 Jan 2024
-
What libraries do you use for machine learning and data visualizing in scala?
There are Java bindings for TensorFlow, but that's quite low level. I tried to see if I can get some Keras API for Scala, but I'm no expert and haven't had enough time to invest in this, so it's stuck in alpha. Maybe I develop it slow burning over the next year. A bit envious that Kotlin has a Keras-like library.
- Choosing Java as your language for a Machine Learning project
-
TensorFlow introduction that works with Java
Hope this is not too late to answer your question. In theory there are no official Java tutorials for Tensorflow 2. The Java implementation is still under development at https://github.com/tensorflow/java
-
[D] Java vs Python for Machine learning
To give a contrasting perspective, I think the Java ecosystem is much better suited for many data science tasks, and has a growing and well-maintained set of libraries for general purpose machine learning. I won't list them all, but TF-Java, DJL et al. have implementations of many modern architectures and there are a number of excellent libraries (CoreNLP, Lucene et al.) for working with text.
-
Can we use the keras model in another programming language, such as java or etcs?
Here's the latest java git repo https://github.com/tensorflow/java
nitro
-
Ollama Python and JavaScript Libraries
I'd like to see a comparison to nitro https://github.com/janhq/nitro which has been fantastic for running a local LLM.
- FLaNK Weekly 08 Jan 2024
-
Nitro: A fast, lightweight 3MB inference server with OpenAI-Compatible API
Look... I appreciate a cool project, but this is probably not a good idea.
> Built on top of the cutting-edge inference library llama.cpp, modified to be production ready.
It's not. It's literally just llama.cpp -> https://github.com/janhq/nitro/blob/main/.gitmodules
Llama.cpp makes no pretense at being a robust safe network ready library; it's a high performance library.
You've made no changes to llama.cpp here; you're just calling the llama.cpp API directly from your drogon app.
Hm.
...
Look... that's interesting, but, honestly, I know there's this wave of "C++ is back!" stuff going on, but building network applications in C++ is very tricky to do right, and while this is cool, I'm not sure 'llama.cpp is in c++ because it needs to be fast' is a good reason to go 'so lets build a network server in c++ too!'.
I mean, I guess you could argue that since llama.cpp is a C++ application, it's fair for them to offer their own server example with an openai compatible API (which you can read about here: https://github.com/ggerganov/llama.cpp/issues/4216, https://github.com/ggerganov/llama.cpp/blob/master/examples/...).
...but a production ready application?
I wrote a rust binding to llama.cpp and my conclusion was that llama.cpp is pretty bleeding edge software, and bluntly, you should process isolate it from anything you really care about, if you want to avoid undefined behavior after long running inference sequences; because it updates very often, and often breaks. Those breaks are usually UB. It does not have a 'stable' version.
Further more, when you run large models and run out of memory, C++ applications are notoriously unreliable in their 'handle OOM' behaviour.
Soo.... I know there's something fun here, but really... unless you had a really really compelling reason to need to write your server software in c++ (and I see no compelling reason here), I'm curious why you would?
It seems enormously risky.
The quality of this code is 'fun', not 'production ready'.
- Apple Silicon Llama 7B running in docker?
- Is there any LLM that can be installed with out python
What are some alternatives?
Deep Java Library (DJL) - An Engine-Agnostic Deep Learning Framework in Java
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
JNA - Java Native Access
bionic-gpt - BionicGPT is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality
CoreNLP - CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.
nnl - a low-latency and high-performance inference engine for large models on low-memory GPU platform.
Zeppelin - Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.
csvlens - Command line csv viewer
tensorflow-keras-scala - Scala-based Keras API for the Java bindings to TensorFlow. Mirror of https://codeberg.org/sciss/tensorflow-keras-scala
Tribuo - Tribuo - A Java machine learning library
java-models - Models in Java
hyperfine - A command-line benchmarking tool