lleaves
m2cgen
Our great sponsors
lleaves | m2cgen | |
---|---|---|
4 | 8 | |
297 | 2,707 | |
- | 0.6% | |
6.7 | 0.0 | |
27 days ago | 6 months ago | |
Python | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
lleaves
- LLeaves: A LLVM-based compiler for LightGBM decision trees
-
Cold Showers
I built this decision tree (LightGBM) compiler last summer: https://github.com/siboehm/lleaves
It get's you ~10x speedups for batch predictions, more if your model is big. It's not complicated, it ended up being <1K lines of Python code. I heard a couple of stories like yours, where people had multi-node spark clusters running LightGBM, and it always amused me because by if you compiled the trees instead you could get rid of the whole cluster.
-
Tree compiler that speeds up LightGBM model inference by ~30x
In a near-future version I'll expose some of the compilation parameters, I was somewhat afraid of having an API that's too complicated deterring people who just want a no-fuzz drop-in replacement for LightGBM. But as long as I keep sane defaults and have the parameters optional it should be fine. Relevant parameters are definitely block size (needs to adjust to L1i size and tree size) as well as the LLVM codemodel (a smaller adress space increases single-batch prediction speeds but doesn't work for large models). The thread-size specific compilation I'm still looking into, it makes the API more complicated and so might not be worth it.
m2cgen
-
How to use python ML script in tauri?
Check out: https://github.com/BayesWitnesses/m2cgen
- EleutherAI announces it has become a non-profit
-
Redis as a Database — Data Migration With RedisOM, RedisGears and Redlock
Notice that I’m using random values to populate the Sentiment field. You might compute the values for your fields based on other fields or actually use an ML model to perform the transformation. E.g. you could make use of m2cgen to transform trained models to pure python code and load them in **RedisGears **to be executed in a *GearsBuilder *instance. Another option is to pull out the big guns and go straight to RedisAI.
-
Why isn’t Go used in AI/ML?
I wish that it was more common for model outputs to be converted the way bayeswitness does with mc2gen https://github.com/BayesWitnesses/m2cgen
-
Use your decision tree model in your Javascript project today with m2cgen
And that’s it! All the magic in just two lines of code. I would like to thank the authors of the m2cgen library and encourage you to try it out.
-
We use Rust for an opensource malware detection engine. It's great at detecting ransomwares and we want to share results and ideas with you.
I forgot to update the README. We just replaced RNN with xgboost that has a better f1 and is very quick, as the decision trees are translated to plain rust using m2cgen.
-
Is data science/engineering in Rust practical, does it provide any benefit over Python, and what are the best crates?
Probably, as many frameworks come with a Rust support (or there are wrappers). Some models, like decision tree, can also be automatically translated to plain Rust (in my company we use m2cgen to translate xgboost models to plain rust code).
-
Flutter Machine Learning App
These repositories on GitHub are good start I think: https://github.com/BayesWitnesses/m2cgen and https://github.com/vickylance/dart_nn
What are some alternatives?
mljar-supervised - Python package for AutoML on Tabular Data with Feature Engineering, Hyper-Parameters Tuning, Explanations and Automatic Documentation
TensorFlow.NET - .NET Standard bindings for Google's TensorFlow for developing, training and deploying Machine Learning models in C# and F#.
ngboost - Natural Gradient Boosting for Probabilistic Prediction
Synapses - A group of neural-network libraries for functional and mainstream languages
miceforest - Multiple Imputation with LightGBM in Python
R Provider - Access R packages from F#
catboost - A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.
gorse - Gorse open source recommender system engine
randomforest - Random Forest implementation in golang
gago - :four_leaf_clover: Evolutionary optimization library for Go (genetic algorithm, partical swarm optimization, differential evolution)
go-fann - Go bindings for FANN, library for artificial neural networks
sklearn - bits of sklearn ported to Go #golang