nannyml
deep-significance
Our great sponsors
nannyml | deep-significance | |
---|---|---|
7 | 6 | |
1,754 | 315 | |
2.2% | - | |
8.8 | 4.0 | |
6 days ago | 6 months ago | |
Python | Python | |
Apache License 2.0 | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
nannyml
-
Introduction to NannyML: Model Evaluation without labels
In order to try to solve this issue, NannyML was created. NannyML is an open-source Python library designed in order to make it easy to monitor drift in the distributions of our model input variables and estimate our model performance (even without labels!) thanks to the Confidence-Based Performance Estimation algorithm they developed. But first of all, why do models need to be monitored and why their performance might vary over time?
- Detecting silent model failure. NannyML estimates performance for regression and classification models using tabular data. It alerts you when and why it changed. It is the only open-source library capable of fully capturing the impact of data drift on performance.
-
[D] Data drift is not a good indicator of model performance degradation
But I may have it haha. What we propose in the blog post instead of relying solely on data drift is using performance estimation methods (eg: https://github.com/NannyML) with them you can estimate the performance of the ml model without having access to ground truth.
-
[HIRING][Full Time, Part Time, Temporary, Internship, Freelance] Data Science Intern (Remote)
Description NannyML - creators of an Open Source Python library, are looking for multiple Data Science interns to help across research, prototyping, and product. Github: https://github.com/NannyML/nannyml About Us NannyML is an Open Source Python lib …
-
What do you think about Detecting Silent ML Failure with an Open Source Python library?
If you think this could add value to your daily life, check it out here: https://github.com/NannyML/nannyml.
-
Can I estimate the impact of data drift on performance?
I found it implemented here: https://github.com/NannyML/nannyml
- Show HN: OSS Python library for detecting silent ML model failure
deep-significance
- [P] deep-significance: Enabling easy statistical significance testing for deep neural networks
-
[D] Statistical Significance in Deep RL Papers: What is going on?
Because I was so frustrated by this topics as well, I actually reimplemented and packaged a test specifically for NNs and gave it a lot of documentation in the hope of lowering the entry barrier as much as possible https://github.com/Kaleidophon/deep-significance
- deep-significance: Easy and Better Significance Testing for Deep Neural Networks
- [P] deep-significance: Easy and Better Significance Testing for Deep Neural Networks
- [Project] deep-significance: Easy and Better Significance Testing for Deep Neural Networks (link below)
- [P] deep-significance: Easy and Better Significance Testing for Deep Neural Networks (link below)
What are some alternatives?
evidently - Evaluate and monitor ML models from validation to production. Join our Discord: https://discord.com/invite/xZjKRaNp8b
Note - Easily implement parallel training and distributed training. Machine learning library. Note.neuralnetwork.tf package include Llama2, Llama3, CLIP, ViT, ConvNeXt, SwiftFormer, etc, these models built with Note are compatible with TensorFlow and can be trained with TensorFlow.
cuttle-cli - Cuttle automates the transformation of your Python notebook into deployment-ready projects (API, ML pipeline, or just a Python script)
ludwig - Low-code framework for building custom LLMs, neural networks, and other AI models
barfi - Python Flow Based Programming environment that provides a graphical programming environment.
horovod - Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet. [Moved to: https://github.com/horovod/horovod]
ydata-profiling - 1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
openrec - OpenRec is an open-source and modular library for neural network-inspired recommendation algorithms
eurybia - ⚓ Eurybia monitors model drift over time and securizes model deployment with data validation
horovod - Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
cyclops - Toolkit for health AI implementation
clearml - ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution