model-validation-toolkit
Model Validation Toolkit is a collection of tools to assist with validating machine learning models prior to deploying them to production and monitoring them after deployment to production. (by FINRAOS)
nannyml
nannyml: post-deployment data science in python (by NannyML)
Our great sponsors
model-validation-toolkit | nannyml | |
---|---|---|
2 | 7 | |
29 | 1,754 | |
- | 2.2% | |
7.2 | 8.8 | |
5 months ago | 8 days ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
model-validation-toolkit
Posts with mentions or reviews of model-validation-toolkit.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-01-06.
-
[P] Deepchecks: an open-source tool for high standards validations for ML models and data.
Looks very cool, thanks for sharing! I see someone else already mentioned great_expectations -- in addition, here's another library it might be interesting to integrate with: https://github.com/FINRAOS/model-validation-toolkit
-
New open source project: Model Validation Toolkit [P]
Please feel free to reach out to me here, on Gitter, or via Github issues for questions and comments!
nannyml
Posts with mentions or reviews of nannyml.
We have used some of these posts to build our list of alternatives
and similar projects.
-
Introduction to NannyML: Model Evaluation without labels
In order to try to solve this issue, NannyML was created. NannyML is an open-source Python library designed in order to make it easy to monitor drift in the distributions of our model input variables and estimate our model performance (even without labels!) thanks to the Confidence-Based Performance Estimation algorithm they developed. But first of all, why do models need to be monitored and why their performance might vary over time?
- Detecting silent model failure. NannyML estimates performance for regression and classification models using tabular data. It alerts you when and why it changed. It is the only open-source library capable of fully capturing the impact of data drift on performance.
-
[D] Data drift is not a good indicator of model performance degradation
But I may have it haha. What we propose in the blog post instead of relying solely on data drift is using performance estimation methods (eg: https://github.com/NannyML) with them you can estimate the performance of the ml model without having access to ground truth.
-
[HIRING][Full Time, Part Time, Temporary, Internship, Freelance] Data Science Intern (Remote)
Description NannyML - creators of an Open Source Python library, are looking for multiple Data Science interns to help across research, prototyping, and product. Github: https://github.com/NannyML/nannyml About Us NannyML is an Open Source Python lib …
-
What do you think about Detecting Silent ML Failure with an Open Source Python library?
If you think this could add value to your daily life, check it out here: https://github.com/NannyML/nannyml.
-
Can I estimate the impact of data drift on performance?
I found it implemented here: https://github.com/NannyML/nannyml
- Show HN: OSS Python library for detecting silent ML model failure
What are some alternatives?
When comparing model-validation-toolkit and nannyml you can also consider the following projects:
deepchecks - Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.
evidently - Evaluate and monitor ML models from validation to production. Join our Discord: https://discord.com/invite/xZjKRaNp8b