barfi VS nannyml

NannyML is a Python library specialized in post-deployment monitoring and maintenance of machine learning (ML) models. It enables data scientists to detect and address silent model failure, estimate model performance without immediate ground truth data, and identify data drift that might be responsible for performance degradation.

In machine learning projects, it is difficult to estimate whether improvement in model metrics will result in a positive change in business value. Furthermore, a model’s performance can change with time. NannyML is an open source Python library that focuses on production monitoring and allows users to detect drifts (data and label), check data quality, estimate post-deployment model performance, and intelligently generate an alert if the drift is likely to impact model performance.

In order to try to solve this issue, NannyML was created. NannyML is an open-source Python library designed in order to make it easy to monitor drift in the distributions of our model input variables and estimate our model performance (even without labels!) thanks to the Confidence-Based Performance Estimation algorithm they developed. But first of all, why do models need to be monitored and why their performance might vary over time?

But I may have it haha. What we propose in the blog post instead of relying solely on data drift is using performance estimation methods (eg: https://github.com/NannyML) with them you can estimate the performance of the ml model without having access to ground truth.

Description NannyML - creators of an Open Source Python library, are looking for multiple Data Science interns to help across research, prototyping, and product. Github: https://github.com/NannyML/nannyml About Us NannyML is an Open Source Python lib …

If you think this could add value to your daily life, check it out here: https://github.com/NannyML/nannyml.

I found it implemented here: https://github.com/NannyML/nannyml