model-monitoring

Open-source projects categorized as model-monitoring
Edit details
Language: + Python + Jupyter Notebook + TypeScript
Topics: Machine Learning Mlops Python Monitoring data-drift
The modern identity platform for B2B SaaS
The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
workos.com
Sponsored

Top 13 model-monitoring Open-Source Projects

  • mlops-zoomcamp

    Free MLOps course from DataTalks.Club

    • Project mention: Where do I start to learn MLOPS? | /r/mlops | 2023-07-01

    There is MLOps Zoomcamp course (which shows end-to-end MLOps process with open-source MLOps tools) https://github.com/DataTalksClub/mlops-zoomcamp.

  • evidently

    Evaluate and monitor ML models from validation to production. Join our Discord: https://discord.com/invite/xZjKRaNp8b

    • Project mention: [P] Free open-source ML observability course: starts October 16 🚀 | /r/MachineLearning | 2023-10-15

    Hi everyone, I’m one of the creators of Evidently, an open-source (Apache 2.0) tool for production ML monitoring. We’ve just launched a free open course on ML observability that I wanted to share with the community.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • deepchecks

    Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

    • Project mention: Detect, Defend, Prevail: Payments Fraud Detection using ML & Deepchecks | dev.to | 2024-01-13

    Also if you have any confusion related to it. You can directly go to their discussion section in github :

  • phoenix

    AI Observability & Evaluation (by Arize-ai)

    • Project mention: First 15 Open Source Advent projects | dev.to | 2023-12-15

    11. Phoenix by Arize AI | Github | tutorial

  • nannyml

    nannyml: post-deployment data science in python

    • Project mention: Introduction to NannyML: Model Evaluation without labels | dev.to | 2023-12-15

    In order to try to solve this issue, NannyML was created. NannyML is an open-source Python library designed in order to make it easy to monitor drift in the distributions of our model input variables and estimate our model performance (even without labels!) thanks to the Confidence-Based Performance Estimation algorithm they developed. But first of all, why do models need to be monitored and why their performance might vary over time?

  • openllmetry

    Open-source observability for your LLM application, based on OpenTelemetry

    • Project mention: Show HN: You don't need to adopt new tools for LLM observability | news.ycombinator.com | 2024-02-14

    So why should it be different when the app you're building happened to be using LLMs?

    So today we're open-sourcing OpenLLMetry-JS. It's an open protocol and SDK, based on OpenTelemetry, that provides traces and metrics for LLM JS/TS applications and can be connected to any of the 15+ tools that already support OpenTelemetry. Here's the repo: https://github.com/traceloop/openllmetry-js

    A few months ago we launched the python flavor here (https://news.ycombinator.com/item?id=37843907) and we've now built a compatible one for Node.js.

    Would love to hear your thoughts and opinions!

    Check it out -

    Docs: https://www.traceloop.com/docs/openllmetry/getting-started-t...

    Github:

  • eurybia

    âš“ Eurybia monitors model drift over time and securizes model deployment with data validation

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • whitebox

    [Not Actively Maintained] Whitebox is an open source E2E ML monitoring platform with edge capabilities that plays nicely with kubernetes (by squaredev-io)

  • inferencedb

    🚀 Stream inferences of real-time ML models in production to any data lake (Experimental)

  • cyclops

    Toolkit for health AI implementation (by VectorInstitute)

    • Project mention: Call for open source devs to build the future of healthcare! | /r/github | 2023-06-30

  • openllmetry-js

    Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry

    • Project mention: FLaNK Stack Weekly 19 Feb 2024 | dev.to | 2024-02-19

  • bitbeast

    Experiments with Model Training, Deployment & Monitoring

  • reco-model-monitoring

    fastapi + prometheus + grafana 💣

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

model-monitoring related posts

Index

What are some of the best open-source model-monitoring projects? This list will help you:

Project Stars
1 mlops-zoomcamp 8,735
2 evidently 4,619
3 deepchecks 3,338
4 phoenix 2,586
5 nannyml 1,754
6 openllmetry 1,195
7 eurybia 203
8 whitebox 181
9 inferencedb 77
10 cyclops 65
11 openllmetry-js 33
12 bitbeast 31
13 reco-model-monitoring 3

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com