great_expectations VS seldon-core

Great Expectations (GE) is an open-source data validation tool that helps ensure data quality.

GE is arguably the most well known OSS alternative to Soda Core. The third option is deequ, originally developed and released in OSS by AWS. Our community has told us that Soda Core is different because it’s easy to get going and embed into data pipelines. And it also allows some of the check authoring work to be moved to other members of the data team. I'm sure there are also scenarios where Soda Core is not the best option. For example, when you only use Pandas dataframes or develop in Scala.

great_expectations: https://github.com/great-expectations/great_expectations

For example I just found "Great Expectations" and "Kedro", "Flyte" and I was wondering at which point in time and project complexity should we choose one of these tools instead of the ancient cave man way?

Testing: GreatExpectations

Ingestion / ETL: Airbyte, Singer, Jitsu Transformation: dbt Orchestration: Airflow, Dagster Testing: GreatExpectations Observability: Monosi Reverse ETL: Grouparoo, Castled Visualization: Lightdash, Superset

ZenML is an extensible, open-source MLOps framework to create production-ready machine learning pipelines. Built for data scientists, it has a simple, flexible syntax, is cloud- and tool-agnostic, and has interfaces/abstractions that are catered towards ML workflows. Seldon Core is a production grade open source model serving platform. It packs a wide range of features built around deploying models to REST/GRPC microservices that include monitoring and logging, model explainers, outlier detectors and various continuous deployment strategies such as A/B testing, canary deployments and more.

I am using BentoML to build the docker container for a BERT model, and then deploy that using Seldon on GKE. The model's REST API endpoint works fine. at terms of compatibility with Seldon, the metrics are being scraped by Prometheus and visualized on Grafana. The only Seldon component that doesn't appear to be working is the request logging, which I have working for other applications that were deployed on Seldon. I am using the elastic stack from here. From my understanding, request logging should still be compatible and the ⠀only lost functionality should be Seldon's model metadata. Any insight on how to get the centralized request logging working? No errors were shown; it's just that the logs aren't being captured and sent to ElasticSearch. Anyone have any success using BentoML with Seldon and not losing any of Seldon's features?

While tools in the model experimentation space normally include diagnostic charts on a model's performance, there are also specialised solutions that help ensure that the deployed model continues to perform as they are expected to. This includes the likes of seldon-core, why-labs and fiddler.ai.

Seldon | Multiple positions | London/Cambridge UK | Onsite/Remote | Full time | seldon.io

At Seldon we are building industry leading solutions for deploying, monitoring, and explaining machine learning models. We are an open-core company with several successful open source projects like:

* https://github.com/SeldonIO/seldon-core

* https://github.com/SeldonIO/mlserver

* https://github.com/SeldonIO/alibi

* https://github.com/SeldonIO/alibi-detect

* https://github.com/SeldonIO/tempo

We are hiring for a range of positions, including software engineers(go, k8s), ml engineers (python, go), frontend engineers (js), UX designer, and product managers. All open positions can be found at https://www.seldon.io/careers/

Also note our github repo has a link to our slack where you can ask active users: https://github.com/SeldonIO/seldon-core

If you want to serve your model at scale, with a bunch of production features you should have a look at the open-source framework Seldon Core. It does what you're asking for plus a bunch of other cool stuff like routing, logging and monitoring.

Seldon ticks most of those boxes if you already have some experience with kubernetes. You can set up a/b tests, do payload logging to elastic and then do monitoring on top of that, and it has drift detection and model explainer modules too. Idk about great expectations integration, but you could probably do something with a custom transformer module as part of the inference graph.