Serving Python Machine Learning Models With Ease

• MLServer

  4 582 9.5 Python

  An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more

  

Ever trained a new model and just wanted to use it through an API straight away? Sometimes you don't want to bother writing Flask code or containerizing your model and running it in Docker. If that sounds like you, you definitely want to check out MLServer. It's a python based inference server that recently went GA and what's really neat about it is that it's a highly-performant server designed for production environments too. That means that, by serving models locally, you are running in the exact same environment as they will be in when they get to production.

• mlserver-example

  1 0 0.0 Python

  An example showing how to use multi model serving with MLServer

All of the code from this example can be found here.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• MLflow

  55 17,234 9.9 Python

  Open source platform for the machine learning lifecycle

  

For MLFlow users you can now serve models directly in MLFlow using MLServer and if you're a Kubernetes user you should definitely check out Seldon Core - an open source tool that deploys models to Kubernetes (it uses MLServer under the covers).

• kubernetes

  657 106,778 10.0 Go

  Production-Grade Container Scheduling and Management

  

For MLFlow users you can now serve models directly in MLFlow using MLServer and if you're a Kubernetes user you should definitely check out Seldon Core - an open source tool that deploys models to Kubernetes (it uses MLServer under the covers).

Suggest a related project