MLServer VS conductor

You've already mentioned Seldon Core which is well worth looking at but if you're just after the raw multi-model serving aspect rather than a fully-fledged deployment framework you should maybe take a look at the individual inference servers: Triton Inference Server and MLServer both support multi-model serving for a wide variety of frameworks (and custom python models). MLServer might be a better option as it has an MLFlow runtime but only you will be able to decide that. There also might be other inference servers that do MMS that I'm not aware of.

Ever trained a new model and just wanted to use it through an API straight away? Sometimes you don't want to bother writing Flask code or containerizing your model and running it in Docker. If that sounds like you, you definitely want to check out MLServer. It's a python based inference server that recently went GA and what's really neat about it is that it's a highly-performant server designed for production environments too. That means that, by serving models locally, you are running in the exact same environment as they will be in when they get to production.

Seldon | Multiple positions | London/Cambridge UK | Onsite/Remote | Full time | seldon.io

At Seldon we are building industry leading solutions for deploying, monitoring, and explaining machine learning models. We are an open-core company with several successful open source projects like:

* https://github.com/SeldonIO/seldon-core

* https://github.com/SeldonIO/mlserver

* https://github.com/SeldonIO/alibi

* https://github.com/SeldonIO/alibi-detect

* https://github.com/SeldonIO/tempo

We are hiring for a range of positions, including software engineers(go, k8s), ml engineers (python, go), frontend engineers (js), UX designer, and product managers. All open positions can be found at https://www.seldon.io/careers/

We celebrated a remarkable milestone in September when the Netflix Conductor GitHub repository reached 10k stars. It was a momentous achievement for our DevRel team. Just a month later, we're thrilled to announce that we've surpassed 12k stars! ⭐🎉

Also, don’t forget to give us a ⭐ on our Netflix Conductor repo.

One of my favorite workflow engines that has a really simple way to do things was not listed here, so I'll call it out - Netflix Conductor (https://github.com/Netflix/conductor).

Its capabilities comes to light when you model really complex workflows and one real value is how its all very visual not just during modeling but when running it. The history remains visible and you can even see how the whole flow evolved.

Yet another significant milestone on our journey: we've proudly reached the 10,000-star mark on our Netflix Conductor GitHub repository! 🌟

Give something like https://github.com/Netflix/conductor a try to solve this -- makes it very easy to do what you are trying to achieve.

Checkout Conductor https://github.com/Netflix/conductor which is far more scalable and easy on the resources with its own Celery like queues. Fully supports writing task workers in python: