Top 5 Python ml-platform Projects
:rocket: Build and manage real-life data science projects with ease!Project mention: AWS Summit 2022 Australia and New Zealand - Day 2, AI/ML Edition | dev.to | 2022-05-20
As a result of their new DS framework (based on a Metaflow - a DS framework built at Netflix and AWS SageMaker Pipelines), they were able to free up their DS resources so that Software Developers were now trained and equipped to tackle their normal DS projects, at a ratio of 70% DS/ML work was now completed by developers. This leaves the 30% meatier and more difficult problems for the Data Scientists to tackle.
🔥 A tool for visualizing and tracking your machine learning experiments. This repo contains the CLI and Python API. (by wandb)Project mention: Roadmap for learning MLOps (for DevOps engineers) | reddit.com/r/mlops | 2022-01-31
I want to take a look at tools like https://wandb.ai/ and they would integrate into some of the pipelines I'm playing with.
Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
The Unified Model Serving Framework 🍱Project mention: How to Build a Machine Learning Demo in 2022 | dev.to | 2022-01-16
Using a general-purpose framework such as FastAPI involves writing a lot of boilerplate code just to get your API endpoint up and running. If deploying a model for a demo is the only thing you are interested in and you do not mind losing some flexibility, you might want to use a specialized serving framework instead. One example is BentoML, which will allow you to get an optimized serving endpoint for your model up and running much faster and with less overhead than a generic web framework. Framework-specific serving solutions such as Tensorflow Serving and TorchServe typically offer optimized performance but can only be used to serve models trained using Tensorflow or PyTorch, respectively.
Determined: Deep Learning Training PlatformProject mention: How to train large deep learning models as a startup | news.ycombinator.com | 2021-10-07
Check out Determined https://github.com/determined-ai/determined to help manage this kind of work at scale: Determined leverages Horovod under the hood, automatically manages cloud resources and can get you up on spot instances, T4's, etc. and will work on your local cluster as well. Gives you additional features like experiment management, scheduling, profiling, model registry, advanced hyperparameter tuning, etc.
Full disclosure: I'm a founder of the project.
:ledger: Experiment tracking tool and model registryProject mention: What are the differences between MLflow and neptune? | reddit.com/r/mlops | 2022-03-21
Hello u/MLBoi_TM! I was wondering: The pros/cons you've listed, is this comparing Managed MLflow <> neptune.ai or the OSS MLflow compenent <> neptune.ai?
Python ml-platform related posts
[D] Software stack to replicate Azure ML / Google Auto ML on premise
2 projects | reddit.com/r/MachineLearning | 3 Feb 2021
What are some of the best open-source ml-platform projects in Python? This list will help you:
Are you hiring? Post a new remote job listing for free.