MLflow VS examples

Platforms such as MLflow monitor the development stages of machine learning models. In parallel, Data Version Control (DVC) brings version control system-like functions to the realm of data sets and models.

MLFlow - http://mlflow.org

MLflow:

In search for a tool that needs comparable configuration I looked into mlflow and found this. https://github.com/mlflow/mlflow/blob/master/mlflow/environment_variables.py There they define a class _EnvironmentVariable and create many objects out of it, for any variable they need. The get method of this class is in principle a decorated os.getenv. Maybe that is something I can take as orientation.

I have been using DVC and MLflow since then DVC had only data tracking and MLflow only model tracking. I can say both are awesome now and maybe the only factor I would like to mention is that IMO, MLflow is a bit harder to learn while DVC is just a git practically.

Dumb question, how does this lib compare to other libs like MLFlow, https://mlflow.org/?

While Transformers already includes integration with MLflow, users still have to provide their own MLflow server, either locally or on a Cloud provider. And that can be a bit of a pain.

I have an old labmate who uses a similar setup with MLFlow and can endorse it.

MLflow - an open-source platform for managing your ML lifecycle. What’s great is that they also support popular Python libraries like TensorFlow, PyTorch, scikit-learn, and R.

AI's Open Embrace Artificial intelligence (AI) and machine learning (ML) are increasingly leveraging open-source frameworks like TensorFlow [https://www.tensorflow.org/] and PyTorch [https://pytorch.org/]. This democratization of AI tools is driving innovation and lowering entry barriers across industries.

Which label applies to a tool sometimes depends on what you do with it. For example, PyTorch or TensorFlow can be called a library, a toolkit, or a machine-learning framework.

TensorFlow: An open-source machine learning framework for high-performance numerical computations, especially well-suited for deep learning.

The tool used to build the model per se was TensorFlow, a very powerful and end-to-end open source platform for machine learning with a rich ecosystem of tools. And in order to to create the needed script using TensorFlow Jupyter Notebook was used, which is a web-based interactive computing platform.

10. TensorFlow - Make Machine Learning Work for You 🤖

#2 TensorFlow

Machine learning is the process of using algorithms and statistical models to enable computers to learn from data. There are many tools and libraries available for machine learning, but the most popular by far is TensorFlow. TensorFlow is an open-source platform for machine learning developed by Google. It has over 176k stars on Github and is used by companies such as Airbnb and Intel.

Tensorflow is Google's "end-to-end machine learning platform". It's a framework to manage the whole lifecycle of a Machine Learning (and AI) project, from data preparation to production deployment. Remember the math stuff we talked about in the last section? Tensorflow manages that in addition to a lot of other stuff. Its core API is written for Python and you have to know your math just a little bit in order to play with it. It's more for deep learning models (neural networks) and has a lot of already implemented "layers" for you to use in your network. You can prepare data (images included with the option of image augmentation for small data sets ... yay! 😃), experiment with different model architectures, tune the model's hyperparameters (a fancy name for model configs), train, validate and test your models and monitor your models in production. It's a great framework, but it is not an easy one to learn, especially if you don't like math that much!

Click to Learn more...