MLflow VS tensorflow

I have an old labmate who uses a similar setup with MLFlow and can endorse it.

MLflow - an open-source platform for managing your ML lifecycle. What’s great is that they also support popular Python libraries like TensorFlow, PyTorch, scikit-learn, and R.

even for people who don't have an ML background there's now a lot of very fully-featured model deployment environments that allow self-hosting (kubeflow has a good self-hosting option, as do mlflow and metaflow), handle most of the complicated stuff involved in just deploying an individual model, and work pretty well off the shelf.

Here, we’ll implement the experimentation workflow using DagsHub, Google Colab, MLflow, and data version control (DVC). We’ll focus on how to do this without diving deep into the technicalities of building or designing a workbench from scratch. Going that route might increase the complexity involved, especially if you are in the early stages of understanding ML workflows, just working on a small project, or trying to implement a proof of concept.

MLflow

I am seeing growing demand for MLflow (https://mlflow.org/) and I am seeing a lot of people looking at Databricks as commercial offering for MLflow. Alternatively, some popele are implementing something like Managing your Machine Learning lifecycle with MLflow. Therefore, I think this was on my wish list last year, but I really hope AWS announce a Managed MLFlow Service. I know version 2.X is too new but at least 1.X would be great start.

🔗 https://mlflow.org

She made mentions to ML-Ops and MLFlow including Vertex AI the GCP implementation. I will post the video as soon as it is available. In the meantime, you can enjoy any other talk from Nerea Luis

In our last post, we discussed the importance of tracking Machine Learning experiments, metrics and parameters. We also showed how easy it is to get started in these topics by leveraging the power of MLflow (for those who are not aware, MLflow is currently the de-facto standard platform for machine learning experiment and model management).

In their report they show the 10 projects with the biggest number of contributors. The first one is microsoft/vscode with 19.8K contributors in 2022 and the 10th place is tensorflow/tensorflow with 4.4K contributors. That's really nice, but my guess is that most repositories have very few contributors.

TensorFlow - This one needs no introduction. It’s widely-used and it has several tools and community resources for training and deploying ML/DL models. This one is JS based and I’m not too familiar with JS except that I know a lot of people use it for web development. What I like about it is that it’s used for speech and image recognition. And one of my favorites are text summarization.

If you mean StableHLO, then it has an MLIR dialect: https://github.com/openxla/stablehlo/blob/main/stablehlo/dia....

In the StableHLO spec, we are talking about this in more abstract terms - "StableHLO opset" - to be able to unambiguously reason about the semantics of StableHLO programs. However, in practice the StableHLO dialect is the primary implementation of the opset at the moment.

I wrote "primary implementation" because e.g. there is also ongoing work on adding StableHLO support to the TFLite flatbuffer schema: https://github.com/tensorflow/tensorflow/blob/master/tensorf.... Having an abstract notion of the StableHLO opset enables us to have a source of truth that all the implementations correspond to.

Install necessary dependencies: sudo apt-get update sudo apt-get upgrade sudo apt-get install build-essential git libhdf5-dev python-dev python-pip python-numpy python-wheel python-mock python-mockito python-pytest python-six python-h5py sudo pip install --upgrade pip sudo pip install setuptools Install Bazel: sudo apt-get install pkg-config zip g++ zlib1g-dev unzip python wget https://github.com/bazelbuild/bazel/releases/download/0.26.1/bazel-0.26.1-installer-linux-x86_64.sh chmod +x bazel-0.26.1-installer-linux-x86_64.sh ./bazel-0.26.1-installer-linux-x86_64.sh --user Clone TensorFlow 1.x: git clone https://github.com/tensorflow/tensorflow.git cd tensorflow git checkout r1.15 Configure TensorFlow: ./configure Build TensorFlow: bazel build --config=opt --local_resources=1024,1.0,1.0 --verbose_failures //tensorflow/tools/pip_package:build_pip_package bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg sudo pip install /tmp/tensorflow_pkg/tensorflow-1.15.0-cp27-cp27mu-linux_armv7l.whl Set up swap memory: sudo fallocate -l 4G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile Test TensorFlow: python -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

Heck, I don't know how involved you feel like getting, but providing Python 3.11 support for tensorflow would be super awesome.

Uite ce scrie la limbaj aici: https://github.com/tensorflow/tensorflow

I am pretty sure it used to be 2 spaces as well. Some public repositories such as https://github.com/tensorflow/tensorflow/tree/master/tensorf... appear to use 2 space indent throughout.

I think all these will be covered by the "be consistent" clause, and whoever made the first commit decides the style.

Converting the images to a tensor: Deep learning models work with tensors, so the images should be converted to tensors. This can be done using the to_tensor function from the PyTorch library or convert_to_tensor from the Tensorflow library.

Tensorflow 238k contributors

C++ is actually used in machine learning. More than 60% of TensorFlow code is in C++: https://github.com/tensorflow/tensorflow. With high level configs and prototyping is done in python.