Our great sponsors
-
minimal-ml-template
A very minimal ml project template that uses HF transformers and wandb to train a simple NN and evaluate it, in a stateless manner compatible with Spot instances kubernetes workflows
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
kubeproject
A set of tools built to simplify daily driving of cloud resources for individual VM access, Kubernetes batch jobs and miscellaneous useful functionality related to cloud-based ML research
I am here today to share with you all a minimal ml project template that I've recently built, which can be found at https://github.com/AntreasAntoniou/minimal-ml-template/. I became increasingly annoyed at how there weren't any repos out there that provided **stateless** ML project templates, which are absolutely necessary when using kubernetes on spot instances, and I decided to build one. By stateless I mean a repo that by default can store model weights in a remote repo and then download them to continue from where it left off if the previous machine dies. The result was this repository.
It provides mature configuration support via [Hydra-Zen](https://github.com/mit-ll-responsible-ai/hydra-zen) and automates configuration generation via [decorators](https://github.com/BayesWatch/minimal-ml-template/blob/af387e59472ea67552b4bb8972b39fe95952dd8a/mlproject/decorators.py#L10) implemented in this repo.
It provides plug and play functionality that allows easy hyperparameter search on Kubernetes clusters using [BWatchCompute](https://github.com/BayesWatch/bwatchcompute) and some readily available scripts and yaml templates.