guildai VS wandb

Experiment tracking in DvC is implemented using git to store snapshots of a project and related artifacts. You might take a look at Guild AI's support for DvC, which is tightly integrated with DvC stages. You can run any of the stages defined for a project and you get a properly isolated run (each run is a project copy to ensure that you're not corrupting the run if you modify files while it's running - as well as properly supporting concurrent runs). Once you have runs in Guild, you can use any number of tools to study, compare, export, etc.

I built an experiment tracking tool (Guild AI) that focuses on code/model reuse and so this question is dear to my heart :) Best of luck!

I'm not aware of experiment tracking in Jupyter notebooks themselves. Guild AI is able to run notebooks as experiments however.

Disclosure - I'm the author of Guild AI so take this for the biased opinion that it is.

I'm the author of Guild AI (open source experiment tracking). For some time now Guild users have asked for DvC support. This is now available as a pre-release.

Guild AI now has support for running DvC stages as experiments. DvC uses git under the covers to manage project state for each experiment, along with the experiment results. Guild doesn't touch your git repo and instead copies your project source to a new run directory. This ensures that you have a correct record of your experiment without churning your project state.

Guild.ai - https://guild.ai/

Use guild and pytorch-lightning. Make it easy for new contributors to get your data by using dvc as a data access tool.

Weights & Biases — The developer-first MLOps platform. Build better models faster with experiment tracking, dataset versioning, and model management. Free tier for personal projects only, with 100 GB of storage included.

HuggingFace and wandb.ai (both competitors of OpenAI) both also have "do own research"

Wandb is the best! https://wandb.ai/

To track stuff I've used wandb.ai in a company in the past, as someone else pointed out. Regarding metrics... This is really specific to your domain, and it is such a broad question. You could count color pixels, the distribution of intensity histograms, etc etc.

Step 1: The colab notebook uses wandb.ai, so you need to register for a wandb.ai account beforehand if you want to use the colab notebook. After registering you need to go to your homepage and copy the API key and paste/keep it somewhere.

I want to take a look at tools like https://wandb.ai/ and they would integrate into some of the pipelines I'm playing with.

does current cgi and ai tech can bring back leslie nielsen? might use unreal engine and https://www.resemble.ai/ or https://wandb.ai/?

I'm currently working for a MLOps company so I'm heavily using their tools (Weights & Biases) but I've used custom C++ for deployment, Pytorch + fastai for quick experimentation, Weights & Biases for experiment tracking, hyper-parameter tuning + model versioning (hence why I went to work for them), custom database + data pipeline, HoloViz for data visualisation (really nice dashboarding tool), Jenkins for CI/CD, I also love Github Actions.