Our great sponsors
- CodiumAI - TestGPT | Generating meaningful tests for busy devs
- ONLYOFFICE ONLYOFFICE Docs — document collaboration in your environment
- Sonar - Write Clean Python Code. Always.
- InfluxDB - Access the most powerful time series database as a service
-
Unfortunately, there are some issues with `dvc exp` --- the set of experiment tracking subcommands. In particular, I rely heavily on git submodules to partition the code that instantiates a model from the code that runs an experiment. But `dvc exp` doesn't work with submodules ATM. (Bug filed here.) This is unfortunate because, if `dvc exp` worked, it would make experiment tracking a little more convenient for us. It's not a deal breaker though. I use git branches to organize individual experiments and tags to organize stages of the same experiment. I use a shared dvc cache so that I can run multiple experiments at a time without using up too much workspace storage.
-
-
CodiumAI
TestGPT | Generating meaningful tests for busy devs. Get non-trivial tests (and trivial, too!) suggested right inside your IDE, so you can code smart, create more value, and stay confident when you push.
-
I find the value proposition of experiment trackers a bit off, at least in my domain (classic ML, DL is a different story). When developing a model, what gives you the biggest performance improvements is better data cleaning, rarely, hyperparameters have an important effect. So I do not use experiment trackers because I mostly work on iterating on my data and just generate a Jupyter notebook report (converted to HTML) for each experiment with some diagnostic plots.
-
aim
Aim 💫 — An easy-to-use & supercharged open-source AI metadata tracker (experiment tracking, AI agents tracing)
Same! Neptune is great. I'm also keeping an eye on Aim which is a very similar open-source solution but not as mature as Neptune yet. I can't wait to switch to this one because Neptune gets kind of pricey if working in a team.
-
Guild AI now has support for running DvC stages as experiments. DvC uses git under the covers to manage project state for each experiment, along with the experiment results. Guild doesn't touch your git repo and instead copies your project source to a new run directory. This ensures that you have a correct record of your experiment without churning your project state.
-
ONLYOFFICE
ONLYOFFICE Docs — document collaboration in your environment. Powerful document editing and collaboration in your app or environment. Ultimate security, API and 30+ ready connectors, SaaS or on-premises
