ZnTrack
hpc-rocket
| ZnTrack | hpc-rocket | |
|---|---|---|
| 2 | 1 | |
| 41 | 24 | |
| - | - | |
| 7.7 | 4.8 | |
| 6 days ago | 2 months ago | |
| Python | Python | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ZnTrack
-
What are some good examples of well-engineered pipelines
I expaned a bit on them with my own package https://zntrack.readthedocs.io/ - a general framework for building DVC pipelines through python scripts (and more). This finally brings me to the project I'm actually working on https://github.com/zincware/IPSuite which brings all of this together for the specific use case of machine learned interatomic potentials.
-
HPC Rocket - A tool to run Slurm jobs from CI pipelines
This looks really interesting! I have a similar scenario but haven't looked into it yet. Have you looked at dvc.org - I'm planning on using it together with slurm and what they call CML for my projects. On that context I also wrote a tool that makes DVC more pythonic https://github.com/zincware/ZnTrack altough I'm currently restructuring it a bit but having backwards compatibility in mind.
hpc-rocket
-
HPC Rocket - A tool to run Slurm jobs from CI pipelines
And finally, a link to the GitHub repo: https://github.com/SvenMarcus/hpc-rocket
What are some alternatives?
fiftyone - The open-source tool for building high-quality datasets and computer vision models
jacamar-ci
mlem - 🐶 A tool to package, serve, and deploy any ML model on any platform. Archived to be resurrected one day🤞
Buildbot - Python-based continuous integration testing framework; your pull requests are more than welcome!
Activeloop Hub - Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. https://activeloop.ai [Moved to: https://github.com/activeloopai/deeplake]
pypyr automation task runner - pypyr task-runner cli & api for automation pipelines. Automate anything by combining commands, different scripts in different languages & applications into one pipeline process.
messages - xcompute sub-module (CAE schema layer) including serialization utilities and bindings
dvc - 🦉 ML Experiments and Data Management with Git
IPSuite - Machine Learned Interatomic Potential Tools
example-get-started - Get started DVC project