hpc-rocket
ZnTrack
| hpc-rocket | ZnTrack | |
|---|---|---|
| 1 | 2 | |
| 24 | 42 | |
| - | - | |
| 4.8 | 7.7 | |
| 3 months ago | 4 days ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
hpc-rocket
-
HPC Rocket - A tool to run Slurm jobs from CI pipelines
And finally, a link to the GitHub repo: https://github.com/SvenMarcus/hpc-rocket
ZnTrack
-
What are some good examples of well-engineered pipelines
I expaned a bit on them with my own package https://zntrack.readthedocs.io/ - a general framework for building DVC pipelines through python scripts (and more). This finally brings me to the project I'm actually working on https://github.com/zincware/IPSuite which brings all of this together for the specific use case of machine learned interatomic potentials.
-
HPC Rocket - A tool to run Slurm jobs from CI pipelines
This looks really interesting! I have a similar scenario but haven't looked into it yet. Have you looked at dvc.org - I'm planning on using it together with slurm and what they call CML for my projects. On that context I also wrote a tool that makes DVC more pythonic https://github.com/zincware/ZnTrack altough I'm currently restructuring it a bit but having backwards compatibility in mind.
What are some alternatives?
jacamar-ci
fiftyone - The open-source tool for building high-quality datasets and computer vision models
Buildbot - Python-based continuous integration testing framework; your pull requests are more than welcome!
mlem - 🐶 A tool to package, serve, and deploy any ML model on any platform. Archived to be resurrected one day🤞
pypyr automation task runner - pypyr task-runner cli & api for automation pipelines. Automate anything by combining commands, different scripts in different languages & applications into one pipeline process.
Activeloop Hub - Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. https://activeloop.ai [Moved to: https://github.com/activeloopai/deeplake]
dvc - 🦉 ML Experiments and Data Management with Git
messages - xcompute sub-module (CAE schema layer) including serialization utilities and bindings
IPSuite - Machine Learned Interatomic Potential Tools
example-get-started - Get started DVC project (NLP, random forest)