HPC Rocket - A tool to run Slurm jobs from CI pipelines

1. hpc-rocket

   1 1 26 3.4 Python

   

   And finally, a link to the GitHub repo: https://github.com/SvenMarcus/hpc-rocket

2. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

3. ZnTrack

   2 2 52 8.0 Python

   Create, visualize, run & benchmark DVC pipelines in Python & Jupyter notebooks.

   

   This looks really interesting! I have a similar scenario but haven't looked into it yet. Have you looked at dvc.org - I'm planning on using it together with slurm and what they call CML for my projects. On that context I also wrote a tool that makes DVC more pythonic https://github.com/zincware/ZnTrack altough I'm currently restructuring it a bit but having backwards compatibility in mind.

4. dvc

   3 121 14,454 8.9 Python

   🦉 Data Versioning and ML Experiments

   

   This looks really interesting! I have a similar scenario but haven't looked into it yet. Have you looked at dvc.org - I'm planning on using it together with slurm and what they call CML for my projects. On that context I also wrote a tool that makes DVC more pythonic https://github.com/zincware/ZnTrack altough I'm currently restructuring it a bit but having backwards compatibility in mind.

5. jacamar-ci

   4 2 - -

   If you get the chance take a look at Jacamar CI, it is a Gitlab custom executor for Slurm developed by the Exascale Computing Project. While this doesn't address your specific problem set, needs installation on a login node, maybe you can get some inspiration.

Suggest a related project