HyperQueue: ergonomic HPC task executor written in Rust

1. hyperqueue

   1 2 397 9.6 Rust

   Scheduler for sub-node tasks for HPC systems with batch scheduling

   

   Hi! Me and my colleagues are working on an HPC (High Performance Computing)-focused tool for executing workflows/pipelines/task graphs in an ergonomic and efficient manner. It is called HyperQueue, it is open-source (https://github.com/It4innovations/hyperqueue) and it is of course written in Rust :) (previous incarnations of the tool were written in C++, which was a nightmare to maintain). It leverages async Rust heavily, not just for I/O (we use Tokio), but also for communicating between a number of independent "processes" (actors) running on a single thread.

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. node_crunch

   2 15 78 0.0 Rust

   Discontinued Allows to distribute computations across several nodes

4. nextflow

   3 9 2,882 9.8 Groovy

   A DSL for data-driven computational pipelines

   

5. aiida-hyperqueue

   4 1 6 6.6 Python

   AiiDA plugin for the HyperQueue metascheduler.

   

Suggest a related project