make-booster
snakemake-wrappers
make-booster | snakemake-wrappers | |
---|---|---|
3 | 1 | |
8 | 206 | |
- | 2.4% | |
10.0 | 9.9 | |
almost 2 years ago | 4 days ago | |
Makefile | Python | |
MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
make-booster
-
Snakemake – A framework for reproducible data analysis
For a very different approach, check out make-booster:
https://github.com/david-a-wheeler/make-booster
Make-booster provides utility routines intended to greatly simplify data processing (particularly a data pipeline) using GNU make. It includes some mechanisms specifically to help Python, as well as general-purpose mechanisms that can be useful in any system. In particular, it helps reliably reproduce results, and it automatically determines what needs to run and runs only that (producing a significant speedup in most cases). Released as open source software.
-
A Love Letter to Make
https://github.com/david-a-wheeler/make-booster
I think a lot of hate on make is due to poor use. If your makefile is complex, refactor it. Auto-generate dependencies (it only takes a few lines in GNU make). And don't use recursive make, that way lies madness. I also think GNU make is the wiser tool; POSIX make lacks too much in many cases.
-
The Unreasonable Effectiveness of Makefiles
https://github.com/david-a-wheeler/make-booster
From its readme:
"This project (contained in this directory and below) provides utility routines intended to greatly simplify data processing (particularly a data pipeline) using GNU make. It includes some mechanisms specifically to help Python, as well as general-purpose mechanisms that can be useful in any system. In particular, it helps reliably reproduce results, and it automatically determines what needs to run and runs only that (producing a significant speedup in most cases)."
"For example, imagine that Python file BBB.py says include CC, and file CC.py reads from file F.txt (and CC.py declares its INPUTS= as described below). Now if you modify file F.txt or CC.py, any rule that runs BBB.py will automatically be re-run in the correct order when you use make, even if you didn't directly edit BBB.py."
This is NOT functionality directly provided by Python, and the overhead with >1000 files was 0.07seconds which we could live with :-).
snakemake-wrappers
-
Snakemake – A framework for reproducible data analysis
1. Command-line tools are often used in steps of a bioinformatics pipeline, and they bridge the gap (e.g. https://github.com/snakemake/snakemake-wrappers).
What are some alternatives?
tclmake - Partial make clone in pure Tcl
mandala - A powerful and easy to use Python framework for experiment tracking and incremental computing
checkexec - CLI tool to conditionally execute commands only when files in a dependency list have been updated. Like `make`, but standalone.
tes-azure-legacy - [DEPRECATED] - A GA4GH Task Execution Service (TES) compatible implementation for Azure Compute
oxen-release - Lightning fast data version control system for structured and unstructured machine learning datasets. We aim to make versioning datasets as easy as versioning code.
dagger - Application Delivery as Code that Runs Anywhere
just - 🤖 Just a command runner
handlebars.c - C implementation of handlebars.js
bake - A Bash-based Make alternative.