make-booster VS bake

For a very different approach, check out make-booster:

https://github.com/david-a-wheeler/make-booster

Make-booster provides utility routines intended to greatly simplify data processing (particularly a data pipeline) using GNU make. It includes some mechanisms specifically to help Python, as well as general-purpose mechanisms that can be useful in any system. In particular, it helps reliably reproduce results, and it automatically determines what needs to run and runs only that (producing a significant speedup in most cases). Released as open source software.

https://github.com/david-a-wheeler/make-booster

I think a lot of hate on make is due to poor use. If your makefile is complex, refactor it. Auto-generate dependencies (it only takes a few lines in GNU make). And don't use recursive make, that way lies madness. I also think GNU make is the wiser tool; POSIX make lacks too much in many cases.

https://github.com/david-a-wheeler/make-booster

From its readme:

"This project (contained in this directory and below) provides utility routines intended to greatly simplify data processing (particularly a data pipeline) using GNU make. It includes some mechanisms specifically to help Python, as well as general-purpose mechanisms that can be useful in any system. In particular, it helps reliably reproduce results, and it automatically determines what needs to run and runs only that (producing a significant speedup in most cases)."

"For example, imagine that Python file BBB.py says include CC, and file CC.py reads from file F.txt (and CC.py declares its INPUTS= as described below). Now if you modify file F.txt or CC.py, any rule that runs BBB.py will automatically be re-run in the correct order when you use make, even if you didn't directly edit BBB.py."

This is NOT functionality directly provided by Python, and the overhead with >1000 files was 0.07seconds which we could live with :-).

Honestly, I only find Makefiles useful when I have a tiny C/C++ project and need stuff just to compile quickly and easily without the overhead of a real build system.

For literally everything else, I found myself using it more as a task runner - and Make doesn't do a great job at it. You end up mixing Bash and Make variables, string interpolation, and it becomes really messy, really fast. Not to mention the footguns associated with Make.

I found bake (https://github.com/hyperupcall/bake) to suit my needs (disclaimer: I wrote it). It's literally just a Bash script with all the boilerplate taken care of you - what a task runner is meant to be imo

Nope, I didn't add that in bakefiles because it seemed like supporting that in a declarative way would be overkill. Very simple dependencies can just be ran manually just as you think they could (calling `task.dev`), but I intentionally didn't want to bake in support for longer ones (at least not right now)