The Unreasonable Effectiveness of Makefiles

• just

  163 16,971 9.1 Rust

  🤖 Just a command runner

For a task runner I really like just and its Justfile format: https://github.com/casey/just It is heavily inspired by make but doesn't focus on the DAG stuff (but does support tasks and dependencies). Crucially it has a much better user experience for listing and documenting tasks--just comment your tasks and it will build a nice list of them in the CLI.

If most of your make usage is a bunch of .PHONY nonsense and tricks to make it so developers can run a simple command to get going, check out just. You will find it's not difficult to immediately switch over to its task format.

• Airflow

  169 34,397 10.0 Python

  Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

  

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• bake

  2 22 4.7 Shell

  A Bash-based Make alternative. (by hyperupcall)

Honestly, I only find Makefiles useful when I have a tiny C/C++ project and need stuff just to compile quickly and easily without the overhead of a real build system.

For literally everything else, I found myself using it more as a task runner - and Make doesn't do a great job at it. You end up mixing Bash and Make variables, string interpolation, and it becomes really messy, really fast. Not to mention the footguns associated with Make.

I found bake (https://github.com/hyperupcall/bake) to suit my needs (disclaimer: I wrote it). It's literally just a Bash script with all the boilerplate taken care of you - what a task runner is meant to be imo

• dagger

  92 10,190 9.9 Go

  Application Delivery as Code that Runs Anywhere (by dagger)

  

I'd encourage anyone thinking of using make to look at alternatives. Make is great, but is quickly becomes a ball of duct-tape. Make works very well when you spend the time to express your dependency tree, but realistically that never happens and people tend to add hacks upon hacks for Makefiles. Not only that, but they don't scale well as your project adds more components, such as integration testing, documentation, etc.

I found Earthly[0] to be a great replacement. Everything runs in Docker. Your builds are reproducible, cache-able, and parallelization by default. I've heard Dagger[1] is another good tool in the space.

[0]: https://earthly.dev/

[1]: https://dagger.io/

• checkexec

  4 80 0.0 Rust

  CLI tool to conditionally execute commands only when files in a dependency list have been updated. Like `make`, but standalone.

You might find checkexec useful to pair with just, it is basically a tool that only does the file-based dependency part of make: https://github.com/kurtbuilds/checkexec

• run

  17 472 3.5 Go

  Task runner that helps you easily manage and invoke small scripts and wrappers (by TekWizely)

For those looking for a powerful task runners that feel like a makefile, please take a look at Run:

https://github.com/TekWizely/run

It's better a managing and invoking tasks and generates help text from comments.

• sub

  4 1,733 0.0 Shell

  a delicious way to organize programs

  

Scripts in bin have no documentation, no easy way to enumerate them, etc. There is definitely a time and a place for bin scripts, especially as things grow in complexity. However the beauty of just is that there's one file (the justfile) that defines all of your project's actions. You don't have to go spelunking into bin to figure out how to tweak a compiler flag, etc. And since just will run anything there's no reason why your complex bin scripts can't just be called from a simple one liner task in a justfile.

Could your write a bash script that does stuff like enumerate all the bin scripts, pull out documentation comments, etc.? Absolutely, and people have followed that pattern for a while (see https://github.com/qrush/sub) but it's a bunch of boilerplate to copy between projects. Just pulls out that logic into a simpler config file.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• handlebars.c

  1 33 10.0 C

  C implementation of handlebars.js

Yes, I used autotools[0]. It's definitely hairier than plain make, but you get a lot of useful features on top of it. There's thousands of examples all over the internet so it's easy to reference them.

I like the elegance of pure make, and do use it when appropriate, but I wouldn't really want to reimplement the things autotools does myself in it.

[0]: https://github.com/jbboehr/handlebars.c/blob/master/configur...

• docker-flask-example

  31 545 8.0 Python

  A production ready example Flask app that's using Docker and Docker Compose.

I did this for a while but make isn't well suited for this use case. What I end up doing is have a shell script with a bunch of functions in it. Functions automatically becomes a callable a command (with a way to make private functions if you want) with pretty much no boiler plate.

The benefit of this is it's just shell scripting so you can use shell features like $@ to pass args to another command or easily source and deal with env vars.

I've written about this process at https://nickjanetakis.com/blog/replacing-make-with-a-shell-s... and an example file is here https://github.com/nickjj/docker-flask-example/blob/main/run.

• acdw.net

  1 - -

• make-booster

  3 8 10.0 Makefile

  Utility routines to simplify using GNU make and Python

https://github.com/david-a-wheeler/make-booster

From its readme:

"This project (contained in this directory and below) provides utility routines intended to greatly simplify data processing (particularly a data pipeline) using GNU make. It includes some mechanisms specifically to help Python, as well as general-purpose mechanisms that can be useful in any system. In particular, it helps reliably reproduce results, and it automatically determines what needs to run and runs only that (producing a significant speedup in most cases)."

"For example, imagine that Python file BBB.py says include CC, and file CC.py reads from file F.txt (and CC.py declares its INPUTS= as described below). Now if you modify file F.txt or CC.py, any rule that runs BBB.py will automatically be re-run in the correct order when you use make, even if you didn't directly edit BBB.py."

This is NOT functionality directly provided by Python, and the overhead with >1000 files was 0.07seconds which we could live with :-).

• tclmake

  1 5 4.4 Tcl

  Partial make clone in pure Tcl

Suggest a related project