Task VS doit

So many tools in this space! This one looks a little bit like go-task, but it seems maybe better for production workflows because if timeout support, while go-task seems more aimed to command line work/makefile replacement.

—-

https://github.com/go-task/task

View on GitHub

A similar tool is `task` https://taskfile.dev/ . It is quite capable and also a single executable. I've grown to quite like it.

check out tasks - a bit of a learning curve but arguably more powerful imo

That's when I came across taskfile.dev. Task is an automation tool designed to be more accessible than other options, such as GNU Make.

Full disclosure, I did not review Conda or Hatch fully. Not that there is anything explicitly wrong with either of them. Conda is too specific to the scientific community for my general taste. Hatch seems to go well with Conda and also uses the PyProject manifest as well. It's nice that it gives you several built in tools, similar to commit hooks, but I tend to like to roll my own via a Taskfile and run them with Poetry.

Taskfile is a tool for streamlining repetitive development tasks. It helps automate activities like building, testing, and deploying applications. Unlike Makefile, Taskfile uses YAML for configuration, making it more readable and user-friendly.

9. We test everything with another promotion which runs make targets which build docker containers to run python scripts (pytest)

This is also built by a complicated web of wildcarded makefile targets, which need to be interoperable and support a few if/else cases for specific components.

My plan is to migrate all of this to something simpler and more straightforward, or at least more maintainable, which is honestly probably going to turn into taskfile[0] instead of makefiles, and then simple python scripts for the glue that ties everything together or does more complex logic.

My hope is that it can be more straightforward and easier to maintain, with more component-ized logic, but realistically every step in that labyrinthine build process (and that's just the open-source version!) came from a decision made by a very talented team of engineers who know far more about the process and the product than I do. At this point I'm wondering if it would make 'more sense' to replace it with a giant python script of some kind and get access to all the logic we need all at once (it would not).

[0] https://taskfile.dev/

task - a task runner and a replacement for make

I like mage for Go and doit for Python.

Some competitors - Rake (ruby) - Bake - Earthly - SCons - doit

An alternative to Scons could be Doit (<https://pydoit.org/>), which if I remember correctly was built as a faster alternative to Scons. See also reasons of some users to prefer the later to other mentioned here: <https://pydoit.org/stories.html>.

If you code in Python, your probably should use the language as much as possible and avoid calling shell commands.

E.G:

- manipulate the file system with pathlib

- do hashes with hashlib

- zip with zipfile

- set error code with sys.exit

- use os.environ for env vars

- print to stderr with print(..., file=...)

- sometimes you'll need to install lib. Like, if you want to manipulate a git repo, instead of calling the git command, use gitpython (https://gitpython.readthedocs.io/en/stable/)

But if you don't feel like installing a too many libs, or just really want to call commands because you know them well, then the "sh" lib is going to make things smoother:

https://pypi.org/project/sh/

Also, enjoy the fact Python comes with argparse to parse script arguments (or if you feel like installing stuff, use typer). It sucks to do it in bash .

If what you need is more build oriented, like something to replace "make", then I would instead recommend "doit":

https://pydoit.org/

It's the only task runner that I haven't run away from yet.

Remember to always to everything in a venv. But you can have a giant venv for all the scripts, and just she-bang the venv python executable so that it's transparent. Things don't have to be difficult.

I've been using Doit for a project which involves gathering together documents made up of multiple Markdown files and converting to multiple formats. It's really cool but has some irritations. It didn't end up being much simpler than Make for me. I'm interested in trying some of the alternatives people have posted.

The code for all of this is available here, and described in detail in my article. I'm particularly fan of doit for this type of project, and highly encourage everyone to check it out!

Instead, I use pydoit (which is basically a Python version of make). It's simple, flexible, and quite extensible. So, here's what I do with it: