unify-flowret VS Task

Nice. It is great to see native lightweight opensource (I hope it is considering that someone said that there is no license file yet) solutions hit this space. For what it's worth, I have built something similar to this but for Java programming language. You can find it here -> https://github.com/americanexpress/unify-flowret. My reason for building something like this was that the product market is just too unwieldy to work with and has multiple layers of complexity which most of the time can be done away with. Just my opinion.

On a side note, you will at some point in time have to deal with multi version workflows. I know that this is one feature that limits wide adoption of an orchestrator.

"And that’s when a real problem emerged; nobody knew who I was" and "I felt invisible, like I was all alone in this world. I could have had the best thing in the world, and no one would care": Could not agree with you more on this - over the past many years it has been more and more important to build a digital brand for yourself and have many followers - to be networked with people and know people who can provide you with reach - something which is difficult for introverts and challenging to do now since I have crossed 50. I personally have felt this helplessness in trying to promote a couple of opensource Java libraries I released on behalf of my employer (shameless plug here -> https://github.com/americanexpress/unify-jdocs and https://github.com/americanexpress/unify-flowret). I thought I had done something of value which would be readily adopted by people after they saw it - guess what - first I have not been able to get through to a wide audience and second - I underestimated what it takes to get people out of their comfort zone and their traditional thinking. It is so difficult for people to accept that there may be better ways of doing things once they get used to a certain way. Anyway, I don't think you failed - you learnt and it is never too late to learn and do something new. Something will click eventually and I wish you all the very best.

So many tools in this space! This one looks a little bit like go-task, but it seems maybe better for production workflows because if timeout support, while go-task seems more aimed to command line work/makefile replacement.

—-

https://github.com/go-task/task

View on GitHub

A similar tool is `task` https://taskfile.dev/ . It is quite capable and also a single executable. I've grown to quite like it.

check out tasks - a bit of a learning curve but arguably more powerful imo

That's when I came across taskfile.dev. Task is an automation tool designed to be more accessible than other options, such as GNU Make.

Full disclosure, I did not review Conda or Hatch fully. Not that there is anything explicitly wrong with either of them. Conda is too specific to the scientific community for my general taste. Hatch seems to go well with Conda and also uses the PyProject manifest as well. It's nice that it gives you several built in tools, similar to commit hooks, but I tend to like to roll my own via a Taskfile and run them with Poetry.

Taskfile is a tool for streamlining repetitive development tasks. It helps automate activities like building, testing, and deploying applications. Unlike Makefile, Taskfile uses YAML for configuration, making it more readable and user-friendly.

9. We test everything with another promotion which runs make targets which build docker containers to run python scripts (pytest)

This is also built by a complicated web of wildcarded makefile targets, which need to be interoperable and support a few if/else cases for specific components.

My plan is to migrate all of this to something simpler and more straightforward, or at least more maintainable, which is honestly probably going to turn into taskfile[0] instead of makefiles, and then simple python scripts for the glue that ties everything together or does more complex logic.

My hope is that it can be more straightforward and easier to maintain, with more component-ized logic, but realistically every step in that labyrinthine build process (and that's just the open-source version!) came from a decision made by a very talented team of engineers who know far more about the process and the product than I do. At this point I'm wondering if it would make 'more sense' to replace it with a giant python script of some kind and get access to all the logic we need all at once (it would not).

[0] https://taskfile.dev/

task - a task runner and a replacement for make