pqdm VS subb

If you like tqdm, it's worth checking out pqdm, a parallelized version. If you have embarrassingly parallel work to process in a script, it makes it dead simple to parallelize and monitor the progress of something. Highly recommend:

https://github.com/niedakh/pqdm

I wrote this to simplify just that for my own tools; the subprocess module that comes with the batteries has a very general interface, i think that it is a bit complex for a quick script.

https://github.com/MoserMichael/subb

https://pypi.org/project/subb/

Python doesn't have the problem of shell scripting language, it doesn't get impractical, as the program is getting more complex. In bash you have arrays, and even maps, but these aren't pretty. Also the shell scripting language is being evaluated by an parse tree/AST interpreter, that's significantly slower than even python, in it's byte code interpreted form.

My objective was to get an abstraction, for a one line process run and extraction of the result, similar to what we had in Perl5 with the system library function. https://perldoc.perl.org/functions/system

Also the shell is impractical, when it comes to slightly more complex programs. There is a limit on what you can do with pipes. Maybe that's the reason why perl is that flexible, as they tried to bridge both realms: Perl had to be useful as a replacement for the quick shell like script, and to be useful as a general purpose programming language.

i have a subprocess wrapper, might also be of help: https://github.com/MoserMichael/subb