doit VS ShellCheck

I like mage for Go and doit for Python.

Some competitors - Rake (ruby) - Bake - Earthly - SCons - doit

An alternative to Scons could be Doit (<https://pydoit.org/>), which if I remember correctly was built as a faster alternative to Scons. See also reasons of some users to prefer the later to other mentioned here: <https://pydoit.org/stories.html>.

If you code in Python, your probably should use the language as much as possible and avoid calling shell commands.

E.G:

- manipulate the file system with pathlib

- do hashes with hashlib

- zip with zipfile

- set error code with sys.exit

- use os.environ for env vars

- print to stderr with print(..., file=...)

- sometimes you'll need to install lib. Like, if you want to manipulate a git repo, instead of calling the git command, use gitpython (https://gitpython.readthedocs.io/en/stable/)

But if you don't feel like installing a too many libs, or just really want to call commands because you know them well, then the "sh" lib is going to make things smoother:

https://pypi.org/project/sh/

Also, enjoy the fact Python comes with argparse to parse script arguments (or if you feel like installing stuff, use typer). It sucks to do it in bash .

If what you need is more build oriented, like something to replace "make", then I would instead recommend "doit":

https://pydoit.org/

It's the only task runner that I haven't run away from yet.

Remember to always to everything in a venv. But you can have a giant venv for all the scripts, and just she-bang the venv python executable so that it's transparent. Things don't have to be difficult.

I've been using Doit for a project which involves gathering together documents made up of multiple Markdown files and converting to multiple formats. It's really cool but has some irritations. It didn't end up being much simpler than Make for me. I'm interested in trying some of the alternatives people have posted.

The code for all of this is available here, and described in detail in my article. I'm particularly fan of doit for this type of project, and highly encourage everyone to check it out!

Instead, I use pydoit (which is basically a Python version of make). It's simple, flexible, and quite extensible. So, here's what I do with it:

These projects use Caddy as my local development server, Dart Sass for converting my Sass files to CSS, elm, elm-format, elm-optimize-level-2, elm-review, elm-test (only in Calculator), ShellCheck to find bugs in my shell scripts, and Terser to mangle and compress JavaScript code.

ncurse, dialog, zenity[2]. i/o buffering may be an issue [3a,3b]

Assuming using same account, use history command to show past commands[0a, 0b]

'load random example' on shellcheck using own custom examples from history command.[1]

--------

[3a] : http://www.gnu.org/software/coreutils/manual/html_node/stdbu...

[3b] : http://unix.stackexchange.com/questions/25372/how-to-turn-of...

[2] : http//funprojects.blog/2021/01/25/zenity-command-line-dialogs/

[1] : http://www.shellcheck.net/

[0a] : http://www.tecmint.com/history-command-examples/

[0b] : http://www.tecmint.com/remember-linux-commands/

web based documentation: https://www.tecmint.com/linux-commands-cheat-sheet/

commands grouped by typical usage patterns : https://www.tecmint.com/essential-linux-commands/

... #************************** Terraform ************************************* ARG TERRAFORM_VERSION=1.7.3 RUN set -ex \ && curl -O https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip && unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d /usr/local/bin/ RUN set -ex \ && mkdir -p $HOME/.terraform.d/plugin-cache && echo 'plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"' > ~/.terraformrc #************************* Terragrunt ************************************* ARG TERRAGRUNT_VERSION=0.55.1 RUN set -ex \ && wget https://github.com/gruntwork-io/terragrunt/releases/download/v${TERRAGRUNT_VERSION}/terragrunt_linux_amd64 -q \ && mv terragrunt_linux_amd64 /usr/local/bin/terragrunt \ && chmod +x /usr/local/bin/terragrunt #*********************** Terramate **************************************** ARG TERRAMATE_VERSION=0.4.5 RUN set -ex \ && wget https://github.com/mineiros-io/terramate/releases/download/v${TERRAMATE_VERSION}/terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && tar -xzf terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && mv terramate /usr/local/bin/terramate \ && chmod +x /usr/local/bin/terramate #*********************** tfsec ******************************************** ARG TFSEC_VERSION=1.28.5 RUN set -ex \ && wget https://github.com/aquasecurity/tfsec/releases/download/v${TFSEC_VERSION}/tfsec-linux-amd64 \ && mv tfsec-linux-amd64 /usr/local/bin/tfsec \ && chmod +x /usr/local/bin/tfsec \ && terragrunt --version #**********************Terraform docs ************************************ ARG TERRRAFORM_DOCS_VERSION=0.17.0 RUN set -ex \ && curl -sSLo ./terraform-docs.tar.gz https://terraform-docs.io/dl/v${TERRRAFORM_DOCS_VERSION}/terraform-docs-v${TERRRAFORM_DOCS_VERSION}-$(uname)-amd64.tar.gz \ && tar -xzf terraform-docs.tar.gz \ && chmod +x terraform-docs \ && mv terraform-docs /usr/local/bin/terraform-docs #********************* ShellCheck ***************************************** ARG SHELLCHECK_VERSION="stable" RUN set -ex \ && wget -qO- "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION?}/shellcheck-${SHELLCHECK_VERSION?}.linux.x86_64.tar.xz" | tar -xJv \ && cp "shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/bin/ \ && shellcheck --version ...

ShellCheck: https://github.com/koalaman/shellcheck

If I want to write better shell scripts I usually run shellcheck and adjust accordingly or if I need facilities not provided by the shell i switch to a full fledged programming language. Ans oh yes, `sh` is present almost on every BSD and Linux box for free so I consider it an important thing to at least be comfortable with.

shellcheck: https://www.shellcheck.net/

When I run nix-shell at the root of the project it puts me in a Nix shell that contains, among other programs, caddy and shellcheck. Notice that in the shellHook I add the project's shell scripts to the PATH. So once I'm in the Nix shell I can, among other things:

I have also seen some projects on github like ShellCheck which first make a library, expose all the modules and then simple add that do build-depends of the final executable. Is this the recommended approach than having just one executable and adding all the modules to other-modules:?

The error checks can be pretty arcane:

   https://github.com/koalaman/shellcheck/wiki/Checks

Similar to for instance shellcheck to check the syntax of shell scripts, is there an equivalent for the set of roff commands typically used in a (Linux) man page? I'm aware that e.g. pandoc permits the conversion of an other format (e.g., org) to both roff man and roff ms.