lsofer VS ShellCheck

/proc is amazing once you get the hang of it and get a good understanding of what's all in there. Especially if you're doing low level performance tuning.

It's particularly helpful in larger infrastructures where tool the variability means differences in available tooling, and their output plus cli options. I'm sure /proc iteration has its own issues of variability across large infrastructres, but I haven't seen it. It's a fairly consistent API. Or at least it was, since I haven't touched a large infrastructure in some time.

When I got tired of `lsof` not being installed on hosts (or when its `-i` param isn't available) I ended up writing a script [1] that just iterates through /proc over ssh and grabs all inet sockets, environment variables, command line, etc from a set of hosts. Results in a null-delimited output that can then be fed into something like grafana to create network maps. Biggest problem with it is the use of pipes means all cores go to 100% for the few seconds it takes to run.

[1] https://github.com/red-bin/lsofer

Oh yeah, bash functions are great and absolutely abusable. Sometimes you need some grand hacks to get it to work well, but when it works well, it can do some magic. You can even export functions over ssh!

I wrote this a few years back which ran on bunches of hosts and fed into a infrastructure network mapper based on each hosts' open network sockets to other known hosts. It wasn't really feasible to install a set of tools on random hosts.. but I still had root ssh access across the board. So I needed something tool agnostic, short, auditable, and effectively guaranteed to work:

https://github.com/red-bin/lsofer/blob/master/lsofer.sh

ncurse, dialog, zenity[2]. i/o buffering may be an issue [3a,3b]

Assuming using same account, use history command to show past commands[0a, 0b]

'load random example' on shellcheck using own custom examples from history command.[1]

--------

[3a] : http://www.gnu.org/software/coreutils/manual/html_node/stdbu...

[3b] : http://unix.stackexchange.com/questions/25372/how-to-turn-of...

[2] : http//funprojects.blog/2021/01/25/zenity-command-line-dialogs/

[1] : http://www.shellcheck.net/

[0a] : http://www.tecmint.com/history-command-examples/

[0b] : http://www.tecmint.com/remember-linux-commands/

web based documentation: https://www.tecmint.com/linux-commands-cheat-sheet/

commands grouped by typical usage patterns : https://www.tecmint.com/essential-linux-commands/

... #************************** Terraform ************************************* ARG TERRAFORM_VERSION=1.7.3 RUN set -ex \ && curl -O https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip && unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d /usr/local/bin/ RUN set -ex \ && mkdir -p $HOME/.terraform.d/plugin-cache && echo 'plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"' > ~/.terraformrc #************************* Terragrunt ************************************* ARG TERRAGRUNT_VERSION=0.55.1 RUN set -ex \ && wget https://github.com/gruntwork-io/terragrunt/releases/download/v${TERRAGRUNT_VERSION}/terragrunt_linux_amd64 -q \ && mv terragrunt_linux_amd64 /usr/local/bin/terragrunt \ && chmod +x /usr/local/bin/terragrunt #*********************** Terramate **************************************** ARG TERRAMATE_VERSION=0.4.5 RUN set -ex \ && wget https://github.com/mineiros-io/terramate/releases/download/v${TERRAMATE_VERSION}/terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && tar -xzf terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && mv terramate /usr/local/bin/terramate \ && chmod +x /usr/local/bin/terramate #*********************** tfsec ******************************************** ARG TFSEC_VERSION=1.28.5 RUN set -ex \ && wget https://github.com/aquasecurity/tfsec/releases/download/v${TFSEC_VERSION}/tfsec-linux-amd64 \ && mv tfsec-linux-amd64 /usr/local/bin/tfsec \ && chmod +x /usr/local/bin/tfsec \ && terragrunt --version #**********************Terraform docs ************************************ ARG TERRRAFORM_DOCS_VERSION=0.17.0 RUN set -ex \ && curl -sSLo ./terraform-docs.tar.gz https://terraform-docs.io/dl/v${TERRRAFORM_DOCS_VERSION}/terraform-docs-v${TERRRAFORM_DOCS_VERSION}-$(uname)-amd64.tar.gz \ && tar -xzf terraform-docs.tar.gz \ && chmod +x terraform-docs \ && mv terraform-docs /usr/local/bin/terraform-docs #********************* ShellCheck ***************************************** ARG SHELLCHECK_VERSION="stable" RUN set -ex \ && wget -qO- "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION?}/shellcheck-${SHELLCHECK_VERSION?}.linux.x86_64.tar.xz" | tar -xJv \ && cp "shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/bin/ \ && shellcheck --version ...

ShellCheck: https://github.com/koalaman/shellcheck

If I want to write better shell scripts I usually run shellcheck and adjust accordingly or if I need facilities not provided by the shell i switch to a full fledged programming language. Ans oh yes, `sh` is present almost on every BSD and Linux box for free so I consider it an important thing to at least be comfortable with.

shellcheck: https://www.shellcheck.net/

When I run nix-shell at the root of the project it puts me in a Nix shell that contains, among other programs, caddy and shellcheck. Notice that in the shellHook I add the project's shell scripts to the PATH. So once I'm in the Nix shell I can, among other things:

I have also seen some projects on github like ShellCheck which first make a library, expose all the modules and then simple add that do build-depends of the final executable. Is this the recommended approach than having just one executable and adding all the modules to other-modules:?

The error checks can be pretty arcane:

   https://github.com/koalaman/shellcheck/wiki/Checks

Similar to for instance shellcheck to check the syntax of shell scripts, is there an equivalent for the set of roff commands typically used in a (Linux) man page? I'm aware that e.g. pandoc permits the conversion of an other format (e.g., org) to both roff man and roff ms.