Shell Script Best Practices, from a decade of scripting things

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • ShellCheck

    ShellCheck, a static analysis tool for shell scripts

  • Use a linter.

    Pass all scripts through https://www.shellcheck.net/ or use `shellcheck` on the commandline.

    Learn the things it tells you and implement them in future scripts.

  • isort

    A Python utility / library to sort imports.

  • And this rule has been followed the majority(if not all) interpreted and scripting languages. The likes of Ruby, python and JS have multiple examples. Whatever executable is in your $PATH it won’t have an extension.

    Not sure if this convention is actually documented anywhere.

    Random examples:

    - https://github.com/PyCQA/isort/blob/main/pyproject.toml#L100

    - https://github.com/pypa/pip/blob/main/setup.py#L78

    - https://github.com/11ty/eleventy/blob/master/package.json#L1...

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • pip

    The Python package installer

  • And this rule has been followed the majority(if not all) interpreted and scripting languages. The likes of Ruby, python and JS have multiple examples. Whatever executable is in your $PATH it won’t have an extension.

    Not sure if this convention is actually documented anywhere.

    Random examples:

    - https://github.com/PyCQA/isort/blob/main/pyproject.toml#L100

    - https://github.com/pypa/pip/blob/main/setup.py#L78

    - https://github.com/11ty/eleventy/blob/master/package.json#L1...

  • eleventy 🕚⚡️

    A simpler site generator. Transforms a directory of templates (of varying types) into HTML.

  • And this rule has been followed the majority(if not all) interpreted and scripting languages. The likes of Ruby, python and JS have multiple examples. Whatever executable is in your $PATH it won’t have an extension.

    Not sure if this convention is actually documented anywhere.

    Random examples:

    - https://github.com/PyCQA/isort/blob/main/pyproject.toml#L100

    - https://github.com/pypa/pip/blob/main/setup.py#L78

    - https://github.com/11ty/eleventy/blob/master/package.json#L1...

  • bash-modules

    Useful modules for bash

  • Template in article is awful. It's better to use this one, which is a real CLI tool: https://github.com/vlisivka/bash-modules/blob/master/bash-mo...

  • murex

    A smarter shell and scripting environment with advanced features designed for usability, safety and productivity (eg smarter DevOps tooling)

  • > getting the shell quoting hell right

    Shameless plug coming, it this has been a pain point for me too. I found the issue with quotes (in most languages, but particularly in Bash et al) is that the same character is used to close the quote as is used to open it.m. So in my own shell I added support to use parentheses as quotes in addition to the single and double quotation ASCII symbols. This then allows you to nest quotation marks.

    https://murex.rocks/docs/parser/brace-quote.html

    You also don’t need to worry about quoting variables as variables are expanded to an argv[] item rather than expanded out to a command line and then any spaces converted into new argv[]s (or in layman’s terms, variables behave like you’d expect variables to behave).

    https://github.com/lmorg/murex

  • mg.sh

    Mitigram's shell library of reusable script snippets

  • I agree, getting to know where a script "comes from" can be complex though. You can `readlink -f` (or equivalent) in many cases, but when implementing a library this might not be entirely practical. I have had to rely on this ugly if-statement [1] for that purpose.

      [1]: https://github.com/Mitigram/mg.sh/blob/cbeb206d67fe08be2107deee50acf877f990dbdf/bootstrap.sh#L6

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • rq

    Record Query - A tool for doing record analysis and transformation (by dflemstr)

  • Not sure what it is doing more...I'm referring to this rq: https://github.com/dflemstr/rq#format-support-status

    It converts to/from the listed formats.

    There is also `jc` (written in Python) with the added benefit that it converts output of many common unix utilities to json. So you would not need to parse `ip` for example.

    https://github.com/kellyjonbrazil/jc

  • jc

    CLI tool and python library that converts the output of popular command-line tools, file-types, and common strings to JSON, YAML, or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.

  • Not sure what it is doing more...I'm referring to this rq: https://github.com/dflemstr/rq#format-support-status

    It converts to/from the listed formats.

    There is also `jc` (written in Python) with the added benefit that it converts output of many common unix utilities to json. So you would not need to parse `ip` for example.

    https://github.com/kellyjonbrazil/jc

  • learn_programing

    Discontinued Learning Shell,Python,Golang,System,Network [UnavailableForLegalReasons - Repository access blocked]

  • > Don't you find its syntax cumbersome?

    Not really; just seems the same as most other dynamic languages. Awk does a lot of stuff for you (the "implied loop" your program runs in, field splitting) that's certainly possible (even easy) to replicate in Python or Ruby, but Awk it's just so much more convenient.

    I use it for things like processing the Unicode data files, making some program output a bit nicer (e.g. go test -bench), ad-hoc spreadsheets, few other things. I got started with it as I needed to process some C header files and the existing script for that was in Awk; it worked pretty well for that too.

    The Awk Programming Language book is pretty good. GNU Awk has a bunch of very useful extensions, but pretty much everything in the book still works and is useful today. You can get it at e.g. https://archive.org/details/awkprogrammingla00ahoa or https://github.com/teamwipro/learn_programing/blob/master/sh...

    The GNU Awk docs are also pretty decent.

  • dmenu-scripts

    Serious fun with dmenu

  • Yes, for creating ad-hoc mini-UIs so the user can select an option. Same with fzf, but it's terminal-bound (rather than X-bound).

    The scripts are similar to this one:

    https://github.com/debxp/dmenu-scripts/blob/master/dmenu-kil...

  • Command-line-text-processing

    Discontinued :zap: From finding text to search and replace, from sorting to beautifying text and more :art:

  • Submitted yesterday:

    Learn to use Awk with hundreds of examples

    https://github.com/learnbyexample/Command-line-text-processi...

    https://news.ycombinator.com/item?id=33349930

  • Submitted yesterday:

    Learn to use Awk with hundreds of examples

    https://github.com/learnbyexample/Command-line-text-processi...

    https://news.ycombinator.com/item?id=33349930

  • PowerShell

    PowerShell for every system!

  • > "don't care about free software and commons"

    https://github.com/PowerShell/PowerShell/ MIT License

  • yq

    yq is a portable command-line YAML, JSON, XML, CSV, TOML and properties processor

  • zx

    A tool for writing better scripts

  • shellcheck-repl

    Validation of Shell Commands Before Evaluation

  • > "Use shellcheck."

    (Disclaimer: I'm one of the authors)

    After falling in love with ShellCheck several years ago, with the help of another person, I made the ShellCheck REPL tool for Bash:

      https://github.com/HenrikBengtsson/shellcheck-repl>

  • bootstrap

  • Though I'll probably just settle for one of those keywords, I like 'all' the best atm. This would run the trace only if this special keyword is given, or if the variable contains the name of the script. I initially had `basename "${0%%.sh}"` but that makes things like `test,test1,test2` impossible, though that only helps if the extension is used ;)

    While not needing be part of this list, I personally also always use functions, unless the script really is less then a handful lines, like google's bash-guide, always have a main etc.

    In the end though I do disagree writing bash and bashisms. There's plenty of posts about that though; kind of like the whole C++ 'which version to use' discussion, where bash tends to be inconsistent in itself.

    Shameless plug: See some of my older stuff here https://gitlab.com/esbs/bootstrap/-/blob/master/docker_boots...

  • nushell

    A new type of shell

  • I expect nushell to massively change how I work:

    https://www.nushell.sh/

    It's a shell that is actually built for structured data, taking lessons learned from PowerShell and others.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts