When I started using the Bash (Bourne Again) shell and producing my subsequent first scripts, I treated it just like the old plain Bourne shell, the one we all already love and use in quite a few environments. Fortunately, Bash is much more than it appears to the eye and could be a real game changer in the life of the average developer.

This article will give more emphasis to the scripting side of Bash, but the occasional concept can be adapted to be applied when used as a command launcher in an interactive way.

Regarding scripts, although the strict POSIX compatibility is sometimes required, preventing us from using Bash directly and reverting to the standard Bourne shell capabilities, it is often easier and quicker to use bashisms (Bash-only features) to write our scripts.

So below let's see some tricks I learned throughout my Bash adventure.

Interactive interpreter

!

For interactive Bash use, I have only one small trick to remember: !. ! is part of a bigger world called history expansion.

Long before the up arrow let you search in your history, there was a mean of referring to a past history line and, you guessed it, was !.

$ !42 # this would execute the command at line 42 of your history

What if we need to repeat a command? It is possible to refer to a number of lines relative to the current line. Summarized in one sentence: 'execute the command I typed n lines ago'.

$ !-42 # this would print the command used 42 lines ago

And arguably the most useful of all ! shortcuts, it is possible to repeat command at last line.

$ !-1
$ !! # simpler version, synonym of '!-1'

And !! can be used as a "variable". The classical example in this case is sudo:

$ apt install firefox
E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?
$ sudo !!
$ sudo apt install firefox 

reverse-i-search (or reverse-search-history)

There are a number of situations where so many commands has been typed, that it is impossible to remember them by name or by line. In this cases, it is possible to search through history incrementally, using some parts of the command as search key.

For example, let's suppose that we have to search for the command find . -name "*.rst" -type f | python3 /usr/local/bin/scripts/pelican_importer.py but we don't want to type it all by hand, incurring in the risk of errors and subsequent re-typing. It is possible to press CRTL + R to search:

$
bck-i-search: _

Then type in the search key and press CRTL + R over to scroll through results:

$ find . -name "*.rst" -type f | python3 /usr/local/bin/scripts/pelican_importer.py
bck-i-search: find_

When the command is the one we desire to execute, it is sufficient to press Enter to fire it.

Bash scripts

Shebangs

This one should already be common knowledge to every developer using POSIX systems, but every script should declare at its start the interpreter that is going to execute the actual code. This is accomplished using the so called shebang line. My personal receipt is:

#!/usr/bin/env bash

That line should tell the current shell to execute the current script (if made executable with chmod +x) with the command written after the !# characters. A disclaimer though, in some exotic environments there is no /usr/bin/env executable and thus the shebang line will be invalid. In such cases, we need to revert to:

#!/usr/bin/bash

Or even:

#!/bin/bash

By the way, shebangs are a general scripting "rule", therefore they do work with a number of different scripting languages, such as Python, Perl, PHP and JavaScript.

Bash options

Have you ever noticed Bash does not halt when an error happens? Or that it will happily use uninitialized variables?

Well, there is a solution to all this issues (even if they are not real issues) that involves Bash options. As our trusted manual says:

$ man 1 bash

The options are off by default unless otherwise noted.

We need to enable them one by one if we want to exploit their behavior.

For example, let's make some script up to show some feature.

#!/usr/bin/bash

cd /foo
ls

Clearly this scripts changes directory to /foo and then tries to list its contents. If /foo doesn't exist, will ls be executed? Yes, it will. To prevent this behavior and exit if some command, simple or complex, throws an error, we need to use the errexit option:

#!/usr/bin/bash
set -e # enable errexit option

cd /foo
ls

This time the shell is going to stop execution with a cd: can't cd to /foo error.

Another example (that YOU SHOULD NOT RUN):

#!/usr/bin/bash
set -e

rm -rf $prefix/*

Here $prefix is clearly undefined, so the command will expand to:

#!/usr/bin/bash
set -e

rm -rf /*

In case you haven't understood the situation, this command, granted root privileges, will destroy your installation, even your machine in some situations. To prevent this, use the nounset option

#!/usr/bin/bash
set -e
set -u # enable nounset option

rm -rf $prefix/*

Finally an option useful during the script writing or debugging. Since a lot of expansions are taking place when Bash interprets your commands, a useful capability to have would be to be able to see the command after expansion has been performed. That's why the xtrace exists.

#!/usr/bin/bash
set -e
set -u
set -x # enable xtrace option

prefix="Hello, world!"
echo "This variable content: $prefix"

Once called, this will print out every command executed:

$ bash script.sh
+ prefix=Hello, world!
+ echo This variable content: Hello, world!
This variable content: Hello, world!

Summing all up, I tend to begin my scripts with the following header:

#!/usr/bin/bash
set -e
set -u
set -x

# ...

Then, when I finished writing them, I just comment out the xtrace option (or more elegantly disable it).

#!/usr/bin/bash
set -e
set -u
set +x # disable xtrace

# ...

Expansions

In the Bash world, expansion is a quite a scaring word, especially to a newcomer. Actually, there are just a couple rules to follow when we talk about expansion, or at least its common occurrences.

First, what is expansion? Expansion is a transformation that is performed on a number of constructs, after each line has been split into tokens. One example made earlier is:

prefix="Hello, world!"
echo "This variable content: $prefix"

Here, the variable $prefix gets expanded replacing it with its content, so that the string "This variable content: $prefix" becomes This variable content: Hello, world!. Results of these kind of operations could be unintuitive at times, leading to a lot of common errors.

Let's start from variable expansion. It is possible to decide if expansion is desired in a string using double or single quotes: the first does get transformed, the second doesn't.

$ name="Riccardo"
$ echo "My name is $name"
My name is Riccardo
$ echo 'My name is $name'
My name is $name

Also, let's reason on an example. Suppose we want to compose a string like Name_Surname (e.g. Riccardo_Macoratti), but we want the user to provide the name and surname.

#!/usr/bin/env bash
set -e
set -u
set +x

echo -n "Name: "
read name
echo -n "Surname: "
read surname

echo "$name_$surname"

Easy enough, but when we try to execute the script, an error shows up:

$ bash script.sh
Name: Riccardo 
Surname: Macoratti
name_: parameter not set

It is because Bash doesn't know what variable to expand, $name or $name_, and chooses to expand the longest one, that is to say $name_, which is obviously unset. Preventing this behavior is easy, just wrap the variable name in curly brackets.

#!/usr/bin/env bash
set -e
set -u
set +x

echo -n "Name: "
read name
echo -n "Surname: "
read surname

echo "${name}_$surname"

$ bash script.sh
Name: Riccardo  
Surname: Macoratti
Riccardo_Macoratti

The second (and IMHO most useful) form of expansion is command expansion. It si the power of "transform every command output into a variable", accomplished with the syntax $(command).

Let's say that we are in a hurry and need to perform a one-time task in the most time-efficient way we can think of. In this kind of situations, there is usually next to no time or interest to look for a clean solution on the manual or on the internet. For example, I tend to forget how to read a file in Bash.

$ file_contents=$(cat file.txt)

Now $file_contents contains the contents of file.txt. Another classical example is caching. When a long computation is terminated and it is desirable to store the result in memory, that is the time for a command expansion.

$ cache=$(find / -type f | grep -e '^.+\.conf') # search for every .conf file in your root

And yoy can see, command expansion leads the way to fun (and sometimes ugly...) one-liner, pipe expressions.

Short-circuit boolean operators

Every programmer worth of its name should know boolean operators and surely the most used are logical conjunction (∧), commonly called and, and logical disjunction (∨), commonly called or.

Bash unsurprisingly has them too and uses the short-circuiting variant. Boolean expressions are usually composed of multiple boolean operations chained together in some way, by means of an operator. It can happen that the result of the boolean expression may be known just after evaluating the first operation. Let's think of this examples:

$ a=true
$ b=false
$ $b && echo "no short-circuit here"
$ $a || echo "no short-circuit here"

These strings won't be printed out, because Bash already knows that the result of the two expressions, just having evaluated $b && or $a ||. In fact, false && [𝑥] evaluates to false for every 𝑥 and vice versa true || [𝑥] evaluates to true for every 𝑥.

Now, consider that every simple or advanced set of commands executed leaves a return value that is between 0 and 255 and that 0 is false and everything else is true. We can take advantage of the short-circuit feature as a compact branching devices, read if-else construct. This two scripts are semantically equivalent:

#!/usr/bin/env bash
set -e
set -u
set +x

echo -n "Number: "
read n

if [[ $n -lt 42 ]]; then
    echo "Less than 42"
else
    echo "Greater or equal to 42"
fi

#!/usr/bin/env bash
set -e
set -u
set +x

echo -n "Number: "
read n

[[ $n -lt 42 ]] \
    && echo "Less than 42" \
    || echo "Greater or equal to 42"

FYI: a lot of dynamically-typed languages are able to pull out this kind of technique, one over all JavaScript.