Sometimes you need a really simple way to generate parameterised text without pulling in a full-blown templating language as a dependency — for example, when writing an install script that needs to generate a simple configuration file. Using the classic *nix Bourne shell that’s installed on practically every *nix system is one option. To be honest, it can be a terrible option, but it often gets simple jobs done, so I think it’s a trick worth remembering.

Here’s a simple example that interpolates some variables, and uses the date command to create a timestamp. The config is saved in foo.conf:

#!/bin/sh

: ${SKIN=default}

# Pretend there's installation code here

cat <<EOF > foo.conf
username: ${USER}
skin: ${SKIN}
date_installed: $(date -Iseconds)
EOF

# Can put more installation code here

Don’t forget to chmod +x the script before running. The value of SKIN defaults to “default”, but can be overridden like this:


$ SKIN=blue ./install.sh

This is usually a simpler way to pass parameters than using arguments.

Here’s a slightly more complex example that generates a Linux firewall config in iptables-save format. This time it’s a standalone script that dumps the config to standard output:

#!/bin/sh

# Demonstrates using shell functions and loops for simplifying the generation of a (toy) firewall config

connectionLimit () {
  PORT="$1"
  LIMIT="$2"
  echo -A INPUT -p tcp --syn --dport "${PORT}" -m connlimit --connlimit-above "${LIMIT}" -j LOG --log-prefix '"iptables: too many connections "'
  echo -A INPUT -p tcp --syn --dport "${PORT}" -m connlimit --connlimit-above "${LIMIT}" -j REJECT --reject-with tcp-reset
}

cat <<EOF
*filter

-P INPUT DROP
-P OUTPUT DROP
-P FORWARD DROP

# Open input ports
$(for PORT in 22 25 80
do
  echo -A INPUT -i eth0 -p tcp --dport ${PORT} --syn -m conntrack --ctstate NEW -j ACCEPT
done)

# Connection limiting
$(connectionLimit 22 3)
$(connectionLimit 25 3)
$(connectionLimit 80 30)

# TODO: open some output ports and other stuff

COMMIT
EOF

Using standard output is more flexible. You can still save to a file like this:


$ ./gen_fw_config.sh > iptables.conf

Or pipe the config to another program:


$ ./gen_fw_config.sh | iptables-restore

Or, if a program takes a config filename as an argument, you can use a (Bash) shell trick to pass the config directly from the script without needing to write to disk at all:


$ my-iptables-linter --config <(./gen_fw_config.sh)

What’s the Catch?

The Bourne shell is really convenient, but makes a pretty horrible programming language. I’m keeping one eye on the Oil shell project, but until that matures, /bin/sh is what we’ve got.

I can point out two specific problems with Bourne for templating, though. One is that sh doesn’t have any good data structures — it’s built on an “everything is a string” design. Sure, some shells like Bash extend sh with arrays, but they’re only a small extension to “everything is a string”, and don’t make things much better. No good data structures also means no good ways to transform data, so generating complex JSON/YAML gets messy.

The second problem is even more serious: error handling. Take a look at this:


$ bork bork bork
bash: bork: command not found
$ # ^ sure enough, "bork bork bork" is an error
$ cat gen_conf.sh 
#!/bin/sh

# sh ignores errors by default
# Let's enable "exit on error" mode
set -e

cat <<EOF
important_data: $(bork bork bork)
date_generated: $(date -Iseconds)
signed: me
EOF
$ ./gen_conf.sh > foo.conf
./gen_conf.sh: line 7: bork: command not found
$ echo $?
0
$ # ^ we saw an error message, but the script completed "successfully"
$ cat foo.conf
important_data: 
date_generated: 2018-03-06T17:05:38+11:00
signed: me
$ # ^ broken config

If a command fails inside $(), the shell completely ignores the error and keeps going. Even with set -e. (Gotcha! The idea that set -e and set -u make shell scripting safe makes me wince a little.) If you want the error to be detected, you have to rewrite the script like this:

#!/bin/sh

# sh ignores errors by default
# Let's enable "exit on error" mode
set -e

IMPORTANT_DATA="$(bork bork bork)"

cat <<EOF
important_data: ${IMPORTANT_DATA}
date_generated: $(date -Iseconds)
signed: me
EOF

There are still a few more gotchas. Putting export or (for Bash) local in front of the assignment makes the error get ignored again. Do the local/export on one line, and the assignment on another. Also, a quirk of POSIX shell quoting rules means that if the bork command actually works and outputs something that contains a *, or something else that looks like a glob, it will get expanded as a glob, even though the command substitution is wrapped in double quotes (surprise!). The only way to prevent this seems to be to disable globbing completely with set -o noglob. This wiki page has a good list of shell gotchas.

Despite all these caveats, sh is still a good simple tool for simple templating jobs. The next step up would be using Python scripting (which is still available on most *nix platforms), or just biting the bullet and installing a proper templating language.