1. Overview
Sometimes, we deploy shell scripts from machine to machine with different operating systems. Therefore, it’s crucial to have portable shell scripts that are POSIX-compliant. POSIX is a set of standards designed to maintain compatibility among UNIX-based operating systems.
As a result, if we have POSIX-compliant shell scripts, we can reliably deploy and execute them with no compatibility issues on different machines.
In this article, we’ll discuss how to check for the POSIX compliance of shell scripts. First, we’ll go over the different shells that conform to the POSIX standard. Next, we’ll use the ShellCheck utility to improve the compatibility of shell scripts and eliminate any warnings and errors that might arise.
Lastly, we’ll explain why using a programming language instead of writing complex shell scripts is a better choice.
2. POSIX Compliance of the Shells
There is a variety of freely available shells that conform to the POSIX standard. However, some true POSIX-compliant shells are shipped with certain Linux distributions. By default, some distributions will link /bin/sh to POSIX-compliant shells like dash and ksh.
Usually, these shells (dash, ksh, tcsh) are minimal and don’t provide extra features like command and programmable completions. Therefore, they’re not the best choice for interactive use.
2.1. bash
The Bash shell conforms to the POSIX.1-2008 standard. It includes additional features not present in other POSIX-compliant shells like Ksh and Dash.
However, we should know that some features are not included in the POSIX standards. These features, usually called bashisms, include constructs like arrays, associative arrays, and functions.
The earlier versions of Bash weren’t fully POSIX-compliant. However, Bash 4.0 and the more recent versions are fully POSIX-compliant. So, if we write a POSIX-compliant syntax in Bash, we can run it on most UNIX and Linux-based operating systems.
We can start bash in POSIX mode with the –posix flag:
$ bash --posix
We can also check whether the POSIX mode is enabled in a bash session:
$ echo "$SHELLOPTS" | grep -o posix
posix
If “posix” didn’t print, we aren’t running in POSIX mode. We can run a shell script in POSIX mode by invoking it as sh:
$ /bin/sh script.sh
Alternatively, we can change the Shebang line to #!/bin/sh. Any Bashisms found in the script will result in an error that we can change accordingly.
For instance, let’s write a script that contains a Bashism:
#!/bin/sh
my_array=(one two three)
for item in ${my_array[@]}
do
echo $item
done
When we run this script through bash, it will print each array item:
$ bash script.sh
one
two
three
Now, if we run this using sh, it will print an error:
$ /bin/sh script.sh
script.sh: 3: Syntax error: "(" unexpected
As we can see, we get the error because the interpreter has no clue what this syntax is. It’s because POSIX shells do not implement the array types. So, it doesn’t recognize the array in the shell script.
2.2. dash
dash stands for Debian Almquist Shell. It’s a very minimal shell originally designed for the Debian operating system that is used by Debian derivatives like Ubuntu and Mint. It’s entirely POSIX-compliant by default. So, we don’t have to set any additional options to enable POSIX mode.
dash is four times faster than the ubiquitous Bash shell. Therefore, it’s the best selection for writing shell scripts that must be executed efficiently.
On Ubuntu and Debian, /bin/sh is symlinked to the dash shell:
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Apr 10 22:43 /bin/sh -> dash
On the other distributions, however, it might not be the default execution shell. However, we can make it so through linking:
$ ln -sfT dash /bin/sh
Let’s break this down:
- -s creates a soft or symbolic link
- -f removes the existing destination files
- -T will treat the link name as the file
2.3. ksh
ksh or KornShell is widely used on commercial UNIX systems like AIX and HP-UX. It’s more feature rich than dash and includes advanced constructs for writing shell scripts.
The three most used versions of ksh are:
- ksh88 — the original KornShell
- ksh93 — contains enhanced features
- OpenKsh — based on ksh93 and includes additional features
Ksh has certain features that are not compliant with the POSIX standard. Therefore, we can enable the POSIX mode on shell scripts by setting the POSIXLY_CORRECT=yes environment variable. So, we can either export the variable in our shell profile, or we can simply set it in a shell script.
2.4. tcsh
tcsh is the enhanced version of the csh that includes features like job control, auto-completion, and Vi keybindings. tcsh is not fully POSIX-compliant. However, it implements many of the POSIX features and functions and is a fairly POSIX-compliant shell.
For instance, tcsh includes command substitution using backticks instead of the POSIX $() syntax:
$ echo "Uptime: `uptime`"
Therefore, it should be last on our list if we are considering shells that are truly POSIX-compliant.
3. Checking POSIX Compliance with ShellCheck
ShellCheck is a command-line and static analysis tool that checks shell scripts for common errors, bugs, and vulnerabilities. ShellCheck supports several different shell dialects, including bash, dash, ksh, and zsh.
Not only that, but we can also integrate it with text editors, such as Vim, Emacs, and Visual Studio Code, to provide real-time feedback as we write shell scripts.
3.1. Installation
We can install ShellCheck from the official package repository under the canonical name shellcheck using a package manager.
On Debian-derivatives, we can use apt:
$ sudo apt install shellcheck -y
On Fedora and RHEL, we can use yum:
$ sudo yum install shellcheck -y
Once installed, let’s verify it:
$ shellcheck --version
ShellCheck - shell script analysis tool
version: 0.9.0
3.2. Checking POSIX Compliance
We can check for the POSIX compliance of a shell script by passing ShellCheck the –shell=sh option:
$ shellcheck --shell=sh script.sh
In script.sh line 3:
my_array=(one two three four five) # declare an array
^-----------------------^ SC3030 (warning): In POSIX sh, arrays are undefined.
In script.sh line 5:
for item in ${my_array[@]} # loop over the array
^------------^ SC2068 (error): Double quote array expansions to avoid re-splitting elements.
^------------^ SC3054 (warning): In POSIX sh, array references are undefined.
...
As we can see, ShellCheck prints out informative and readable output. We can make the changes accordingly and make the script fully POSIX-compliant.
3.3. Real-Time Linting
ShellCheck is also a linting tool that can analyze shell scripts to check for issues such as:
- Syntax errors and typos
- Unquoted variables and command substitutions
- Use of undefined variables
- Security vulnerabilities, such as command injection and shell injection
- Performance issues, such as unnecessary command substitutions and inefficient constructs
So, we can integrate it with our text editor through plugins like vim-shellcheck or flymake-shellcheck for real-time feedback. With the POSIX option turned on, it will also provide feedback for POSIX-related warnings.
4. Final Thoughts
The missing features like arrays and specialized loops result in frustration and more lines of code. Thus, over time, when the shell script grows to hundreds of lines, it becomes time-consuming to make the script POSIX-compliant.
For that reason, it is more convenient to use programming languages that are readily available on other operating systems. So, we can write these scripts in languages like Ruby, Perl, or Python. They will all run the same on every operating system — given the right version of the language.
Apart from access to sophisticated features and libraries, another advantage of using a programming language is that we divide the complex script into modules that are more manageable and easier to navigate through.
5. Conclusion
In this article, we went over the different POSIX-compliant shells. Then, we covered the ShellCheck utility to check for the compliance of shell scripts.
Finally, we discussed why we should prefer using a programming language over writing complex POSIX-compliant shell scripts.