1. Overview
In this tutorial, we’ll cover different methods of assigning values to shell variables. We’ll go through piping different commands and assigning the final processed value to a variable.
Finally, we’ll see which method is better for our use case.
2. Using read With heredoc
As an alternative to pipes, we can use heredoc to read values into a variable. A heredoc is a redirection mechanism we use to specify an input source. It begins and ends with a tag. The contents inside the tag are used as the standard input for a command.
Let’s see what a simple heredoc would look like:
command << TAG
command_2
TAG
Mind that there are no trailing spaces beside the last TAG, and we can replace the TAG tag with any word of our choice. In our case, we’d like to read values from a heredoc and assign them to a shell variable:
#!/bin/sh
IFS= read platform << EOF
$(uname -a)
EOF
echo "$platform"
The IFS (Internal Field Separator) environment variable contains delimiters that we can use to split strings. We can set it to any delimiter that we need. In this scenario, we set it to nothing because we want to prevent word splitting. Afterward, we used the read keyword to read a value into the platform variable. The value that gets assigned to the platform variable will be the output of the command inside the heredoc — which is a simple uname command.
We named the heredoc tag EOF, which stands for “End of File”. It’s a convention that we often use when we write bash scripts.
Now, let’s see what value the platform variable holds:
$ ./uname.sh
Linux laptop 5.15.5-arch1-1 #1 SMP PREEMPT Thu, 25 Nov 2021 22:09:33 +0000 x86_64 GNU/Linux
We can append pipes to the command inside the heredoc, and it will function the same.
3. Using read With a while Loop
If we want to read a file that contains a lot of data, we can use a while loop and read the file line-by-line into a variable. We can then use the variable inside our while loop’s scope:
#!/bin/sh
cat packages.txt | while read pkg; do
echo "$pkg"
done
Executing the script should output the contents of the package.txt file line-by-line:
$ ./listpackages.sh
alacritty 0.9
bash 5.1
firefox 94.0
gcc 11.1
mpv 0.34
nvim 0.5
rust 1.56
sway 1.6
The loop will execute until it reaches the EOF. Mind that we cannot use the pkg variable outside the while loop. If the loop finishes by itself, pkg will contain a trailing newline. Otherwise, it will contain the last value read from the file.
Alternatively, we can drop the cat statement and rewrite the script this way:
while read pkg version; do
echo "Package: $pkg Version: $version";
done < packages.txt
In this example, we performed word splitting on each line. We put the first word of a line into the pkg variable and its corresponding version into the version variable. At the end of the loop, we used a redirect operator to feed the contents of the packages.txt file to the loop.
4. Using the Input File Descriptor Operator “<“
The less-than redirector or input file descriptor operator is used to specify an input source. Using this operator, we can quickly assign a simple value to a variable. By simple, we mean a single line. However, if we want to assign a multiline value to a variable using the redirect operator, our script will behave unexpectedly.
Let’s say we want to save the shebang of our uname.sh script into the shebang variable. We can achieve this using the command:
$ read shebang < <(cat uname.sh)
$ echo "$shebang"
#!/bin/sh
The “<” operator is an input file descriptor that we use to provide an input to a command. In this case, the “< <” are two separate redirections. The syntax “*<(command)*” is known as process substitution. Process substitution is similar to piping the standard output of one command as an input to another command. Therefore, we can redirect the output of the cat process to the shebang variable through the first “<” redirect operator. The shebang variable should contain the first line of the uname.sh file after executing the command.
Similarly, we can use pipes inside in process substitution and assign the evaluated value to a variable:
$ read shell < <(cat uname.sh | cut -d'/' -f3)
$ echo "$shell"
sh
The advantage of using process substitution instead of the command substitution operator “$” is that it returns the line as soon as it’s available. In contrast, the command substitution operator waits for the output to be complete before it can be processed.
5. Using the lastpipe Feature of Bash
Piping read to assign a value to a variable, the shell creates a new sub-shell where the piped command is executed. Therefore, the value is lost, and the variable cannot be used. However, when we use the lastpipe option i****n the recent versions of bash, we can pipe the output of a command, such as echo, to read. The read statement will then assign the output to a variable.
Using the lastpipe feature, the last command is executed in the current shell instead of a new sub-shell. For that reason, the environment retains the value. By default, this option is disabled. We can turn it on using shopt:
$ shopt -s lastpipe
If we’re on an interactive shell, we should also disable job control for the current shell using the set command:
$ set +m
However, we don’t need to disable job control in a script because job control is disabled by default for it.
We’re now ready to use read with pipes. Let’s rewrite our example to read from stdout:
$ uname -a | read kernel_version;
Now, when we echo the kernel_version variable, it should contain the output of the uname command:
$ echo "$kernel_version"
Linux laptop 5.15.5-arch1-1 #1 SMP PREEMPT Thu, 25 Nov 2021 22:09:33 +0000 x86_64 GNU/Linux
6. Command Substitution
The command evaluation or command substitution method is probably the simplest and a neat way to assign an evaluated value to a variable. We can use the command evaluation operator “$()” or backticks to put our commands inside them:
$ ip_address="$(curl -Ls ifconfig.me)"
Let’s break it down:
- We used the curl command inside the command evaluation parenthesis to pingback our IP address from ifconfig.me
- The output or response of the command is then assigned to the ip_address variable
$ echo ip_address
39.112.32.90
We also use pipes inside the expression:
$ ff_ver="$(pacman -Ss firefox | grep "extra/firefox[[:space:]]")"
$ echo "$ff_ver"
extra/firefox 95.0-1 [installed: 94.0.2-2]
Note that we should always use double quotes around the expression to prevent expansion.
Furthermore, if we’re on zsh, we can have nested substitution:
$ echo $(expr length $(curl -Ls ifconfig.me))
12
7. Which One to Use?
There’s almost always a use case for the above methods. For files, we can use a while loop and read the file’s contents into variables. For straightforward tasks, we can use the input redirect operator. If we want to input a large stream of data into a variable, we can use a heredoc or a herestring.
If we’re on bash, we can use the lastpipe option because it’s neat and performant. The downside of lastpipe is that we cannot port our scripts to other shells such as dash. Nevertheless, we should prefer it for internal use where there is no requirement to port our scripts to other platforms.
All that aside, for simple commands and expressions, we should stick to the command evaluation or command substitution method as it is simple and readable. Sometimes, readability trumps efficiency. Moreover, it’s POSIX compliant, so it should work with almost any shell.
8. Conclusion
In this article, we went through the different methods to dynamically assign values to variables. We also made some recommendations for when to use each method.