1. Overview
Bash iteration statements are simply the repetition of a process within a shell script. These iteration statements or loops are very useful when dealing with a list of elements. Arrays, most of the time, are coupled with loops. Without them, we can’t automate repetitive tasks such as retrieving the contents of a list. There are multiple loop formats available for Bash programmers. Each of these loop types has a certain way to be implemented to iterate over a list of strings until the loop control condition is met.
In this tutorial, we’ll go over the different cases and methods to handle iterating through a list of strings.
2. Iterating Over a Single-Word List
One of the simplest ways to iterate over a list of strings (single words) is to feed the list directly in the for loop statement. Without going into the trouble of declaring a containing variable:
for i in Every journey needs a first step
do
echo $i
done
The above script will result in the following output:
$ bash code.sh
Every
journey
needs
a
first
step
We can notice that the loop iterated the echo command six times. Going over each element of the list.
Another common way is to use a scalar variable. For instance, let’s assign the list above to a variable and use it with our iteration for statement:
list="Every journey needs a first step"
for i in $list; do
echo $i
done
The output will be similar to before. Whitespaces (delimiters) are incorporated into the string variable, and bash will detect each element in the list as a single parameter (word). In later sections, we will explain why bash treated these spaces as delimiters.
As an aside, if the do keyword is used on the same line as for, we should add the semicolon (;) after the list.
3. Iterating Over Arrays
We’ve been able to list an array of single parameters. But how can we achieve the same if dealing with multiple-word strings in each list element?
In case we want to iterate over a list that contains multiple strings in each of its elements. We can use the mentioned above for loop statement with a small tweak. That is, wrapping each list element in quotes:
for i in "Cicada 3301" "VIM 9" "Bouhannana 33" "http request smuggling"
do
echo $i
done
The output of the above script should look just like the following:
$ bash code.sh
Cicada 3301
VIM 9
Bouhannana 33
http request smuggling
In contrast to the previous example, each of the quoted multi-words in the list is treated as a single string.
3.1. Retrieving Strings From an Array Variable
To be more practical, let’s declare an array variable and assign the list above to it:
list=("Cicada 3301" "VIM 9" "Bouhannana 33" "http request smuggling")
for i in $list
do
echo $i
done
If we tried to execute the above script, we would get the subsequent result:
$ bash code.sh
Cicada
3301
Since no index is assigned, the first array element (0 index) is assumed.
As we know by now, array variables can contain multiple values. These arrays are indexed by integers or strings. However, since we didn’t supply the indices manually for the list array variable, bash will do it automatically for us. The index starts at 0; based on that, let’s suppose we want to display the second element of the list array:
list=("Cicada 3301" "VIM 9" "Bouhannana 33" "http request smuggling")
echo ${list[1]}
We should remember to collect the contents of an array or to dereference it. We are required to use curly braces notation.
The script will output the following:
$ bash code.sh
VIM 9
At this point, all we have to do to retrieve the rest of the elements is to incorporate an iteration command.
3.2. Understanding @ and * Special Indices
To iterate over the list array, we can use the @ or the * special indices. These special characters are analogous. Both return all elements of an array. When we use the array reference unquoted with either * or @, word splitting is performed:
list=("Cicada 3301" "VIM 9" "Bouhannana 33" "http request smuggling")
for i in ${list[@]} # OR ${list[*]}
do
echo $i
done
Output:
$ bash code.sh
Cicada
3301
VIM
9
Bouhannana
33
http
request
smuggling
As seen above, bash considered spaces as delimiters.
3.3. Using the * Special Index With Quoting
In contrast, when the array reference is wrapped in double quotes. The * special index will expand to a single string consisting of all our list elements’ values. This means we can print all array values with a single statement:
list=("Cicada 3301" "VIM 9" "Bouhannana 33" "http request smuggling")
echo "${list[*]}"
Let’s check the results:
$ bash code.sh
Cicada 3301 VIM 9 Bouhannana 33 http request smuggling
Our list elements were separated with a white space character. This is the first character in the IFS variable. To change that, we can assign a different value to the IFS environment variable.
3.4. Using the @ Special Index With Quoting
The @ character behaves differently than * when the array reference is quoted. For instance, when using the @ as an index in the previously mentioned manner. Each array element will be expanded to a separate argument. Having said that, using an iteration statement here is the way to list array elements that contain multiple parameters (words):
list=("Cicada 3301" "VIM 9" "Bouhannana 33" "http request smuggling")
for i in "${list[@]}"
do
echo $i
done
Let’s run our script to check the results:
$ bash code.sh
Cicada 3301
VIM 9
Bouhannana 33
http request smuggling
4. Using C Style for Loop Statement
Bash also uses the C programming language for loop form. This statement consists of three control expressions. The first expression is the initializer, the second is the condition, and the third one is the update (new initializer’s value). To dereference our list of strings using this looping method. Surely, we must know the length of the array.
Here comes the # operator, which is helpful in finding the length of arrays. Moreover, any element contained in the array.
So to iterate over a list using a C style for loop. we should use the # operator when referencing an array element (${#array[1]}). Let’s check the following example that will retrieve and print all the contents of an array:
#Previous array
len=${#list[@]} # has a length of 4
for(( i=0; i<$len; i++ ))
do
echo ${list[i]}
done
In the same way, the output should look similar to the previous one.
5. Retrieving Delimiter-Separated Strings
All the variables that we’ve focused on so far have been array variables; that is, they contain multiple elements and values. Contrary to scalar variables, which contain single values.
Bash has the IFS (internal field separator) special shell variable that can help us split string values and retrieve the desired ones. We can set the IFS variable to any character to act as a word separator (token). Suppose we want to iterate over the following string variable and retrieve its elements:
list="Cicada 3301-VIM 9-Bouhannana 33-http request smuggling"
Assigning the hyphen character (–) to the IFS variable will cause word splitting at hyphens. Straightaway, let’s iterate over the separated elements and print them to the screen:
default_IFS="$IFS"
IFS="-"
for i in $lists
do
echo $i
done
It’s a good practice to assign the default IFS value to another variable before changing it.
$ bash code.sh
Cicada 3301
VIM 9
Bouhannana 33
http request smuggling
The for loop above goes through, assigning each of the hyphen delimited strings in list to i. Then, print out those values.
6. Retrieving List of Strings From a File
One of the common ways of reading data from a file; is by using a statement of while read loop and a file input redirection. Before getting into the script example, first, let’s check the list of strings within our file:
$ cat list.txt
Cicada 3301
VIM 9
Bouhannana 33
http request smuggling
We can use read to get the lines from the above file. As the script loops, read assigns each line of the input into the provided variable(s). Let’s check the subsequent code snippet for more understanding:
while read line
do
echo $line
done < list.txt
The while statement will read the provided file line by line in each pass through the loop. As we can see, the path of the text file is provided after the done closing statement. By using the left angle bracket character (<), we can feed the content of our file as input to the while loop.
After executing the command, the output should look like the previous one.
7. Conclusion
In this tutorial, we’ve discussed the most common situations for looping over a list of strings. These are, iterating over array elements of multiple parameters (words), scalar string variables with the help of IFS, and files with multiple lines. Not to mention getting acquainted with important notions and features Such as the * and @ special indices and word-splitting using the IFS (internal field separator) environment variable.