1. Overview
Sometimes we need to execute sequences of multiple commands in Linux (command chaining) to concatenate their results to achieve a very specific task. For instance, downloading a file, uncompress it, and delete the compacted file.
In this brief tutorial, we’ll see some ways for serializing commands using bash in Linux. Even when some of them are already running.
2. Inline Command Chaining
The bash syntax allows great flexibility and power to the user. We can craft long sequences of commands to perform the most diverse duties:
<command1> <arguments1> <chaining operator> <command2> <arguments2> ...
There are several chaining operations, let’s take a look at the most used:
- ; (semicolon) – The command on the right executes after the one on the left, regardless of its result
- & (ampersand) – The command on the left will run in the background, and the command on the right starts immediately
- | (pipe) – The left command’s standard output pipes into the right command’s standard input
- && (AND) – The command on the right executes if the one on the left was successful (zero return code)
- || (OR) – The command on the right executes if the one on the left was not successful (non-zero return code)*
It is also possible to use statement chains circled by parenthesis to define blocks to be executed in a subshell:
( <command1> && <command2> ) & ( <command3> && <command4> ) &
This would run two sequences of commands in parallel. In each, the first command result conditions the last.
3. Command Chaining with Already Running Processes
Let’s see how we can use the wait program to execute a command after another already running. This utility is a wrapper for the system call with the same name.
It receives a list of pids (process identification number) as its arguments. Consequently, it hangs the shell until all of the processes are finished. If no arguments are informed, will wait for all background process of the current shell.
If we do not know the pid of a running process you can use the pidof command to retrieve the pid of all running processes with the same name. For instance, we’ll start three background 10-second sleeps, and wait for them afterward:
$ sleep 10 &
[1] 26176
$ sleep 10 &
[2] 26178
$ sleep 10 &
[3] 26179
$ wait $(pidof sleep) && echo Waited for 3 10-second sleeps
[1] Done sleep 10
[2]- Done sleep 10
[3]+ Done sleep 10
Waited for 3 10-second sleeps
The same result of pidof can be achieved using some command chaining together with the help of ps, grep, tr & cut. Just for the kicks, it’d go like this:
$ wait $(ps -ef | grep <process name> | tr -s ' ' | cut -f 2 -d ' ' ) && echo Waited for 3 10-second sleeps
The command chain shown above is another good example of how we can chain commands to achieve a specific result. The construct $() means that the result of the block inside the parenthesis will be used as a parameter for the wait command. In this case:
- ps -ef – gathers all processes with details
- grep
– selects from the ps -ef output the lines containing - tr -s ‘ ‘ – remove, from grep‘s output any duplicated occurrences of ‘ ‘ (space)
- cut -d ‘ ‘ -f 2 – select from tr output the 2nd field using ‘ ‘ (space) as the field delimiter. In this case, the pid of the processes listed and filtered on the previous steps
4. Limitations of the wait Command
As for the wait return code, wait tries to return the exit code of the last command it was waiting for. However, it is really not as reliable, if we use a pid of an already finished command, for instance, it can’t return its actual result. Also, with return codes greater than 128, there is no way to know if the application ended due to a signal or on its own.
Lastly, another limitation of wait is that it can only wait for applications started on the same shell session. If we need to wait for processes of other users or opened in other bash sessions, we’ll have to use other less direct ways like a pidof loop:
while ( pidof -q <command_to_wait> ); do sleep 1; done && <command_to_execute_after>
This example serves as a more general and less powerful substitute for wait. It depends on a sleeping time and cannot evaluate the target process’s return code. Please note that using sleep between iterations on shell loops is always a good practice to properly give back CPU cycles to the system, especially when used commands end too quickly.
5. Conclusion
In this tutorial, we’ve seen some common ways to use command chaining in bash: both inline and waiting for other running processes. However, there are even more options and flexibility available. We can even associate these techniques with user-defined functions, script files, conditionals, or loops.