1. Overview
When we work with the Linux command line, sometimes we want to start a process and let it run in the background as a job.
In this way, it doesn’t block the terminal, and we can do some other work while it’s running.
There are several ways to achieve this. In this tutorial, we’re going to discuss three common approaches: the disown and nohup commands, and the & operator.
2. Linux Job Control in a Nutshell
Let’s get a basic understanding of Linux job control.
2.1. Example Long-running Process
First of all, let’s create a long-running shell script as an example process:
$ cat long_running.sh
#!/bin/bash
while true; do
sleep 3
date +"The Process [$$] says: The current date and time is %F %T"
done
The script is pretty simple. It prints its process id with the current timestamp every three seconds. The special variable $$ holds the process id of the current process.
Once the script starts, it won’t be terminated until we manually stop it, for example, by pressing Ctrl-C.
2.2. Jobs Control in a Nutshell
Let’s look at some job control commands and keyboard shortcuts first:
- Ctrl-C: Kill the process running in the foreground by sending the signal SIGINT
- Ctrl-Z: Suspend the process running in the foreground by sending the signal SIGTSTP
- jobs: Display a list of the jobs with their status
- fg: Move a background job into the foreground
- bg: Resume suspended jobs by running them as background jobs
To understand the commands and keyboard shortcuts above better, we’ll see how they are used through an example.
First, let’s start the example process and redirect the output to a file called /tmp/out:
$ ./long_running.sh > /tmp/out
Good! Our shell script is running in the foreground. Since the script runs permanently, it blocks the terminal.
Now, let’s try to suspend it by pressing Ctrl-Z:
$ ./long_running.sh > /tmp/out
^Z
[1] + 61299 suspended ./long_running.sh > /tmp/out
$
The output shows the job with the PID 61299 is suspended, and we have the command prompt again so that we can input other commands if we like.
Let’s see if it is in the job list using the jobs command:
$ jobs
[1] + suspended ./long_running.sh > /tmp/out
Now, we’ll wait for about half a minute, then resume it using the bg command:
$ bg
[1] + 61299 continued ./long_running.sh > /tmp/out
$
Great! The suspend job is resumed, and it is running in the background. If we check the job list now:
$ jobs
[1] + running ./long_running.sh > /tmp/out
Let’s bring it back to the foreground with the fg command:
$ fg
[1] + 61299 running ./long_running.sh > /tmp/out
It’s running in the foreground again, and it resumes blocking the terminal. We can press Ctrl-C to terminate it.
After we terminate the job, let’s check the output file /tmp/out:
The Process [61299] says: The current date and time is 2020-05-29 15:50:01
The Process [61299] says: The current date and time is 2020-05-29 15:50:04
The Process [61299] says: The current date and time is 2020-05-29 15:50:07
The Process [61299] says: The current date and time is 2020-05-29 15:50:10
The Process [61299] says: The current date and time is 2020-05-29 15:51:00
The Process [61299] says: The current date and time is 2020-05-29 15:51:03
The Process [61299] says: The current date and time is 2020-05-29 15:51:06
...
If we read the output file carefully, we can see the interval between line #4 and line #5 is 50 seconds. In this period, we don’t have any output. This is because we suspended the job by pressing Ctrl-Z.
3. The & Operator
3.1. Introduction to the & Operator
If a command ends with the & operator, the shell will run the command in the background in a subshell. The shell doesn’t wait for it to finish, and returns with status 0.
Let’s give it a try with our long_running.sh script:
$ ./long_running.sh > /tmp/out &
[1] 176933
$ jobs
[1] + running ./long_running.sh > /tmp/out
$ fg
[1] + 176933 running ./long_running.sh > /tmp/out
^C
$ cat /tmp/out
The Process [176933] says: The current date and time is 2020-05-29 21:28:59
The Process [176933] says: The current date and time is 2020-05-29 21:29:02
The Process [176933] says: The current date and time is 2020-05-29 21:29:05
The Process [176933] says: The current date and time is 2020-05-29 21:29:08
The Process [176933] says: The current date and time is 2020-05-29 21:29:11
The Process [176933] says: The current date and time is 2020-05-29 21:29:14
The Process [176933] says: The current date and time is 2020-05-29 21:29:17
The example shows that if we start a process with the & operator, the process will run as a background job. We can do some other work in the foreground in parallel.
3.2. The Output
Just now, when we started the long_running.sh script, we redirected the stdout to a file /tmp/out.
What happens if we don’t redirect its output? Let’s try it:
In the demo, we see that even if the job is running in the background, the output is printed to the terminal. This is because the background process started by the & operator will inherit the stdout and stderr from the shell.
3.3. The SIGHUP Signal
We can start a process with the & operator to make it run in the background. We might ask, will it still run if we close the terminal? Let’s find out.
First, we start the script in the background:
$ ./long_running.sh > /tmp/out &
[1] 184314
$ jobs
[1] + running ./long_running.sh > /tmp/out
$ ps -ef | grep '[l]ong'
kent 184314 55850 0 22:01 pts/1 00:00:00 /bin/bash ./long_running.sh
The output shows that the process is running with the PID 184314. Now we close the current terminal and start a new one, to check if the job is still running:
$ jobs
$ ps -ef | grep '[l]ong'
$
Oops! The job list is empty, and the process is gone, too. Why has that happened? The process tree of our background job process may help to answer the question.
Let’s check the process tree of the background job process using the pstree command:
$ ./long_running.sh > /tmp/out &
[1] 191536
$ pstree -s 191536
systemd───xfsettingsd───urxvt───bash───long_running.sh───sleep
The output shows clearly that our background job process is a sub-process of the shell, which is bash in this case. Moreover, the terminal process, which is urxvt in this example, is the parent process of the shell process.
We know that SIGHUP is used to signal that a terminal hangup has occurred. Let’s understand what happens when we kill the terminal process:
- We close the terminal. The terminal sends a SIGHUP signal to its sub-processes.
- The shell process receives the SIGHUP signal. After that, it sends a SIGHUP signal to its sub-processes.
- As a sub-process of the shell, our background job process receives a SIGHUP signal.
If a process receives a SIGHUP, the default action is to stop executing immediately. Thus, the background running job process is terminated.
It isn’t hard to imagine that, if we want to keep a background job alive after the attached terminal is closed, we should somehow prevent the job process from receiving the SIGHUP signal.
4. The disown Command
4.1. Introduction to the disown Command
The disown command is a built-in command in many modern shells, for example, Bash and Zsh.
The disown command from different shells can behave slightly differently. In this tutorial, we’ll talk about the Bash built-in disown command.
The basic syntax to use this command is:
disown [options] jobID1 jobID2 ... jobIDN
If we don’t give it any options, the disown command will remove the given jobs from the table of active jobs.
A jobID begins with a % character. For example, %x identifies the job x.
Let’s see an example of the default usage of this command:
$ ./long_running.sh > /tmp/out &
[1] 414860
$ jobs -l
[1] + 414860 running ./long_running.sh > /tmp/out
$ disown %1
$ jobs -l
$
After we execute the disown command, the job is removed from the job list. However, the process of the job is still running:
$ ps -ef | grep '[l]ong'
kent 414860 185196 0 21:31 pts/0 00:00:00 /bin/bash ./long_running.sh
The disown command has three options:
- -a: Delete all jobs if no jobID is supplied
- -r: Delete only jobs with status running
- -h: Each job is not removed from the table. Instead, it is marked so that SIGHUP is not sent to the job if the shell receives a SIGHUP
The -h option is an important one. It allows us to keep a background job alive after exiting the terminal or disconnecting the connection to a remote server.
We’re going to discuss this option in detail in a later section.
4.2. The Output
Same as the job started with the & operator, the input and output of the *disown-*ed job won’t be changed. It reuses the stdout and stderr from the shell.
4.3. SIGHUP and the -h Option
We’ve learned that the disown command has three options: -a, -r, and -h. The -h option is a special one.
If we execute disown -h JobID, the job will not be removed from the job list. Instead, it marks the job. When the shell sends a SIGHUP signal, the signal won’t be sent to the marked job. Thus, the marked background job will keep running even if the controlling terminal is closed.
Let’s understand this through an example:
$ ./long_running.sh > /tmp/out &
[1] 546754
$ jobs -l
[1]+ 546754 Running ./long_running.sh > /tmp/out &
$ disown -h %1
$ jobs -l
[1]+ 546754 Running ./long_running.sh > /tmp/out &
As the output shows, after we execute disown -h, the job %1 is still in the job table. Let’s have a look at the process tree of the job process 546754:
$ pstree -s 546754
systemd───xfsettingsd───urxvt───bash───long_running.sh───sleep
$ ps -ef | grep '[l]ong'
kent 546754 1 0 13:53 pts/0 00:00:00 /bin/bash ./long_running.sh
The pstree command’s output shows the background job process is a sub-process of the terminal urxvt, while the output of the ps command shows the process is running in terminal pts/0.
Now, let’s close the terminal:
$ exit
Then we open another terminal, and check if the process 546754 is still alive:
$ ps -ef | grep '[l]ong'
kent 546754 1 0 13:53 ? 00:00:00 /bin/bash ./long_running.sh
$ pstree -s 546754
systemd───long_running.sh───sleep
The ps command tells us that the long_running.sh process is still running. However, if we read it’s output carefully, a question mark “*?”* shows in the TTY column. It means there is no terminal attached to the process.
And the output of the pstree command verifies it — the long_running.sh process is a direct sub-process of the systems process now.
We know that the background job will reuse the stdout and the stderr from the shell.
It’s worthwhile to mention that if we execute disown -h %X, then after the controlling terminal is closed, the outputs – which are written to stdout and stderr by the job X – will be discarded. This is because the shell process attached to the terminal is gone.
5. The nohup Command
The nohup utility is a member of the GNU Coreutils package. Its name stands for “no hup” — that is, it can protect a command from the SIGHUP signal.
5.1. Introduction to the nohup Command
In an earlier section, we’ve learned that disown -h can protect a background job from SIGHUP. The nohup command is quite similar. However, it’s not limited to a background job. We can use the nohup utility on a regular command:
$ nohup ./long_running.sh
nohup: ignoring input and appending output to 'nohup.out'
Because we didn’t add the & operator at the end of the command line, the script will run in the foreground.
Let’s take a look at the process hierarchy of the long_running.sh process:
$ ps -ef | grep '[l]ong'
kent 566207 565675 0 18:46 pts/0 00:00:00 /bin/bash ./long_running.sh
$ pstree -s 566207
systemd───xfsettingsd───urxvt───bash───long_running.sh───sleep
The output of the ps command shows the process id, and it also tells us the process is attached to the terminal pts/0, whereas the pstree output tells us that the process is attached to the terminal process urxvt.
Next, we’ll close the terminal and run the ps and the pstree commands again. Let’s see what they report:
$ ps -ef | grep '[l]ong'
kent 566207 1 0 18:59 ? 00:00:00 /bin/bash ./long_running.sh
$ pstree -s 566207
systemd───long_running.sh───sleep
After we close the terminal, the ps command shows the long_running.sh process is still running but doesn’t attach to a terminal. Then, the output of the pstree command shows that it’s the direct sub-process of the systemd process.
In the above example, we let nohup start a process in the foreground. We can, if we like, start a process with the nohup utility and let it run as a background job. We just add the & operator at the end:
$ nohup ./long_running.sh &
[1] 571215
nohup: ignoring input and appending output to 'nohup.out'
$ jobs -l
[1] + 571215 running nohup ./long_running.sh
If we close the terminal, the long_running.sh process will be still running
5.2. The Output
A process started by the nohup command will not inherit the shell’s stdout and stderr. The nohup command will ignore stdin and will redirect stdout and stderr to a file called nohup.out. Actually, we’ve seen the message in earlier nohup examples:
nohup: ignoring input and appending output to 'nohup.out'
6. Comparing Job Control Approaches
So far, we’ve discussed three approaches to start a background job process and how they handle the SIGHUP signal.
Let’s summarize the strategies we discussed:
Function
stdout/stderr
After Closing the terminal (SIGHUP)
&
Run a command as a background job
Inherited from the shell
The job will be stopped
disown
– Remove jobs in the job list
– Protect jobs from the SIGHUP
– Inherited from the shell
– Discarded if with the -h option and the controlling terminal is closed
The job process will keep running only if with the -h option
nohup
Protect a command from the SIGHUP
Redirected to a nohup.out file
The command process will keep running
7. Conclusion
In this article, we briefly introduced the concept of job control in Linux. Later, we addressed the & operator and the disown and nohup commands.
Also, we’ve discussed how to protect a process from the SIGHUP signal so that it won’t be killed when the attached terminal is closed.