1. Introduction
We commonly use SSH to establish a secure connection to remote Linux machines. Moreover, SSH allows us to run commands and interact with the server as if we’re physically sitting in front of it.
People often believe that their programs running on the remote machine are killed right away when they get disconnected from an SSH session. However, this is a common misconception.
In this tutorial, we’ll explore this misconception and understand what really happens when we disconnect an SSH session.
2. What Happens When an SSH Session Is Disconnected?
Let’s first take a closer look at how SSH works to understand what happens when an SSH session disconnects.
When we establish an SSH connection to a remote server, we’re essentially opening a shell session on that server. The SSH connection ties to the shell session. Hence, when we disconnect from the session, the shell session is terminated.
We should note that the shell session’s termination doesn’t automatically imply the termination of any programs or processes initiated within it.
In fact, these processes continue to run even after we disconnect from the SSH session. Meanwhile, they wait for the hangup signal, SIGHUP, for some time. The programs will no longer be tied to the shell session, which means that we cannot interact with them directly.
To demonstrate this, let’s run a simple Python program on a remote server using SSH:
$ ssh remote-server
$ python my_program.py
Assuming that my_program.py is a long-running process, if we get disconnected from the SSH session, the program will continue to run on the remote server. We can confirm this by logging back into the server and checking the process list:
$ ssh remote-server
$ ps aux | grep my_program.py
user 12345 0.0 0.2 123456 7890 pts/0 S+ 10:00 0:00 python3 my_program.py
In the above example, we used the ps command with aux to show all processes for all users, display the process’s user, and show processes not attached to a terminal. From the output, we can conclude that the program isn’t killed, however, it’s placed in sleep mode, as indicated by the S+ symbol.
On the other hand, if the SIGHUP signal is sent, then there would be no output for the ps command.
3. Keeping Programs Running
In this section, we’ll look at two different ways to further illustrate what happens when we disconnect from an SSH session and how to keep our programs running. Particularly, we’ll discover how to use the nohup and tmux commands for this purpose.
3.1. The nohup Command
First, let’s create a simple Bash script, long_running_program.sh, that runs a long-running program:
#!/bin/bash
while true
do
echo "Program is still running"
sleep 1
done
This script simply echoes a message every second and continues to run indefinitely until stopped.
Now, let’s SSH into a remote server and run the script:
$ ssh remote-server
./long_running_script.sh
The program keeps running in the background after disconnecting from the SSH session. We already confirmed this using the ps command earlier. In essence, the process is still running, even though we disconnected from the SSH session.
Consequently, let’s use the nohup command to run the script in the background and detach it from the current shell session:
$ ssh remote-server
nohup ./long_running_script.sh &
exit
In this example, we run the script using nohup and the & symbol to run it in the background. This allows the program to continue running even after we exit the SSH session. We can confirm that the program is still running by logging back into the server and checking the process list:
$ ssh remote-server
$ ps aux | grep my_program.py
user 12345 0.0 0.2 123456 7890 pts/0 S+ 10:00 0:00 python3 my_program.py
As we can see, the process continues to run in the background, and it’ll remain unaffected by any disconnection.
3.2. The tmux Command
We frequently use terminal multiplexers to maintain the continuity of our shell environments in case of disconnections. They enable us to detach from our shell processes so that we can later rejoin them, regardless of whether the disconnection was intentional or accidental.
A tmux session is a virtual terminal that allows us to run multiple terminal sessions within a single terminal window. Moreover, it’s useful when we need to keep multiple shell sessions open simultaneously.
Now, let’s use the tmux terminal multiplexer to run the script in a separate session:
$ ssh remote-server
tmux new-session -s my_session
./long_running_script.sh
In this example, we start a new tmux session with the name my_session. Furthermore, we use the -s option to define the session name.
When we disconnect from the SSH session, the tmux session and the program continue to run in the background. We can reconnect to the server and reattach to the tmux session to interact with the program using the -t option:
$ ssh remote-server
tmux attach -t my_session
Here, the second line reattaches the tmux session named my_session. Therefore, it allows us to return to the previously created tmux session that we’d detached from.
If we have multiple tmux sessions running, we can use the -t option to specify which session to reattach to.
4. Conclusion
In this article, we learned that disconnecting from an SSH session doesn’t result in the termination of programs running on the remote machine.
Using the nohup command, or a terminal multiplexer like tmux, allows us to run programs in the background. Furthermore, it detaches them from the current shell session. This allows them to continue running even after we disconnect from the SSH session.