1. Overview
SCP or Secure Copy Protocol is a file transfer protocol that is used to securely transfer files from one host to another. But as scp only supports full file transfers, not partial transfers, there isn’t a way to resume downloading a partially downloaded file.
In this tutorial, we’ll take a look at how to resume copying files in Linux.
2. Using rsync
rsync or Remote Synchronization is a utility for Unix-like systems. It lets us efficiently synchronize files and directories both to local and remote hosts. It does so by comparing file sizes and modification times. It’s typically used for synchronizing files or directories between two different systems.
Once the rsync command is run, it uses SSH to connect to the remote host. After successfully connecting, it invokes the remote rsync process. Then the two negotiate which parts of which files need to be transferred. They do this by comparing the files.
2.1. Uploading
This command will send the file at /home/dungbael/file.txt to baeldung.com using the user dungbael. The file will be uploaded to the path of /home/baeldung:
$ rsync -P /home/dungbael/file.txt [email protected]:/home/baeldung
The option -P is there to make sure that rsync will keep any partially transferred files if the transfer is interrupted. Keeping the partial file would make the subsequent transfer much faster. If the transfer is interrupted, we just use the same command to continue from where it left off.
2.2. Downloading
Instead of uploading a file to a remote host, let’s download it from the remote instead.
This command will download the file at*/home/baeldung/file.txt* to our localhost in /home/dungbael/:
$ rsync -P [email protected]:/home/baeldung/file.txt /home/dungbael
As with uploading, if the transfer is interrupted, we can just use the same command to continue from where it left off.
3. Using SFTP
On Unix-like systems, SFTP is a command-line interface for using the Secure File Transfer Protocol. It’s an encrypted version of FTP. Authentication is always needed since all SFTP connections are encrypted.
This solution is useful in case our server doesn’t allow login via SSH. In that case, using rsync wouldn’t work.
3.1. Uploading
This command uploads the file named file.txt to a remote server to directory /home/baeldung:
$ echo "put -a /home/dungbael/file.txt" | sftp -r [email protected]:/home/baeldung
The option -a will resume partial transfers of existing files.
3.2. Downloading
This copies the remote directory of /home/dungbael to our current local directory. Directory dungbael will be created in our current directory:
$ echo "get -a /home/dungbael/" | sftp -r [email protected]
4. Handling Stalled scp Processes
While we have looked at some workarounds for resuming aborted copy processes, scp may still be our preferred solution for file copying. Let’s look at why it stalls and how to fix it.
4.1. Preventing scp Processes from Stalling
scp processes get stalled from time to time. This is because while scp is transferring files, it is trying to use as much bandwidth as possible. This means that a delay, for example, by a firewall or a busy network, can make it stall. In order to stop this from happening, we could try limiting the allocated bandwidth:
$ scp -l 8192 /home/dungbael [email protected]:/home/baeldung
This command, along with the option -l, will help to limit the bandwidth to 1 MB/s (8192 Kbit/s).
4.2. Resuming a Hung scp Process
If we have a stalled scp process currently open, the following might help us without the need to stop the file transfer.
First, we’ll need to press the combination ctrl + z. This will pause the process and keep it in the background.
Next, we’ll need to ssh to the receiving server, login, and then exit. Now we’ll need to use the command fg to bring the background job in the current shell to the foreground. After that, scp should resume.
5. Conclusion
In this article, we covered various ways how to resume the transfer of files from one host to another.
This includes both downloading a file from a remote host and uploading a file to a remote host.
We also looked at how to try to resume a stalled scp process and how to prevent the stalling from ever happening.