1. Overview
Secure File Transfer Protocol (SFTP) is a commonly used protocol for transferring files between a client and a server over a secure channel. In Linux, we can use the sftp command to start an SFTP session and manage files on a remote server.
In this tutorial, we’ll explore how to remove multiple files using sftp in a Linux environment.
2. Sample Task
Let’s suppose we have a server on our local area network (LAN) with an IP address of 192.168.0.104. Furthermore, the 192.168.0.104 machine has the openssh-server package installed, enabling both SSH and SFTP connections.
We also assume that we’ve already generated an SSH key pair and copied our public key to the server’s ~/.ssh/authorized_keys file.
The server has three files located in the dir subdirectory under the ~/Documents path.
From the client side, we can connect to the server via the sftp command and list the files using ls:
$ sftp [email protected]:Documents/dir
Connected to 192.168.0.104.
Changing to: /home/sysadmin/Documents/dir
sftp> ls
file1 file2 file3
In this case, we connect to the server at the ~/Documents/dir location using the sysadmin username. Then, we issue the ls command at the sftp prompt. Consequently, we see that the three files in dir are named file1, file2, and file3.
Our objective is to remove these files conveniently, i.e., without having to do so manually one by one.
If we try to use the rm command at the sftp prompt to delete all the files by listing them, we see that only the first file gets deleted:
sftp> rm file1 file2 file3
Removing /home/sysadmin/Documents/dir/file1
This occurs because the rm command expects only one file path at the sftp prompt and ignores any remaining ones.
So, let’s explore how we can remove multiple files over SFTP.
3. Using Globbing
One way we can remove all files at once is by using globbing within the sftp environment:
sftp> rm file*
Removing /home/sysadmin/Documents/dir/file1
Removing /home/sysadmin/Documents/dir/file2
Removing /home/sysadmin/Documents/dir/file3
This approach works well if the files we wish to remove are the only ones the globbing pattern returns. Otherwise, we’d end up deleting more files, potentially causing data loss.
4. Iteration and Piping to sftp
Another approach is to use a for loop to iterate over the list of files and remove each one sequentially by piping an rm command to sftp:
$ files="file1 file2 file3"
$ for file in $files; do echo "rm $file" | sftp [email protected]:Documents/dir; done
Connected to 192.168.0.104.
Changing to: /home/sysadmin/Documents/dir
sftp> rm file1
Removing /home/sysadmin/Documents/dir/file1
Connected to 192.168.0.104.
Changing to: /home/sysadmin/Documents/dir
sftp> rm file2
Removing /home/sysadmin/Documents/dir/file2
Connected to 192.168.0.104.
Changing to: /home/sysadmin/Documents/dir
sftp> rm file3
Removing /home/sysadmin/Documents/dir/file3
One disadvantage of this method is that we need to initiate as many SFTP connections as there are files we want to remove. In other words, removing each file in this case requires a separate SFTP connection, which can cause a lot of overhead.
5. Using a Batch File
Alternatively, instead of initiating multiple SFTP connections, we can use the sftp -b option to specify a batch file for use over a single connection. Batch files contain commands to execute automatically as a script upon connecting to the remote destination.
First, we set up the batch file by listing the commands we want to run over SFTP, one per line:
$ files="file1 file2 file3"
$ for file in $files; do echo "rm $file" >> batch_file; done
$ cat batch_file
rm file1
rm file2
rm file3
In this case, we use a for loop to iterate over each filename and append it to the rm command. Then, we add each resulting line to the batch file, which we’ve named batch_file.
Finally, we execute the sftp command while specifying the batch file with the -b option:
$ sftp -b batch_file [email protected]:Documents/dir
sftp> rm file1
sftp> rm file2
sftp> rm file3
Consequently, we see that each rm command listed in the batch file was executed successfully. Importantly, we can use this approach to non-interactively execute any number of commands in an SFTP session.
6. Using ssh
If using sftp isn’t a requirement for removing the files, we can instead opt for a single rm command issued over SSH:
$ ssh [email protected] 'cd Documents/dir; rm file{1..3}'
In this case, we connect to the server via ssh and issue the commands enclosed in quotes. In particular, we first cd to the ~/Documents/dir directory and then remove the files with rm while using brace expansion to list the files.
Importantly, when using the ssh command, we can list multiple files to remove with rm, unlike the case at the sftp prompt. The reason behind this is the direct access to the shell.
7. Conclusion
In this article, we explored several ways to remove multiple files over SFTP. In particular, we can use globbing if the files we wish to remove can be exclusively listed by globbing. Otherwise, we can iterate over the filenames in a for loop, and delete each one over a separate SFTP connection.
A better approach is to specify a batch file using the -b option when using the sftp command. This allows the execution of multiple commands within a single SFTP connection. Lastly, if using sftp isn’t a requirement, we can delete multiple files with rm using the ssh command instead.