1. Overview
The File Transfer Protocol, also known as FTP, is a TCP protocol that allows for the downloading of files between computers. Basically, it uses the client-server model. The server part, called an FTP daemon, is always listening for FTP requests from remote systems.
In this tutorial, we’re going to see how to download all files from an FTP server. We’ll use an Ubuntu client machine for running the commands discussed in this article. However, most steps should be the same for many Linux distributions.
2. The SFTP Protocol
The Secure File Transfer Protocol (SFTP) is a safe file transfer protocol. It provides a secure, interactive FTP session via an encrypted channel. As SFTP allows for encrypted data transmission, we choose it over FTP. Still, we use SFTP and FTP interchangeably within the article.
Indeed, in the subsequent sections, we’ll use the sftp prefix to access or download files from an FTP server. Of course, this also requires an SFTP server on the other side.
3. Using the lftp Command
lftp is a CLI-based tool for downloading and uploading files between two systems. It can work with many protocols like FTP, HTTP, SFTP, and others.
3.1. Installing lftp
By default, major distributions of Linux contain the lftp package in their official repository. Let’s install it on our Ubuntu system. For Debian-based Linux distributions, like Ubuntu, we can install the lftp package using apt:
$ sudo apt install lftp -y
Aside from the official repository, we can also use source code and precompiled binaries to install lftp.
3.2. Logging Into the FTP Server
Once we have lftp on board, we can use a standard command to connect to our FTP server:
$ lftp -u "$username","$password" $host_IP
Here, we just need to replace $user and $password with the user’s credentials on the FTP server, while $host_IP specifies the name or IP of the FTP server.
Alternatively, we can also use another syntax for logging into the FTP server:
$ lftp sftp://username@host_IP
The username parameter is the name of the user on the remote system, while host_IP is the name or IP of the server.
Let’s try connecting to an FTP server, for example, with IP 192.168.80.201:
$ lftp sftp://[email protected]
Password:
Further, we need to enter the correct password.
3.3. Basic Usage
Once connected to the server, we’ll see the lftp command prompt:
lftp [email protected]:~>
Here, we can now use the various commands. For example, the FTP ls command can be used to list files and directories:
lftp [email protected]:~> ls
drwxr-xr-x 23 remote remote 4096 Mar 8 05:38 .
drwxr-xr-x 3 root root 4096 Nov 8 2021 ..
-rw------- 1 remote remote 161 Mar 8 05:38 .Xauthority
-rw------- 1 remote remote 8966 Mar 8 11:37 .bash_history
...
Now that we’re connected to the FTP server, let’s now look at how to securely download all files from our SFTP server with lftp.
3.4. Downloading Files Using lftp
The pget utility is part of the lftp package. Basically, pget is a command within an lftp session that can download files from FTP servers by utilizing several connections.
For example, let’s download a file named sample.txt from the current directory of our FTP server:
lftp [email protected]:~> pget sample.txt
Similarly, we can download multiple files with a single pget command by specifying the names of the files we want to retrieve:
lftp [email protected]:~/Desktop/test> pget test1.txt test2.txt test3.txt
Total 3 files transferred
In the above snippet, lftp has transferred three files.
3.5. Mirroring
Additionally, we can also get a whole directory from the FTP server. Let’s do this with the mirror command:
lftp [email protected]:~> mirror /home/remote/Desktop/test /home/local/Downloads/
The mirror command copies the full remote directory to the current local directory. For example, here, the contents of the FTP server’s /home/remote/Desktop/test directory are copied to the local machine’s /home/local/Downloads directory.
We can even automate this instead of using an interactive session:
lftp -u "$USER","$PASSWORD" $HOST <<EOF
mirror --use-pget-n=10 $REMOTE_DIR $LOCAL_DIR;
exit
EOF
In this case, we just replace the variables with their respective values:
- $USER: username
- $PASSWORD: user’s password
- $HOST: name or IP of the FTP server
- $REMOTE_DIR: path of the remote directory
- $LOCAL_DIR: path of the local destination directory
In addition, we use a heredoc construct. Basically, this causes the shell to take multi-line input until the initial string (EOF) is encountered at the beginning of a line without any trailing spaces. This enables us to pass the SFTP session commands directly after connecting.
Finally, the parameter –-use-pget-n=10 denotes the number of segments to use per file with pget.
4. Using the curl Command
The curl command can download single and multiple files from an FTP server. To automate the process, we can use the -u or –user flag, followed by username:password, and the path to the desired file.
For example, let’s download a single file with the curl command:
$ curl "sftp://192.168.80.201/home/remote/Desktop/test/test.txt" --user remote:123 -O
Similarly, to download multiple files, we can specify files inside curly brackets:
$ curl "sftp://192.168.80.201/home/remote/Desktop/test/{test.txt,file.pdf}" --user remote:123 -O
The –O option keeps the name of the downloaded file the same as that of the remote file. Alternatively, if we know the URLs of the target files, we can download them all at once. For this, we can use a for loop:
for addr in $files_urls do curl -O "$addr" --user remote:123 done
Here, the variable $files_urls comprises the complete addresses of the target files. Consequently, the above downloads all the specified files to the current directory. Also, we can download files from URLs listed in a file using curl.
5. Using the FileZilla Application
Let’s also explore the steps for downloading files from an FTP server using FileZilla:
- Open FileZilla
- Select Site Manager from the File menu
- Select My Sites
- Select New site
- On the right panel:
– Set the Protocol as SFTP – SSH File Transfer Protocol– Set the Host as the FTP server hostname or IP address
– Keep the Logon Type as Ask for password– Set the User as the correct Linux username - Hit the Connect Button and enter the password when prompted.
Once connected, we can use the FileZilla interface to select and download any and all files. Optionally, we can set the destination directory on our Linux client system. By default, the files will go into the local machine’s current directory.
6. Conclusion
In this article, we discussed how to download all files from an FTP server. Using SFTP, we’ve covered three basic approaches. Overall, FileZilla may be a good choice when looking for a GUI solution. On the other hand, the other methods are better options in a command-line environment.