1. Overview
In this tutorial, we’ll take a look at how to leverage the Apache Commons Net library to interact with an external FTP server.
2. Setup
When using libraries, that are used to interact with external systems, it’s often a good idea to write some additional integration tests, in order to make sure, we’re using the library correctly.
Nowadays, we’d normally use Docker to spin up those systems for our integration tests. However especially when used in passive mode, an FTP server isn’t the easiest application to run transparently inside a container if we want to make use of dynamic port mappings (which is often necessary for tests being able to be run on a shared CI server).
That’s why we’ll use MockFtpServer instead, a Fake/Stub FTP server written in Java, that provides an extensive API for easy use in JUnit tests:
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>3.6</version>
</dependency>
<dependency>
<groupId>org.mockftpserver</groupId>
<artifactId>MockFtpServer</artifactId>
<version>2.7.1</version>
<scope>test</scope>
</dependency>
It’s recommended to always use the latest version. Those can be found here and here.
3. FTP Support in JDK
Surprisingly, there’s already basic support for FTP in some JDK flavors in the form of sun.net.www.protocol.ftp.FtpURLConnection.
**However, we shouldn’t use this class directly and it’s instead possible to use the JDK’s java.net.URL class as an abstraction.
This FTP support is very basic, but leveraging the convenience APIs of java.nio.file.Files, it could be enough for simple use cases:
@Test
public void givenRemoteFile_whenDownloading_thenItIsOnTheLocalFilesystem() throws IOException {
String ftpUrl = String.format(
"ftp://user:password@localhost:%d/foobar.txt", fakeFtpServer.getServerControlPort());
URLConnection urlConnection = new URL(ftpUrl).openConnection();
InputStream inputStream = urlConnection.getInputStream();
Files.copy(inputStream, new File("downloaded_buz.txt").toPath());
inputStream.close();
assertThat(new File("downloaded_buz.txt")).exists();
new File("downloaded_buz.txt").delete(); // cleanup
}
Since this basic FTP supports is already missing basic features like file listings, we are going to use FTP support in the Apache Net Commons library in the following examples.
4. Connecting
We first need to connect to the FTP server. Let’s start by creating a class FtpClient.
It will serve as an abstraction API to the actual Apache Commons Net FTP client:
class FtpClient {
private String server;
private int port;
private String user;
private String password;
private FTPClient ftp;
// constructor
void open() throws IOException {
ftp = new FTPClient();
ftp.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
ftp.connect(server, port);
int reply = ftp.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftp.disconnect();
throw new IOException("Exception in connecting to FTP Server");
}
ftp.login(user, password);
}
void close() throws IOException {
ftp.disconnect();
}
}
We need the server address and the port, as well as the username and the password. After connecting it’s necessary to actually check the reply code, to be sure connecting was successful. We also add a PrintCommandListener, to print the responses we’d normally see when connecting to an FTP server using command line tools to stdout.
Since our integration tests will have some boilerplate code, like starting/stopping the MockFtpServer and connecting/disconnecting our client, we can do these things in the @Before and @After methods:
public class FtpClientIntegrationTest {
private FakeFtpServer fakeFtpServer;
private FtpClient ftpClient;
@Before
public void setup() throws IOException {
fakeFtpServer = new FakeFtpServer();
fakeFtpServer.addUserAccount(new UserAccount("user", "password", "/data"));
FileSystem fileSystem = new UnixFakeFileSystem();
fileSystem.add(new DirectoryEntry("/data"));
fileSystem.add(new FileEntry("/data/foobar.txt", "abcdef 1234567890"));
fakeFtpServer.setFileSystem(fileSystem);
fakeFtpServer.setServerControlPort(0);
fakeFtpServer.start();
ftpClient = new FtpClient("localhost", fakeFtpServer.getServerControlPort(), "user", "password");
ftpClient.open();
}
@After
public void teardown() throws IOException {
ftpClient.close();
fakeFtpServer.stop();
}
}
By setting the mock server control port to the value 0, we’re starting the mock server and a free random port.
That’s why we have to retrieve the actual port when creating the FtpClient after the server has been started, using fakeFtpServer.getServerControlPort().
5. Listing Files
The first actual use case will be listing files.
Let’s start with the test first, TDD-style:
@Test
public void givenRemoteFile_whenListingRemoteFiles_thenItIsContainedInList() throws IOException {
Collection<String> files = ftpClient.listFiles("");
assertThat(files).contains("foobar.txt");
}
The implementation itself is equally straightforward. To make the returned data structure a bit simpler for the sake of this example, we transform the returned FTPFile array is transformed into a list of Strings using Java 8 Streams:
Collection<String> listFiles(String path) throws IOException {
FTPFile[] files = ftp.listFiles(path);
return Arrays.stream(files)
.map(FTPFile::getName)
.collect(Collectors.toList());
}
6. Downloading
For downloading a file from the FTP server, we’re defining an API.
Here we define the source file and the destination on the local filesystem:
@Test
public void givenRemoteFile_whenDownloading_thenItIsOnTheLocalFilesystem() throws IOException {
ftpClient.downloadFile("/buz.txt", "downloaded_buz.txt");
assertThat(new File("downloaded_buz.txt")).exists();
new File("downloaded_buz.txt").delete(); // cleanup
}
The Apache Net Commons FTP client contains a convenient API, that will directly write to a defined OutputStream. This means we can use this directly:
void downloadFile(String source, String destination) throws IOException {
FileOutputStream out = new FileOutputStream(destination);
ftp.retrieveFile(source, out);
}
7. Uploading
The MockFtpServer provides some helpful methods for accessing the content of its filesystem. We can use this feature to write a simple integration test for the uploading functionality:
@Test
public void givenLocalFile_whenUploadingIt_thenItExistsOnRemoteLocation()
throws URISyntaxException, IOException {
File file = new File(getClass().getClassLoader().getResource("baz.txt").toURI());
ftpClient.putFileToPath(file, "/buz.txt");
assertThat(fakeFtpServer.getFileSystem().exists("/buz.txt")).isTrue();
}
Uploading a file works API-wise quite similar to downloading it, but instead of using an OutputStream, we need to provide an InputStream instead:
void putFileToPath(File file, String path) throws IOException {
ftp.storeFile(path, new FileInputStream(file));
}
8. Conclusion
We’ve seen, that using Java together with the Apache Net Commons allows us, to easily interact with an external FTP server, for read as well as write access.
As usual, the complete code for this article is available in our GitHub repository.