Java IOException: 'Too many open files'

1. Introduction

A common pitfall when working with files in Java is the possibility of running out of available file descriptors.

In this tutorial, we’ll take a look at this situation and offer two ways to avoid this problem.

2. How the JVM Handles Files

Although the JVM does an excellent job isolating us from the operating system, it delegates low-level operations like file management to the OS.

This means that for each file we open in a Java application, the operating system will allocate a file descriptor to relate the file to our Java process. Once the JVM finishes with the file, it releases the descriptor.

Now, let’s dive into how we can trigger the exception.

3. Leaking File Descriptors

Recall that for every file reference in our Java application, we have a corresponding file descriptor in the OS. This descriptor will be closed only when the file reference instance is disposed of. This will happen during the Garbage Collection phase.

However, if the reference remains active and more and more files are being open, then eventually the OS will run out of file descriptors to allocate. At that point, it will forward this situation to the JVM, which will result in an IOException being thrown.

We can reproduce this situation with a short unit test:

@Test
public void whenNotClosingResoures_thenIOExceptionShouldBeThrown() {
    try {
        for (int x = 0; x < 1000000; x++) {
            FileInputStream leakyHandle = new FileInputStream(tempFile);
        }
        fail("Method Should Have Failed");
    } catch (IOException e) {
        assertTrue(e.getMessage().containsIgnoreCase("too many open files"));
    } catch (Exception e) {
        fail("Unexpected exception");
    }
}

On most operating systems, the JVM process will run out of file descriptors before completing the loop, thereby triggering the IOException.

Let’s see how can we avoid this condition with proper resource handling.

4. Handling Resources

As we said before, file descriptors are released by the JVM process during Garbage Collection.

But if we didn’t close our file reference properly, the collector may choose not to destroy the reference at the time, leaving the descriptor open and limiting the number of files we could open.

However, we can easily remove this problem by making sure that if we open a file, we ensure we close it when we no longer need it.

4.1. Manually Releasing References

Manually releasing references was a common way to ensure proper resource management before JDK 8.

Not only do we have to explicitly close whatever file we open, but also ensure that we do it even if our code fails and throws exceptions. This means using the finally keyword:

@Test
public void whenClosingResoures_thenIOExceptionShouldNotBeThrown() {
    try {
        for (int x = 0; x < 1000000; x++) {
            FileInputStream nonLeakyHandle = null;
            try {
                nonLeakyHandle = new FileInputStream(tempFile);
            } finally {
                if (nonLeakyHandle != null) {
                    nonLeakyHandle.close();
                }
            }
        }
    } catch (IOException e) {
        assertFalse(e.getMessage().toLowerCase().contains("too many open files"));
        fail("Method Should Not Have Failed");
    } catch (Exception e) {
        fail("Unexpected exception");
    }
}

As the finally block is always executed, it gives us the chance to properly close our reference, thereby limiting the number of open descriptors.

4.2. Using try-with-resources

JDK 7 brings us a cleaner way to perform resource disposal. It’s commonly known as try-with-resources and allows us to delegate the disposing of resources by including the resource in the try definition:

@Test
public void whenUsingTryWithResoures_thenIOExceptionShouldNotBeThrown() {
    try {
        for (int x = 0; x < 1000000; x++) {
            try (FileInputStream nonLeakyHandle = new FileInputStream(tempFile)) {
                // do something with the file
            }
        }
    } catch (IOException e) {
        assertFalse(e.getMessage().toLowerCase().contains("too many open files"));
        fail("Method Should Not Have Failed");
    } catch (Exception e) {
        fail("Unexpected exception");
    }
}

Here, we declared nonLeakyHandle inside the try statement. Because of that, Java will close the resource for us instead of us needing to use finally.

5. Conclusion

As we can see, failure to properly close open files can lead us to a complex exception with ramifications all across our program. With proper resource handling, we can ensure this problem will never present itself.

The complete source code for the article is available over on GitHub.