1. Introduction
Limits are a vital part of any computer system. Having them ensures consistent and defined behavior even when processes don’t enforce their own constraints. One category of limits involves the filesystem. In particular, there are restrictions on the number of filesystem objects, as well as their size.
In this tutorial, we explore size limits for files, volumes, and directories for different filesystems. First, we talk about the subdivisions we can encounter when working with storage components. Next, we go over native Linux filesystems, their characteristics, directory size, and other limitations. After that, we turn to more universal filesystem types. Finally, we cover the native storage organization of another major operating system (OS).
We tested the code in this tutorial on Debian 12 (Bookworm) with GNU Bash 5.2.15. It should work in most POSIX-compliant environments unless otherwise specified.
2. Size Containers
In theory, storage can be arbitrarily big. In practice, physical constraints make storage mediums of a limited size. Moreover, these limits are applied to different levels of the storage component.
Further, Linux rarely uses components like a hard disk drive (HDD) and solid-state disk (SSD) without preparing them with some structure.
2.1. Partitioning
To begin with, we separate the empty storage medium into partitions, providing different areas depending on the operating system, data types, files, and others:
$ fdisk --list /dev/sda
Disk /dev/sda: 32 GiB, 34359738368 bytes, 67108864 sectors
Disk model: xdisk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x16bfb916
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 40677375 40675328 19.4G 83 Linux
/dev/sda2 40677376 67108863 26431488 12.6G 5 Extended
/dev/sda5 40679424 61364223 20684800 9.9G 83 Linux
/dev/sda6 61366272 63365119 1998848 976M 82 Linux swap / Solaris
/dev/sda7 63367168 63913983 546816 267M 83 Linux
/dev/sda8 63916032 67108863 3192832 1.5G 83 Linux
Here, we employ the fdisk command to –list all partitions on drive /dev/sda.
The partition table is very much like the interior walls within a house that separate the rooms. Exactly like walls, this separation takes up space, albeit not very much.
2.2. Apply Filesystems
Next, we usually format each partition with a filesystem. The formatting process provides even more structure to an empty partition:
- indexing table, e.g., inode database, for metadata
- blocks and block allocation for data
- minimum allocation unit
Again, this process reduces the available storage, since the formatting structure itself takes away from the free space.
Importantly, the size limits set on sub-containers like directories are dictated by the filesystem type. So, let’s look at specific constraints different filesystems impose.
3. ext* Filesystems
The ubiquitous ext* filesystems have had several versions over the years:
Actually, one of the main differences between them comes down to the object counts and sizes they support.
3.1. ext2
To begin with, some constraints of the older ext2 depend on the chosen Block Size, i.e., the minimum allocation unit:
-----------------------------------------------------------------------------------------+
| Block Size [KiB] | Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|------------------+-------------+---------------+--------------+-----------+-------------|
| 1 | | | | 16GiB | 4TiB |
|------------------| | | |-----------+-------------|
| 2 | | | | 256GiB | 8TiB |
|------------------| 10^18 | ~ 1.3 * 10^20 | 31998 |-----------+-------------|
| 4 | | | | 2TiB | 16TiB |
|------------------| | | |-----------+-------------|
| 8 | | | | 2TiB | 32TiB |
+-----------------------------------------------------------------------------------------+
Here, we have the block size in the first column, while the next columns provide different hard maximum limits.
So, from the total number of files in a directory Files Dir and the maximum File Size, we can extrapolate the maximum theoretical directory size to be between around 1.3 * 10^20 * 16 and 1.3 * 10^20 * 2^11 GiB. Yet, since these limits are way over the Volume Size, the practical constraints are much less and near the Volume Size bottleneck.
In fact, having too many subdirectories or files within a directory can severely degrade the performance to the point of the system becoming unusable in certain cases.
3.2. ext3
Next, the ext3 filesystem improves on some limitations of ext2:
+-------------------------------------------------------------------------------------------+
| Block Size [KiB] | Total Files | Files Dir | Subdirs Dir | File Size | (Vol)ume Size |
|------------------+-------------+---------------+--------------+-----------+---------------|
| 1 | | | | 16GiB | 4TiB |
|------------------| | | |-----------+---------------|
| 2 | min( | | | 256GiB | 8TiB |
|------------------| Vol / 2^13, | ~ 1.3 * 10^20 | 31998 |-----------+---------------|
| 4 | block count | | | 2TiB | 16TiB |
|------------------| ) | | |-----------+---------------|
| 8 | | | | 2TiB | 32TiB |
+-------------------------------------------------------------------------------------------+
Still, despite the increased limits, we’re in the same situation as with ext2: the (Vol)ume Size decides the upper bound for a directory size.
3.3. ext4
When it comes to ext4, the block size doesn’t have an effect to the overall limits:
+-----------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+-----------+-------------|
| 2^32 - 1 | unlimited | unlimited | 2^44b - 1 | 2^48b - 1 |
| | | | 16TiB - 1 | 256TiB - 1 |
+-----------------------------------------------------------------+
Naturally, again, with an unlimited number of files in a directory, the final size of the latter only depends on the Volume Size regardless of the File Size.
4. XFS Filesystem
The kernel-supported XFS is also a classic choice when it comes to native Linux filesystems.
Due to its focus and internal implementation, XFS has noticeably different limits:
+-----------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+-----------+-------------|
| 2^64 | 2^64 | 2^64 | 2^63b - 1 | 2^63b - 1 |
| | | | 8EiB - 1 | 8EiB - 1 |
+-----------------------------------------------------------------+
Similar to the case with ext*, we end up with the upper limit of a directory being the Volume Size, which in this case is around 8EiB.
5. File Allocation Table (FAT32) Filesystem
Although not tailored or native to Linux, the ubiquitous File Allocation Table (FAT32) filesystem plays a key role as it’s fairly simple, universally supported, and easier to implement.
However, these characteristics come with a cost:
+-----------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+-----------+-------------|
| ~ 2^28 | 2^16 - 1 | 2^16 - 1 | 4GB | 2-16TB |
| 268173300 | 65535 | 65535 | | |
+-----------------------------------------------------------------+
With one of the lowest Total Files counts and the absolute lowest File Size, the maximum directory size of a FAT32 structure is again limited by the Volume Size, which varies according to sectors and clusters but is very low overall.
6. exFAT
The Extensible File Allocation Table (exFAT) aimed to upgrade and replace the now outdated FAT32 filesystem, particularly for USB devices.
In many ways, exFAT managed to do that. However, it wasn’t as widely adopted despite its relatively more complex implementation as well as the already-established FAT32 components in many systems, especially embedded ones.
Still, the benefits of exFAT in terms of its characteristics are obvious:
+-----------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+-----------+-------------|
| unlimited | > 2^21 | > 2^21 | 128PB | 128PB |
| | 2796202 | 2796202 | | |
+-----------------------------------------------------------------+
In this case, the number of objects in a directory is again limited, but much higher than that for FAT32. Yet, the total file count is unlimited. So, an exFAT directory size is again only really constrained by the Volume Size.
7. New Technology File System (NTFS)
As the filesystem native to Microsoft Windows, the New Technology File System (NTFS) imposes certain restrictions but manages to compete with Linux filesystems:
+----------------------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+---------------+--------------------|
| ~ 2^32 - 1 | 2^16 - 1 | 2^16 - 1 | 2^44 - 26 | 2^32 - 1 clusters |
| 4294967295 | 65535 | 65535 | 16TiB - 64KiB | 256TiB - 64KiB |
+----------------------------------------------------------------------------+
In this context, a cluster or block is 64KiB. Critically, there are different versions but we’re only looking at the most recent version of NTFS, since, similar to Linux, older ones are now rarely used in special cases.
Further, NTFS has two theoretical limits:
- file size limit of 2^64 – 26 bytes, i.e., 16EiB – 64KiB
- volume size limit of 2^64 – 1 clusters, i.e., 16EiB – 64KiB
Again, we see the same pattern: Volume Size deciding the maximum directory size.
8. Summary
In this article, we talked about different major filesystems, their constraints, and the maximum directory size of each.
In conclusion, we established that the main limitations for a directory are the permitted maximum volume size and the maximum number of files per directory for the filesystem.