1. Overview
Sometimes, we want to avoid copying a subdirectory and its contents while replicating a directory structure.
In this tutorial, we’ll explore tools that enable us to exclude one or more subdirectories while copying a directory in Linux.
2. Selective Copy Using mkdir and cp
First, let’s create a directory structure for our experiment and preview the contents with tree:
$ mkdir -p /tmp/baeldung/{dir1,dir2,dir3}/{sub1,sub2,sub3}
$ touch > /tmp/baeldung/{dir1,dir3}/{sub1,sub2}/{file1,file2}
$ tree /tmp/baeldung
/tmp/baeldung
├── dir1
│ ├── sub1
│ │ ├── file1
│ │ └── file2
│ ├── sub2
│ │ ├── file1
│ │ └── file2
│ └── sub3
├── dir2
│ ├── sub1
│ ├── sub2
│ └── sub3
└── dir3
├── sub1
│ ├── file1
│ └── file2
├── sub2
│ ├── file1
│ └── file2
└── sub3
12 directories, 8 files
For our example, let’s say we want to copy /tmp/baeldung to /tmp/baeldung-new, but we don’t want to retain /tmp/baeldung/dir1/sub1/ and /tmp/baeldung/dir3/sub2/, and their contents. Notably, there are several empty directories that should also be replicated.
In a naive approach, we can create the desired directory structure using mkdir in the target location first, and subsequently copy relevant files using cp:
$ mkdir /tmp/baeldung-new
$ cd /tmp/baeldung
$ find -path './dir1/sub1' -prune -o -path './dir3/sub2' -prune -o -type d -printf "%P\0" | xargs -0I{} mkdir '/tmp/baeldung-new/{}'
$ find -path './dir1/sub1' -prune -o -path './dir3/sub2' -prune -o -type f -printf "%P\0" | xargs -0I{} cp {} '/tmp/baeldung-new/{}'
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│ ├── sub2
│ │ ├── file1
│ │ └── file2
│ └── sub3
├── dir2
│ ├── sub1
│ ├── sub2
│ └── sub3
└── dir3
├── sub1
│ ├── file1
│ └── file2
└── sub3
10 directories, 4 files
The first command creates the target directory. Then, we change our current working directory to /tmp/baeldung and this location becomes our current working directory throughout the rest of our experiment.
In the third line, we use the find command to exclude the undesired directories using the path parameters and prune action. The output contains a null-delimited list of our desired directories. The xargs command separates each item in the list and creates the directories in the storage using mkdir. A null delimited list enables us to avoid any problem due to whitespace.
Similarly, in the fourth line, we pipe the output of find to xargs and cp to copy the files to our target location. However, as we can see, this is an extensive process, and we can employ better tools to accomplish the same in a more succinct way.
3. Selective Copy Using find and cpio
We can combine the creation of a directory and copying files in a single command using find and cpio.
If the cpio command isn’t available, we need to install it first:
$ sudo apt install cpio
Let’s review the flags that we can use for our purpose:
-p, --pass-through
Pass-through. Read a list of file names from the standard input and copy them to the specified directory.
-0, --null
Filenames in the list are delimited by null characters instead of newlines
-d, --make-directories
Create leading directories where needed.
So, we can filter out undesired directories using the find command and pipe the output to cpio to copy:
$ rm -r /tmp/baeldung-new
$ find -path './dir1/sub1' -prune -o -path './dir3/sub2' -prune -o -print0 | cpio -pd0 /tmp/baeldung-new
1 block
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│ ├── sub2
│ │ ├── file1
│ │ └── file2
│ └── sub3
├── dir2
│ ├── sub1
│ ├── sub2
│ └── sub3
└── dir3
├── sub1
│ ├── file1
│ └── file2
└── sub3
10 directories, 4 files
In this method, we have filtered and copied the directory structure in a single command using find and cpio. This way, we avoid the use of a more complex pipeline and multiple commands.
4. Selective Copy Using tar
We can further reduce dependency on the find command if we use tar. The tar command can create and extract an archive. Furthermore, if we pipe the output of the creation command to the extraction command, there is very little overhead.
First, let’s review the interesting flags:
-c, --create
Create a new archive.
--exclude=PATTERN
Exclude files matching PATTERN
-f, --file=ARCHIVE
Use archive file or device ARCHIVE. If ARCHIVE is "-", the output is redirected to stdout
-C, --directory=DIR
Change to DIR before performing any operations.
-x, --extract, --get
Extract files from an archive
Now, we can create a TAR archive excluding undesired directories and extract them in the new location using the tar command:
$ rm -r /tmp/baeldung-new
$ mkdir /tmp/baeldung-new
$ tar -cf - --exclude './dir1/sub1' --exclude './dir3/sub2' . | tar -xC /tmp/baeldung-new
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│ ├── sub2
│ │ ├── file1
│ │ └── file2
│ └── sub3
├── dir2
│ ├── sub1
│ ├── sub2
│ └── sub3
└── dir3
├── sub1
│ ├── file1
│ └── file2
└── sub3
10 directories, 4 files
As we can see, the tar method looks more efficient. In the second line, we ensure the target location exists before performing the TAR operation. This step isn’t necessary for other methods such as when using cpio.
5. Selective Copy Using rsync
The rsync command provides fast incremental file transfer.
In case it’s not available locally, we can install the utility using our package manager:
$ sudo apt install rsync
The exclude flag in the rsync command enables us to specify directories we do not want to copy:
$ rm -r /tmp/baeldung-new
$ rsync -a --exclude={'dir3/sub2','dir1/sub1'} . /tmp/baeldung-new
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│ ├── sub2
│ │ ├── file1
│ │ └── file2
│ └── sub3
├── dir2
│ ├── sub1
│ ├── sub2
│ └── sub3
└── dir3
├── sub1
│ ├── file1
│ └── file2
└── sub3
10 directories, 4 files
Here, the brace expansion and exclude flag syntax performs our task well. Importantly, a leading slash isn’t required in the parameter to the exclude flag.
6. Selective Copy Using cp and Globbing
As with most scenarios that involve including or excluding more than a single object, we can turn to globbing as a potential solution.
In fact, by using the basic cp with advanced globbing, we can achieve our aims:
$ rm -r /tmp/baeldung-new
$ shopt -s extglob
$ cp --verbose --archive ./!(dir3) /tmp/baeldung-new
'./dir1' -> '/tmp/baeldung-new/dir1'
'./dir1/sub1' -> '/tmp/baeldung-new/dir1/sub1'
'./dir1/sub1/file1' -> '/tmp/baeldung-new/dir1/sub1/file1'
'./dir1/sub1/file2' -> '/tmp/baeldung-new/dir1/sub1/file2'
'./dir1/sub2' -> '/tmp/baeldung-new/dir1/sub2'
'./dir1/sub2/file1' -> '/tmp/baeldung-new/dir1/sub2/file1'
'./dir1/sub2/file2' -> '/tmp/baeldung-new/dir1/sub2/file2'
'./dir1/sub3' -> '/tmp/baeldung-new/dir1/sub3'
'./dir2' -> '/tmp/baeldung-new/dir2'
'./dir2/sub1' -> '/tmp/baeldung-new/dir2/sub1'
'./dir2/sub2' -> '/tmp/baeldung-new/dir2/sub2'
'./dir2/sub3' -> '/tmp/baeldung-new/dir2/sub3'
Of course, after clearing the target directory, we use shopt to enable the extglob setting.
Next, we perform a full archive copy, meaning we preserve all file attributes and duplicate everything recursively. To ensure the correct data goes through, we also add more verbosity.
Importantly, the !(dir3) globbing exclusion ensures we don’t perform any operations on the dir3 subdirectory. Similarly, we can use other globbing expression forms:
- ./!(dir1|dir3): exclude both dir1 or dir3
- ./**/!(sub1): exclude any paths with sub1
- ./**/!(sub1|sub2): exclude any paths with sub1 or sub2
Let’s check the resulting structure for ./!(dir3):
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│ ├── sub1
│ │ ├── file1
│ │ └── file2
│ ├── sub2
│ │ ├── file1
│ │ └── file2
│ └── sub3
└── dir2
├── sub1
├── sub2
└── sub3
9 directories, 4 files
As expected, we don’t see the dir3 top-level directory.
7. Conclusion
In this article, we have explored several methods to copy a subset of a directory structure to the destination.
For minor exclusions, using cp with globbing is usually sufficient. In a minimal Linux installation, the tar method is usually the most suitable. Additionally, tar can also be employed remotely. However, rsync is the as it enables the specification of directories via brace expansion.