1. Overview
File copying is a common file operation when we work with the Linux command line. Usually, we pick the cp command to copy files.
In this tutorial, we’re going to discuss how to recursively copy a directory to an existing directory with or without overwriting.
2. Introduction to the Problem
First of all, we need to understand what “copy a directory to an existing directory” exactly means in this problem.
An example can explain it clearly:
$ tree -a
.
├── src
│ ├── .hidden.file
│ ├── srcFile.txt
│ └── subSrc
│ └── subSrcFile.txt
└── target
└── originalTarget.file
3 directories, 4 files
Here, as the output above shows, we have two directories. Under the src directory, there are some files and a sub-directory. Moreover, we have a dotfile .hidden.file.
Now, we want to copy the src directory to the target directory.
If we copy with overwriting, after the copy operation, we want the target directory to contain exactly the same content as the src directory:
.
├── src
│ ├── .hidden.file
│ ├── srcFile.txt
│ └── subSrc
│ └── subSrcFile.txt
└── target
├── .hidden.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
However, if we copy without overwriting, we want everything under the src directory to be recursively copied to the target directory. Also, the original files under target must be untouched:
.
├── src
│ ├── .hidden.file
│ ├── srcFile.txt
│ └── subSrc
│ └── subSrcFile.txt
└── target
├── .hidden.file
├── originalTarget.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
Since the target directory exists already, we cannot simply use the command cp -r src target to do this job. That is because this command will copy the whole src directory under target as a subdirectory: /target/src/. This isn’t what we really want.
Next, we’ll address how to achieve our objective using two different approaches:
- Using the cp command
- Using the rsync command
3. Using the cp Command
We’ll first have a look at how to solve the problem using the cp command since it is pretty common.
3.1. cp Without Overwriting the target Directory
We understand that if the target is an existing directory, we cannot use the simple command cp -r src target to solve this problem.
However, cp provides a nice -T option to treat the destination as a normal file instead of a directory.
That is, if we combine the -rT options, the cp command will recursively copy the content under the src directory to the target directory:
$ cp -rT src target
$ tree -a
.
├── src
│ ├── .hidden.file
│ ├── srcFile.txt
│ └── subSrc
│ └── subSrcFile.txt
└── target
├── .hidden.file
├── originalTarget.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
4 directories, 7 files
tree‘s output shows that we’ve copied everything under the src directory, including the dotfile, to the target directory. And the src directory itself is not copied. Thus, we’ve solved the problem.
3.2. Bash Globbing Trick
We’ve solved the problem using the cp command with -rT options.
If we reconsider the problem, the requirement is only to copy the content under src. Therefore, we may come up with the command cp -r src/* target.
Indeed, this is another possibility to solve the problem. However, there is a little flaw in the command. First, let’s clean the target directory and give it a try, and see what will happen:
$ cp -r src/* target
$ tree -a
.
├── src
│ ├── .hidden.file
│ ├── srcFile.txt
│ └── subSrc
│ └── subSrcFile.txt
└── target
├── originalTarget.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
4 directories, 6 files
As the tree output shows, almost everything under src has been copied to the target directory, except for the dotfile. This is because the default globbing in Bash does not include filenames starting with a dot (“.”).
We have two ways to solve it. One way is to change the default setting and make the globbing include dotfiles. We can use the shopt -s dotglob command to achieve that:
$ ( shopt -s dotglob; cp -r src/* target )
$ tree -a target
target
├── .hidden.file
├── originalTarget.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
1 directory, 4 files
$ shopt dotglob
dotglob off
As we can see in the output above, the dotfile has also been copied. Also, we wrap the shopt command together with the cp command in “(….)” to make them run in a subshell since we want the shopt command only to affect the single cp command.**
Therefore, after the command execution, when we check the dotglob option, it is still disabled.
Changing Bash’s default behavior works. However, it’s not so convenient. Particularly, when we write a shell script, it can bring side effects.
*Another way to include dotfiles is to use “src/.” instead of “src/*” since “src/.” is not a globbing expression. It means everything under the src directory.*
Let’s see if it works as we expected:
$ cp -r src/. target
$ tree -a target
target
├── .hidden.file
├── originalTarget.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
1 directory, 4 files
Good, we’ve got the expected result.
3.3. cp and Overwrite the target Directory
If we want to copy src and overwrite the target directory, it’s an easier requirement than the non-overwriting variant. We can first remove the target directory and apply a cp -r command to solve the problem:
$ rm -r target && cp -r src target
$ tree -a
.
├── src
│ ├── .hidden.file
│ ├── srcFile.txt
│ └── subSrc
│ └── subSrcFile.txt
└── target
├── .hidden.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
4 directories, 6 files
4. Using the rsync Command
rsync is a powerful file copying utility. It can help us to solve this problem easily.
4.1. rsync Without Overwriting the target Directory
Two rysnc commands can help us to copy a directory recursively:
- rsync -a src target: copy the directory src and its content into target
- rsync -a src/ target: copy the contents of the src directory into target
The two commands above look pretty similar. The one with the trailing slash is exactly what we’re looking for.
Let’s test it on our example:
$ rsync -a src/ target
$ tree -a
.
├── src
│ ├── .hidden.file
│ ├── srcFile.txt
│ └── subSrc
│ └── subSrcFile.txt
└── target
├── .hidden.file
├── originalTarget.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
4 directories, 7 file
The tree‘s output shows the command has solved our problem.
4.2. rysnc and Overwrite the target Directory
We’ve learned we can combine two commands, “rm -r target && cp -r src target” to solve the problem.
If we’re armed with rysnc, we can achieve it in one shot:
$ rsync -a --delete src/ target
$ tree -a
.
├── src
│ ├── .hidden.file
│ ├── srcFile.txt
│ └── subSrc
│ └── subSrcFile.txt
└── target
├── .hidden.file
├── srcFile.txt
└── subSrc
└── subSrcFile.txt
4 directories, 6 files
As the output shows, after we execute the rsync command, target and src contain the same content. So, we’ve solved the problem yet another way.
The key to this solution is the option –delete.
The –delete option tells rysnc to delete files from target/ that are not in src/. In this way, it ensures that src and target end up identical.
5. Conclusion
In this article, we’ve addressed how to recursively copy a directory to an existing directory with or without overwriting.
We’ve learned two approaches to solve the problem: using the common cp command and the handy rysnc utility.
Further, we’ve discussed Bash’s dotglob option and how this option will affect globbing behavior.