1. Overview
The rsync tool is a powerful tool for synchronizing files and directories. It can copy the contents of a source directory to a destination directory. Moreover, it can keep the destination directory synchronized if changes happen to the source directory.
In this tutorial, we’ll examine how the rsync tool can produce an output containing the files that will transfer to synchronize two directories.
2. Creating an Example Directory Structure
Let’s create our source directory that we’ll copy and synchronize with the rsync command:
$ mkdir source
$ touch source/file{1..2}.txt
$ mkdir source/subdir
$ touch source/subdir/subfile{1..2}.txt
$ echo Hello World of rsync | tee source/file1.txt source/file2.txt
Hello World of rsync
$ echo Hello World of rsync | tee source/subdir/subfile1.txt source/subdir/subfile2.txt
Hello World of rsync
$ tree
source/
├── file1.txt
├── file2.txt
└── subdir
├── subfile1.txt
└── subfile2.txt
1 directory, 4 files
Here, we created a directory named source that contains two files. We also created a subdirectory named subdir with two additional files. Moreover, we wrote a sample text message to all the files with the help of the tee command. The tee command redirects the output of the echo command to multiple files. Finally, we saw the structure with tree.
3. The rsync Command Default Behaviour
Now that we have a tree structure of files and directories, let’s execute the rsync command for the first time:
$ rsync -an --delete source/ destination
As we can see, the command doesn’t print anything to the standard output. Moreover, we used three options:
- -a makes the command run in archive mode
- -n stands for a dry run, which doesn’t perform any actions on the files, but instead only runs a test
- –delete deletes files or directories of the destination that no longer exist in the source
Basically, archive mode creates a recursive copy of the source directory to the destination, retaining information like the groups, owners, and permissions.
Also, we should note the slash character after the name of the source directory. This means that we want to copy the contents of the source directory to the destination directory.
4. The –out-format Option
The –out-format defines how the rsync command formats its output. The option accepts a string with escape characters, each preceded by the % character. Next, we examine the options that deal with the changed files of the source directory.
4.1. Relative Paths of Files to Copy (%f)
The %f escape character prints the filename of the files that will be copied:
$ rsync -an --delete --out-format="%f" source/ destination
source/.
source/file1.txt
source/file2.txt
source/subdir
source/subdir/subfile1.txt
source/subdir/subfile2.txt
As a result, the command printed a list of the files that will copy to the destination folder. In addition to the filename, it also prints the path relative to the source directory path.
4.2. Filenames of Files To Copy (%n)
If we want to print the filenames in a short form, we can use the %n escape character:
$ rsync -an --delete --out-format="%n" source/ destination
./
file1.txt
file2.txt
subdir/
subdir/subfile1.txt
subdir/subfile2.txt
Indeed, we see that the list of files is missing the reference to the source directory.
4.3. File Metadata
Next, we use some escape characters to print file metadata like permissions, modification dates, and others:
$ rsync -an --delete --out-format="%B %l %M %f" source/ destination
rwxrwxr-x 4096 2022/10/15-17:36:13 source/.
rw-rw-r-- 21 2022/10/15-16:01:49 source/file1.txt
rw-rw-r-- 21 2022/10/15-16:01:49 source/file2.txt
rwxrwxr-x 4096 2022/10/15-17:35:57 source/subdir
rw-rw-r-- 21 2022/10/15-16:02:28 source/subdir/subfile1.txt
rw-rw-r-- 21 2022/10/15-16:02:28 source/subdir/subfile2.txt
Here, in addition to the %f escape character, we’ve used some special characters:
- %B: file permissions
- %l: size of the file in bytes
- %M: last modification date of the file
All of this data can be helpful when deciding what to do with each file.
4.4. File Operation (%o)
The %o escape character prints the operation that the command will perform for each file:
$ rsync -an --delete --out-format="%B %l %M %U %o %f" source/ destination
rwxrwxr-x 4096 2022/10/15-17:36:13 1000 send source/.
rw-rw-r-- 21 2022/10/15-16:01:49 1000 send source/file1.txt
rw-rw-r-- 21 2022/10/15-16:01:49 1000 send source/file2.txt
rwxrwxr-x 4096 2022/10/15-17:35:57 1000 send source/subdir
rw-rw-r-- 21 2022/10/15-16:02:28 1000 send source/subdir/subfile1.txt
rw-rw-r-- 21 2022/10/15-16:02:28 1000 send source/subdir/subfile2.txt
Here, the command prints the send operation for all the files, since none of them exists in the destination folder.
To diversify the output, let’s run the rsync command once, without the dry run parameter, and then update and delete some files. After this, we’ll be able to see more operations in the output of the rsync command:
$ sudo rsync -a --delete source/ destination
$ echo Hello Again >> source/file2.txt
$ rm source/subdir/subfile2.txt
$ sudo rsync -an --delete --out-format="%B %l %M %U %o %f" source/ destination
rw-rw-r-- 33 2022/10/15-17:50:27 1000 send source/file2.txt
--------- 0 1970/01/01-02:00:00 0 del. subdir/subfile2.txt
rwxrwxr-x 4096 2022/10/15-17:50:41 1000 send source/subdir
As can be seen, after synchronizing all files, we’ve edited the file2.txt file and deleted the subfile2.txt file. As a result, the del. (delete) operation appears in the output of the rsync command.
5. Increase Verbosity
The -v or –verbose option makes the rsync command produce some output that also contains a list of the files that will be transferred:
$ rsync -anv --delete source/ destination
sending incremental file list
file2.txt
deleting subdir/subfile2.txt
subdir/
sent 180 bytes received 46 bytes 452.00 bytes/sec
total size is 75 speedup is 0.33 (DRY RUN)
In the output, we can notice the list of files and the operation for each file. We can also see additional information, like the bytes that will be sent and received, the speed, and a notice that this is a dry run.
6. File Differences
The -i or –itemize-changes causes the rsync command to print a list of files with additional data, per file, about what the command found different between the source and destination directories. The command displays this information as a character string, where each character has a special meaning:
$ rsync -ani --delete source/ destination
>f.st...... file2.txt
*deleting subdir/subfile2.txt
.d..t...... subdir/
Let’s decode the output above:
- > appears for file2.txt in the first line, meaning that this file will be copied to the destination directory
- f means that this is a file
- s denotes that the size of the file in the source directory is different (since we edited file2.txt) from the size of this file in the destination directory
- t means that the modification time of the file has changed, which is also true since we’ve edited the file
- * in the second line means that what follows is a message instead of a character list
- deleting appears since the file will be deleted in the destination folder
- d character in the third line means that this is a directory
Also, instead of the -i option, we may use the %i special character of the –out-format option:
$ rsync -an --out-format="%i %f" --delete source/ destination
>f.st...... source/file2.txt
*deleting subdir/subfile2.txt
.d..t...... source/subdir
We can see that the %i special character, combined with the %f special character, produces the same output as the -i option.
7. Conclusion
In this article, we explored the available options of the rsync command that deal with printing a list of the changed files between two directories. In conclusion, the rsync command can provide us with a good insight into the actions that will be performed to synchronize two directories.