1. Overview
The mv command is the most widely used tool for moving files on Linux. However, by default, it doesn’t have the ability to show progress to the user. Sometimes, when we move gigantic files, we need some kind of an indicator that we can use to track the progress of the move operation.
In this tutorial, we’ll learn how we can move files and see their progress in a terminal by using several alternatives to the Linux mv command.
2. progress
progress is a UNIX tool that enables us to monitor the progress of a copy or move operation. Normally, we’ll use the mv command to move a file or a directory and then use the progress command in another terminal to monitor the progress.
2.1. Installation
The progress tool doesn’t ship with the distributions. So, we’d need to install the progress package from our distro’s official repository using a package manager such as yum or apt.
On Debian and Ubuntu, do:
$ apt install progress
On Fedora, CentOS, and RHEL:
$ yum install progress
On Arch and its derivatives:
$ pacman -S progress
Once the installation is complete, let’s verify it:
$ progress -v
progress version 0.16
2.2. Usage
Now, as an example, we’ll move the ~/.cache directory to /tmp. Then, we’ll use the progress command with a few useful options. So, let’s start moving the directory:
$ mv ~/.cache /tmp
As we know, the mv command, by default, doesn’t print anything unless there is an error. We’ll keep it intact and open another terminal instance, where we monitor the command’s progress. So, let’s open another terminal instance and just type in the progress command:
$ progress
[ 5187] mv /home/hey/.cache/paru/clone/android-studio/android-studio-2020.3.1.26-linux.tar.gz
0.0% (0 / 935.2 MiB)
The command will print out the progress at the current state and exit instantly. This is useful if we need to use the progress command in a shell script. On the other hand, if we want to monitor the progress, we’ll need to use the –monitor or -m option:
$ progress -mw
The -w or –wait option enables ETA. After running the command, we’ll see live progress of the mv command:
$ [ 5251] mv /home/hey/.cache/paru/clone/amdgpu-pro-installer/amdgpu-pro-21.30-1290604-ubuntu-20.04.tar.xz
35.31% (241.3 / 683.2 MiB) 0/s
If we’ve got multiple mv operations, we can also provide the –pid option:
$ progress -mw --pid
3. rsync
rsync is a highly configurable copying tool that supports a lot of options as compared to the cp and mv commands. We can use rsync to not only copy/move files locally but remotely as well. By default, rsync doesn’t show us any progress. Fortunately, there’s the -P option that displays a progress indicator for each file it copies.
3.1. Installation
Most complete distributions come with rsync pre-installed. If we’re on a minimal distribution, we can install it from the official repository for our distribution.
On Ubuntu derivatives:
$ apt install rsync
On Fedora, CentOS, and RHEL:
$ yum install rsync
On Arch and its derivatives:
$ pacman -S rsync
Once installed, let’s verify it:
$ rsync -V
rsync version v3.2.3 protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
3.2. Basic Usage
The basic syntax for copying files using rsync is:
$ rsync [OPTIONS] [SOURCE...] [DESTINATION]
We can specify as many sources as we need. Let’s copy our directory with rsync with the -r or —recursive option:
$ rsync -r ~/.cache/* /tmp/cache
You’ll notice that, similar to cp or mv command, it doesn’t print anything, with the exception of errors. Let’s enable the progress bar with the -P option:
$ rsync -r -P ~/cache/* /tmp/cache
chromium/Default/Cache/Cache_Data/2d0111cfb1b11d58_0
6,177 100% 60.32kB/s 0:00:00 (xfr#811, ir-chk=1565/2529)
chromium/Default/Cache/Cache_Data/2d152dfa21362cbf_0
7,724 100% 75.43kB/s 0:00:00 (xfr#812, ir-chk=1564/2529)
chromium/Default/Cache/Cache_Data/2d1e7c47b097a512_0
9,468 100% 92.46kB/s 0:00:00 (xfr#813, ir-chk=1563/2529)
chromium/Default/Cache/Cache_Data/2d490754d7b0a757_0
.
.
.
Once we run the command, you’ll notice that rsync prints progress for each file. Now, it’s all good, but it still doesn’t remove the source files. By default, rsync doesn’t move files from one location to another – rather, it copies them. Luckily, we’ve also got the –remove-source-files option that will remove the source files once they are completely copied:
$ rsync -aP --remove-source-files ~/.cache /tmp/cache
3.3. A Better Way
Although the –remove-source-files option does remove the files, it doesn’t remove the directories from the sources. So, to make rsync emulate the mv command, we’re going to write a very simple bash script that will handle multiple sources and remove the source directories and files after being copied.
The script usage syntax will be similar to how we use rsync. First, we’ll provide one or more input files and the destination directory as the last argument. Then, we’ll remove the source files and directories entirely once the copying is complete:
#!/bin/bash
# Safety check
[ "$#" -eq 0 ] && "No input files" && exit 1
rsync -aP --remove-source-files "$@" && rm -rf "${@:1:$#-1}"
The “$@” operator will expand to the positional parameters, which in our case will be the sources and a destination. The second command removes the sources recursively. The “${@:1:$#-1}” expression will basically expand to the provided arguments minus the last one, which is our destination directory.
The command after “&&” will run only when the rsync commands exit successfully. Otherwise, the files will be kept and not deleted. Once we’re done, let’s take the script for a test after we’ve made it executable:
$ rsmv ~/.cache/ctrlp /tmp/cache
mru/
mru/cache.txt
749 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=0/2)
4. Pipe Viewer
Pipe Viewer or pv is a tool that we use to monitor the progress of data transferring through a pipeline. It’s useful in cases where we only have a huge file that we want to move while monitoring its progress. Although it doesn’t work for multiple files, we could write a script to handle multiple sources.
4.1. Installation
On most major distributions, pv is already installed. We can also install it from our official package repository.
On Debian and Ubuntu-derivatives, do:
# apt install pv
On Fedora, CentOS, and RHEL, do:
# yum install pv
On Arch-derivatives, do:
# pacman -S pv
Once it’s installed, we’ll verify it:
$ pv -V
pv 1.6.20 - Copyright 2015 Andrew Wood <[email protected]>
4.2. Usage
Using pv is pretty straightforward. Either we place the pv command before a pipe operation, or we can place it between two commands:
$ pv INPUT > OUTPUT
$ CMD | pv | CMD
$ CMD | pv [OPTIONS] > OUTPUT
As an example, let’s pipe the contents of one file into another file:
$ pv amdgpu-pro-21.30.tar.xz > /tmp/amdgpu-pro.tar.xz
683MiB 0:00:01 [ 494MiB/s] [==================================================================================>] 100%
Now, this is great, but we still have the original amdgpu-pro-21.30.tar.xz file in the source directory. For that reason, we’ll use the mv command and have pv measure the progress for us. The mv command has a -v or –verbose option that prints feedback for each file that is moved.
We’ll use the -v option for mv as a clue for the pv command to figure out the progress:
$ mv source_dir dest_dir | pv -l -s $(find source_dir | wc -l) > /tmp/log
Let’s break it down:
- The -l option to the pv command will count lines instead of bytes
- The -s option specifies the size of the data being transferred so that pv can estimate progress; by default, this is specified in bytes, but since we’ve used the -l option, it’s the number of files
We find the estimated total number of files through the find command, which we specified to the -s option. This way, pv will now update the progress as more files are moved.
Though this solution is not as elegant as the previous two, it should be a good starting point to writing a customizable shell script.
5. Conclusion
In this tutorial, we discussed several ways to monitor the progress of the move operation. First, we covered the UNIX-y progress tool and, then, went on to use the rsync tool. Finally, we used the Pipe Viewer along with the mv command.