1. Overview
In this tutorial, we’ll see the xargs command along with some of its options. This command helps us to automate tasks by sending command outputs as arguments to other commands.
2. The xargs Command and Its Basic Usage
xargs is a command on Linux and most Unix-derived operating systems whose name means “extended arguments”. With xargs, we can use the standard input (such from another command using pipes) as arguments in a command. We need xargs for commands, such as cp, that can only take inputs as arguments.
The basic syntax of xargs has two parts. First, we’ve got the xargs options. Secondly, we’ve got the command to which xargs passes the arguments, together with other options that the command may need:
xargs [xargs options] [command [command options]]
If no command is provided, the default command is /bin/echo.
Let’s put this knowledge into practice. In the following example, we use cat to see the content of a file that has a list of names that we want to use as folder names:
$ cat list_of_folders.txt
1_0
1_1
1_2
1_3
However, we cannot directly use mkdir to create these directories:
$ mkdir list_of_folders.txt
mkdir: cannot create directory ‘list_of_folders.txt’: File exists
The solution to this problem is xargs! We can use cat again to print the content of the file and pipe the output to xargs coupled with mkdir as the command:
$ cat list_of_folders.txt | xargs mkdir
This command produces no output. Let’s inspect the content of the folder with ls:
$ ls
1_0 1_1 1_2 1_3 list_of_folders.txt
We’ve got the folders that we wanted to create together with the file that originally contained the names of these folders.
3. Basic Options
We already know how to use the basics of xargs but there are many options that we can use to tweak its performance.
3.1. Read Input From File
In many cases, we’ve got the arguments for our command in a text file (as before). Instead of using cat together with pipes, we can directly use -a filename (or –arg-file=filename) to pass this file to xargs:
$ xargs -a list_of_folders.txt mkdir
xargs will then read the arguments from the filename provided instead of the standard input. When using this option, the standard input of xargs remains the same.
3.2. Show Command Line Before Execution
xargs will by default execute the commands that we’ve requested. However, we can request that it prints a line with the command that is going to execute with the -t (or the –verbose) flag:
$ cat list_of_folders.txt | xargs -t mkdir
mkdir 1_0 1_1 1_2 1_3
Each command line is printed to standard error before executing it.
3.3. Interactive Mode and Confirmation Request
Creating directories is not a critical operation, but other operations carry higher risk. One such operation is the rm tool to remove, in this case, directories with the -r option. We can use the -p (or –interactive) flag to request that xargs prints the command that is going to execute and waits until we confirm (or cancel) the command execution:
$ cat list_of_folders.txt | xargs -p rm -r
rm -r 1_0 1_1 1_2 1_3?...y
To confirm the execution of a command, we press y on the keyboard or press any other key to cancel it. We’ll be prompted for each one of the commands that xargs will do and we’ll cancel just one command of them at a time.
3.4. Parallel Operations
Some commands might take some time and running them in serial is not efficient. Thus, we can request that xargs runs the commands in several processes with the -P max-procs (or –max-procs=max-procs) option:
$ cat list_of_folders.txt | xargs -P 2 mkdir
This will run the commands simultaneously in up to max-procs processes. The default value is 1. To request that xargs uses as many processes as possible simultaneously, we need to specify -P 0 (equivalent to –max-procs=0).
4. Advanced Options
We’ve already discussed the basic flags that we can use to tweak the behavior of xargs. However, there are other options that xargs can accept to deal with more complex scenarios.
4.1. Grouping Arguments
We may need to group several arguments in a single command call. In all the examples we’ve seen, all the arguments are passed together and a single command is invoked. However, this may not always be the case.
To illustrate this, let’s consider two generators of integers. We can get a list of the first five integers by requesting a sequence from echo:
$ echo {0..5}
0 1 2 3 4 5
We can get the same numbers in separated new lines with seq:
$ seq 0 5
0
1
2
3
4
5
Let’s plug this into xargs and see what it returns (remember that whenever no command is provided, the default one is /bin/echo):
$ echo {0..5} | xargs -t
echo 0 1 2 3 4 5
0 1 2 3 4 5
$ seq 0 5 | xargs -t
echo 0 1 2 3 4 5
0 1 2 3 4 5
As we’ve seen, all the digits are printed within a single echo command. We’ve got two flags that we can use to split these arguments: -n max-args (or equivalently –max-args=max-args) forces to use a maximum of max-args per command call and -L max-lines (or –max-lines=max-lines) uses a maximum of max-lines (non-empty) standard input lines for each command.
On the one hand, the -n flag will split the input based on the separators (white spaces in the case of echo and new lines in the case of seq):
$ echo {0..5} | xargs -n 2
0 1
2 3
4 5
$ seq 0 5 | xargs -n 2
0 1
2 3
4 5
We must be careful since the last call might receive less than max-args if there aren’t enough of them remaining in the standard input.
On the other hand, the -L flag will split the input based only on the newlines:
$ echo {0..5} | xargs -L 2
0 1 2 3 4 5
$ seq 0 5 | xargs -L 2
0 1
2 3
4 5
Thus, in the case of the -L flag together with echo, we’re getting a single command execution.
In summary, we can think that the -n flag will split the input on any separator while the -L flag will only split the input on newlines. The use of both -n and -L flags at the same time is not allowed.
4.2. Other Useful Flags
Regarding line separators, we might briefly mention the -d delimiter (or –delimiter=delimiter) flag as well. This flag allows us to specify a character as a delimiter to split the items that we receive through the standard input. We can find this option useful to skip potential calls to the cut command in between the pipes.
Finally, we must also shortly introduce the -0 flag*.* It will instruct xargs to expect NULL characters as separators (instead of the spaces and newlines it defaults to). This flag is mainly used with the -print0 flag in find.
4.3. Positional Arguments
We may still encounter an issue with xargs. Assume that we want to place the arguments in a different position than the last one or we want to use one argument several times. We can use the flag -I replace-str (or –replace=replace-str) to replace the occurrences of replace-str in the command options with the argument.
The default replace-str is {} but we can provide whatever we want. Another common replace-str choice is %. The -I replace-str option cannot be used with -n or -L. When we use the -I replace-str option, xargs will also use the -L 1 option.
Let’s see this in action with an example. We want to move all the folders that we’ve listed in a file called list_of_folders.txt to another directory named foo:
$ cat list_of_folders.txt
1_0
1_1
1_2
1_3
$ xargs -a list_of_folders.txt -t mv foo
mv foo 1_0 1_1 1_2 1_3
Here, we use the -a flag to pass the file instead of relying on the standard input. We also use the -t flag to see each command. However, this is not the command we wanted. We want to move the folders 1_0, 1_1, 1_2,… into the directory foo.
For that, we need the -I flag:
$ xargs -a list_of_folders.txt -t -I {} mv {} foo/{}
mv 1_0 foo/1_0
mv 1_1 foo/1_1
mv 1_2 foo/1_2
mv 1_3 foo/1_3
We see that with the -I flag we’ve put the input of each line twice in a command call.
5. Conclusion
In this article, we’ve seen the xargs command and some of its most relevant flags. We’ve described the -a flag to pass directly a file as input, the -t flag to print each command, and the -p flag to ask for confirmation before executing each command. The -P flag is useful to launch the commands in several processes.
There are other advanced flags to group arguments by splitting them in all separators with the -n flag or only on newlines with the -L flag. We can specify the delimiter with -d or use the NULL delimiter with -0. Finally, we’ve seen how to position the arguments and how to use them several times in the same command with the -I flag.