1. Overview
In this tutorial, we’re going to talk about the paste command. This command helps us to merge all the lines of multiple files in a parallel or sequential manner.
2. Using the Paste Command
We’ll start exploring paste with a simple example. Let’s suppose we have a file, File1, with some cities and another file, File2, with some random average temperatures for each one of them:
$ cat File1
New York
Philadelphia
Virginia
Washington
Texas
$ cat File2
29°C
26°C
32°C
31°C
35°C
If we run paste on these two files:
$ paste File1 File2
The output of the command will be:
New York 29°C
Philadelphia 26°C
Virginia 32°C
Washington 31°C
Texas 35°C
We can see paste has combined each line of the input files into a new line. This kind of merging is called a parallel merge. It means that the nth line in the result is the concatenation of the nth line of each input file, separated by a tab.
We can also change the default way of merging and merge files sequentially. We can use -s option for this:
$ paste -s File1 File2
New York Philadelphia Virginia Washington Texas
29°C 26°C 32°C 31°C 35°C
3. Joining Files With Different Number of Lines
What does paste do if the number of lines in each input file is not the same? In the example above, the number of lines in both File1 and File2 is five. Let’s add a new line at the end of File2:
$ echo "12°c" >> File2
If we run paste on File1 and File2 again, this will be the output:
$ paste File1 File2
New York 29°C
Philadelphia 26°C
Virginia 32°C
Washington 31°C
Texas 35°C
12°c
Note paste can merge files with different line numbers. If the nth line of a file is empty, then paste puts an empty string instead.
Let’s revert our changes on File2 and remove the last line inserted, using sed command*:*
$ sed -i '$d' File2
4. Changing the Default Delimiter
The default delimiter for paste command is tab. We can change this delimiter to whatever we want, using -d option:
$ paste -d ":" File1 File2
New York:29°C
Philadelphia:26°C
Virginia:32°C
Washington:31°C
Texas:35°C
In this example, we use “*:*” instead of the default delimiter. We can use any character, even a non-text character like \n as a delimiter.
Note that if we have two input files, we need one delimiter to separate the data but, if there are more than two input files, we can define one or more. Let’s see how we can specify more than one delimiter.
5. Defining Different Delimiters
Let’s assume we have another file, named File3, that also contains the temperature for each city but now, in Fahrenheit degrees:
$ cat File3
84.2°F
78.8°F
89.6°F
87.8°F
95°F
We can add this data to our previous information using a different delimiter:
$ paste -d ":," File1 File2 File3
New York:29°C,84.2°F
Philadelphia:26°C,78.8°F
Virginia:32°C,89.6°F
Washington:31°C,87.8°F
Texas:35°C,95°F
The paste command uses the delimiters in the order we defined them. So, in this example “*:” is the first delimiter and “,*” is the second.
Note that there shouldn’t be any space between delimiters. Defining “*: ,*“, for example, gives a different output because in this case, the second delimiter will be the space:
$ paste -d ": ," File1 File2 File3
New York:29°C 84.2°F
Philadelphia:26°C 78.8°F
Virginia:32°C 89.6°F
Washington:31°C 87.8°F
Texas:35°C 95°F
Notice that, in this case, paste ignored the third delimiter. For three input files, we only need two delimiters.
6. The Order in Using Delimiters
What if we define two delimiters for four or more files? What does paste do in this situation? Let’s specify two delimiters for six files. For simplicity we’ll use our three previous files in random order as we want to just focus on the way paste use delimiters:
$ paste -d ":," File1 File2 File3 File2 File3 File1
New York:29°C,84.2°F:29°C,84.2°F:New York
Philadelphia:26°C,78.8°F:26°C,78.8°F:Philadelphia
Virginia:32°C,89.6°F:32°C,89.6°F:Virginia
Washington:31°C,87.8°F:31°C,87.8°F:Washington
Texas:35°C,95°F:35°C,95°F:Texas
We can see that the order of using delimiters by paste is circular. Notice that if we define only one delimiter, that delimiter will be used to separate every field.
Up to here, we’re just using regular files as input arguments. What if we want to read data from the standard input?
7. Reading From Standard Input
We can read data from standard input or redirect it using pipelines. Let’s modify our previous command:
$ paste -d ":," File1 File2 File3
Assume we want to read File2 from standard input. To read any file from standard input with paste, we should use a dash instead of the filename. Then we can use, for example, the usual combination of cat and pipes to redirect standard input:
$ cat File2 | paste -d ":," File1 - File3
New York:29°C,84.2°F
Philadelphia:26°C,78.8°F
Virginia:32°C,89.6°F
Washington:31°C,87.8°F
Texas:35°C,95°F
8. Merging n consecutive Lines of One File
Another interesting usage for the paste command is when we want to merge n consecutive lines of a file. By the number of dashes we put in front of paste, we specify how many lines of input should be merged together.
For example, if we put two dashes in front of paste, every two consecutive lines of input will be merged:
$ cat File1 | paste - -
New York Philadelphia
Virginia Washington
Texas
Notice that for merging the lines of a single file, passing its name as a parameter won’t work. We need to use the cat and pipe combination, as we did above, or redirect the stdin (<).
We can, of course, combine this with delimiters. In this example, as we use three dashes, every three consecutive lines in the input file will be merged separated by “*:*“:
$ cat File1 | paste -d ":" - - -
New York:Philadelphia:Virginia
Washington:Texas:
Note that File1 only has five lines. That’s why the last member of the second line is empty.
Finally, instead of using the cat and pipe, it is also possible to insert, or paste, the data in the terminal:
$ paste - - > File4
Alaska
Florida
Michigan
Arizona
California
In this case, we redirect the output to a file to save the result. Note that after we’ve inserted all the data, we should press ^D (CTRL + D) to both, end the process and signal the end of the file. If we take a look at File4, we can see that paste merged every two consecutive lines:
$ cat File4
Alaska Florida
Michigan Arizona
California
Note if we don’t use redirection, after every “enter” we press the terminal will echo the input data, and paste command won’t work.
9. Conclusion
In this article, we used paste to merge lines of one or more files in a parallel or serial format. We also saw how to change the default delimiter and redirect inputs.
Although paste is a simple command, it has many applications, and one of the best things about it is that its usage is effortless and straightforward.