1. Overview
We often use the sed or awk command to do “search and replace” jobs in the Linux command line. Usually, the matching text and the replacement occur in the same line.
However, sometimes, when we find some patterns in one line, we would like to do a replacement in another line related to it- for example, finding matching text and applying some text substitution in the next line.
In this tutorial, we’ll explore how to do it.
2. Introduction to the Problem
As usual, let’s understand the problem quickly through an example.
First, let’s say we have an input file:
$ cat revenue.txt
Revenue Report
==============
In 2021, we've made in total 4000 dollars' revenue.
Quarter-based revenue report is shown below:
- Quarter 1:
Revenue: 1200 dollars; Profit: 700 dollars
- Quarter 2:
Revenue: 1000 dollars; Profit: 650 dollars
- Quarter 3:
Revenue: 1200 dollars; Profit: 800 dollars
- Quarter 4:
Revenue: 600 dollars; Profit: -200 dollars
Total
----
Revenue: 4000 dollars
Profit: 1950 dollars
Say we’ve got the revenue report for 2021 from the finance department.
As the output above shows, the word “dollars” occurs many times in the file. Now, we’d like to replace the word “dollars” with “*$*” only in the quarter report part to make the content easier to read.
That is to say, we need to replace the word “dollars” with “*$*” only if the previous line matches the pattern ‘Quarter [1-4]:’.
Next, we’ll address how to solve the problem using sed and awk. Further, since GNU awk and GNU sed are widely used in Linux, we’ll use them for the examples in this tutorial.
Now, let’s see them in action.
3. Using the sed Command
We’ll address two sed one-liner solutions in this section.
3.1. Two sed One-liners
Let’s look at the first one:
$ sed '/Quarter [1-4]:/{ n; s/dollars/$/g }' revenue.txt
Revenue Report
==============
In 2021, we've made in total 4000 dollars' revenue.
Quarter-based revenue report is shown below:
- Quarter 1:
Revenue: 1200 $; Profit: 700 $
- Quarter 2:
Revenue: 1000 $; Profit: 650 $
- Quarter 3:
Revenue: 1200 $; Profit: 800 $
- Quarter 4:
Revenue: 600 $; Profit: -200 $
Total
----
Revenue: 4000 dollars
Profit: 1950 dollars
As the sed output above shows, only “dollars” occurrences in the quarter report part have been replaced by “*$“. Other occurrences, such as “dollars” in the “Total*” section, are not changed.
So, the command solves the problem. The sed command is also straightforward. The key is the ‘n‘ command.
The sed tool’s ‘n‘ command will print the current line and read the next input line into the pattern space.
Therefore, the command follows the logic:
- If the current line matches /Quarter [1-4]:/ – Print the current line and read the next line (n), then apply the substitution (s/../../g) and output the result
- Otherwise – Print the current line as-is
Alternatively, we can use sed‘s ‘n‘ and ‘b LABEL‘ (branch) commands to achieve our goal:
sed '/Quarter [1-4]:/!b; n; s/dollars/$/g' revenue.txt
Sharp eyes may have already spotted that we don’t pass a LABEL to sed‘s ‘b‘ command. *sed will start loading the next line if the LABEL of ‘*b LABEL‘ is omitted.**
Now, let’s understand how this sed command works:
- If the current line doesn’t match (/pattern/!) the pattern – The branch command (b without a label) will output the current line and load the next input line
- Otherwise – Print the current line and read the next line (n), then apply the substitution (s/../../g) and output the result
3.2. Applying the Substitution on the Next X-th Line
In this example, our requirement is to apply the substitution on the next line of the matching one. So, we used the ‘n‘ command once in our solutions.
However, it’s worth mentioning that once our requirement is applying the substitution on the second or the third line after the matching line, we can simply add more ‘n’ commands to achieve our goal.
For example, if we add two more ‘n’s to our second sed solution, sed will do the substitutions on the third line after the matching line:
$ sed '/Quarter [1-4]:/{ n; n; n; s/dollars/$/g }' revenue.txt
Revenue Report
==============
In 2021, we've made in total 4000 dollars' revenue.
Quarter-based revenue report is shown below:
- Quarter 1:
Revenue: 1200 dollars; Profit: 700 dollars
- Quarter 2:
Revenue: 1000 $; Profit: 650 $
- Quarter 3:
Revenue: 1200 dollars; Profit: 800 dollars
- Quarter 4:
Revenue: 600 $; Profit: -200 $
Total
----
Revenue: 4000 dollars
Profit: 1950 dollars
So, it’s easy to adjust the command to fit this kind of requirement change: adding a certain number of ‘n‘s.
However, we may have already realized that, if the number x in the requirement “the x-th line after the matching line” is relatively large, for example, eight, then we have to put eight ‘n‘s in the command. It makes the command harder to read, and it’s error-prone.
4. Using the awk Command
awk allows us to use variables. Therefore, we can solve the problem flexibly.
Let’s first look at the awk solution and then understand how it works:
$ awk '/Quarter [1-4]:/{ rl = NR + 1 } NR == rl { gsub( /dollars/,"$") } 1' revenue.txt
Revenue Report
==============
In 2021, we've made in total 4000 dollars' revenue.
Quarter-based revenue report is shown below:
- Quarter 1:
Revenue: 1200 $; Profit: 700 $
- Quarter 2:
Revenue: 1000 $; Profit: 650 $
- Quarter 3:
Revenue: 1200 $; Profit: 800 $
- Quarter 4:
Revenue: 600 $; Profit: -200 $
Total
----
Revenue: 4000 dollars
Profit: 1950 dollars
As we’ve seen in the output above, the awk command does the job. Next, let’s walk through the awk one-liner quickly to understand how it works:
- If the current line matches the pattern – Save the line number of the next line (NR + 1) in a variable (rl)
- If the current line number is equal to the value in ‘rl‘ – Apply the required substitution by calling the gsub function
- The non-zero number (1) will perform the default action – Print the current line
Since we declared the variable ‘rl‘ to hold the line number that we need to perform the substitution on, we can modify the value of “x” in the “rl = NR + x” expression to adapt to a different requirement.
For example, the command “awk ‘/Pattern/{ rl = NR + 8 } NR == rl { gsub( … ) } …” will apply the substitution on the eighth line after each matching line. From this point of view, the awk approach is more flexible than the sed solutions.
5. Conclusion
In this article, we’ve addressed how to search a pattern and apply some text substitution in the next line using sed and awk.
Moreover, we’ve discussed how to extend our solutions for the requirement “search a pattern and apply some text substitution in the x-th line after the matching line“.