1. Overview
For pattern matching and text manipulation, Linux offers many tools and utilities, such as grep, sed, awk, etc. However, doing arithmetic operations with these utilities can be tricky because these tools are designed for text processing.
In this tutorial, we’ll learn several ways to increment a number in a matched pattern.
2. Scenario Setup
First, let’s create a sample pricing.txt file for simulating our use case scenarios:
$ cat pricing.txt
Product Pricing
P1 $100
P2 $200
P3 $300
P4 $500
We can see that the second column contains the price against a product listed in the first column. Further, each price has the currency ($) prefix.
Our objective is to increase the price for each product by $100. So, we’ll need to match the numeric pattern for searching the price value and then increment the matched value.
Next, let’s see the expected output that we want to get after applying the necessary operations:
Product Pricing
P1 $200
P2 $300
P3 $400
P4 $600
Lastly, we must remember that we’ll reuse the pricing.txt file for each approach without modifying the original file.
3. Using awk
awk is a full-fledged programming language well-suited to work with structured data and perform calculations. In this section, let’s learn how to use awk to solve our use case.
3.1. With match() and substr()
Let’s start by writing the match_then_increment.awk script and take a look at it in its entirety:
$ cat match_then_increment.awk
NR==1 {
print
}
NR>1 {
if (match($2, /^\$[0-9]+$/)) {
old_value = substr($2, RSTART+1, RLENGTH)
new_value = old_value + 100
$2="$"new_value
print $1,$2
}
}
Now, let’s break this down to understand the nitty gritty of the logic.
First, we print the first row from the input file without any modification. After that, we call the match() function for the second field ($2) in the remaining lines to match it against a regex for a valid price value. Internally, match() sets the values for the RSTART and RLENGTH built-in variables as the starting position and total length of the substring matching the pattern, respectively.
Then, we use the substr() function to get the numerical value of the price by trimming out the currency symbol ($) in the prefix. Further, we store the numerical value in the old_value variable and initialize the new_value variable by adding 100 to old_value.
Eventually, we set the value of the second field ($2) by concatenating the currency symbol ($) and new_value. We also print the first ($1) and the modified second field ($2).
Finally, let’s see our match_then_increment.awk script in action:
$ awk -f match_then_increment.awk pricing.txt
Product Pricing
P1 $200
P2 $300
P3 $400
P4 $600
Fantastic! The output is as expected.
4. Using sed
sed is another powerful text-processing language in Linux. However, it doesn’t support arithmetic operations such as addition directly.
In this section, we’ll augment sed with other command-line utilities, such as bc and expr, and arithmetic expansions to solve our use case.
4.1. With bc
First, let’s install the bc utility, as it doesn’t come preinstalled in some Linux distributions:
$ apt-get install bc
Now, let’s see how we can perform simple arithmetic like addition by passing a string as an input to bc through a pipe:
$ echo '100 + 10' | bc
110
Further, let’s use this concept in writing the match_then_increment_bc.sed script to solve our use case:
$ cat match_then_increment_bc.sed
1p
1!{
s/(.*)\$(.*)/echo "\1\\$$(echo "\2+ 100" |bc)"/ep
}
We start by printing the first line as it is. Further, we use the substitution (s) command to replace each of the remaining input lines with an echo statement having a shell command as an argument. Eventually, we execute the shell command with the help of the /e flag and print the result with the following /p flag.
Finally, let’s verify that our script gives us the right results:
$ sed -n -E -f match_then_increment_bc.sed pricing.txt
Product Pricing
P1 $200
P2 $300
P3 $400
P4 $600
Great! It looks like we’ve got this one right.
4.2. With expr
Unlike the bc utility that accepts the input through a file, the expr command accepts the arithmetic operands as arguments:
$ expr 100 + 10
110
So, we can extend the same approach to write the match_then_increment_expr.sed script:
$ cat match_then_increment_expr.sed
1p
1!{
s/(.*)\$(.*)/echo "\1\\$$(expr \2 + 100)"/ep
}
Like earlier, we’ve used the substitution (s) command to replace the input line with a shell command. Later, sed applies the /e flag to execute the command string.
Lastly, let’s check the result by executing the match_then_increment_expr.sed script:
$ sed -n -E -f match_then_increment_expr.sed pricing.txt
Product Pricing
P1 $200
P2 $300
P3 $400
P4 $600
Perfect! It looks like we’ve nailed this one.
4.3. With Arithmetic Expansion
Alternatively, we can use arithmetic expansion $(()) for performing the arithmetic operation:
$ echo $(( 100 + 10 ))
110
Now, let’s write the match_then_increment_arithmetic_expansion.sed script using arithmetic expansion:
$ cat match_then_increment_arithmetic_expansion.sed
1p
1!{
s/(.*)\$(.*)/echo "\1\\$$(( \2 + 100 ))"/ep
}
Like always, let’s execute and verify our script:
$ sed -n -E -f match_then_increment_arithmetic_expansion.sed pricing.txt
Product Pricing
P1 $200
P2 $300
P3 $400
P4 $600
Great! It works as expected.
5. Using Bash Script
In this section, we’ll solve our use case by writing a Bash script using loop constructs, together with grep or cut for pattern matching and text extraction.
5.1. With grep
Let’s s write the match_then_increment_grep.sh Bash script and look at its entirety:
$ cat match_then_increment_grep.sh
#!/bin/bash
input_file="pricing.txt"
while read -r line; do
if echo "$line" | grep -q -P '\$[0-9]+'; then
product=$(echo $line | grep -ow '^[^$ ]*')
old_price=$(echo $line | grep -oP '\$\K[0-9]*')
new_price=$(expr $old_price + 100)
echo "$product \$$new_price"
else
echo "$line"
fi
done < "$input_file"
We used the read command within a while loop to populate the line variable with input text line-by-line. Further, we extracted the product name and old price into the product and old_price variables using grep.
It’s worth noting that we used the -o option with grep to show only the matching part of the text. Additionally, we needed the -P option to support the \K flag available in the PCRE syntax for removing all characters that come before it.
Then, we incremented old_price using expr and displayed the revised values using echo.
Finally, our script is ready for execution, so let’s verify it:
$ ./match_then_increment_grep.sh
Product Pricing
P1 $200
P2 $300
P3 $400
P4 $600
It’s working as expected.
5.2. With cut
Alternatively, instead of using grep, we can extract the product details, including price, using the cut command.
Let’s reuse our approach in writing the match_then_increment_cut.sh script for solving the use case:
$ cat match_then_increment_cut.sh
#!/bin/bash
input_file="pricing.txt"
while read -r line; do
if echo "$line" | grep -q -P '\$[0-9]+'; then
product=$(echo "$line" | cut -d' ' -f1)
old_price=$(echo "$line" | cut -d'$' -f2)
new_price=$(expr $old_price + 100)
echo "$product \$$new_price"
else
echo "$line"
fi
done < "$input_file"
In this case, we use the -d option to define the delimiter and the -f option to specify the field we want. So, for the product name, we could use space as a delimiter and the currency symbol ($) to extract the price value. After that, we perform the necessary arithmetic operations using expr.
Now, let’s confirm that our script is working correctly:
$ ./match_then_increment_cut.sh
Product Pricing
P1 $200
P2 $300
P3 $400
P4 $600
It looks like we’ve got this one right, too!
6. Conclusion
In this article, we learned how to increment a number in a matched pattern. Furthermore, while solving the use case, we explored different command-line utilities, such as awk, sed, expr, bc, grep, and cut.