在Shell中用字符串变量替换特定行

1. Overview

sed and awk are popular and powerful tools for processing text files in the shell.

In this tutorial, we’ll learn how to use these tools to replace specific lines with a string variable.

2. Sample Text

Let’s say we have an account-registration confirmation template message stored in the message_template.txt text file:

$ cat message_template.txt
<<greeting_msg>>
Thank you for registering with us.
<<phone_registration_msg>>

We can notice that <<greeting_msg>> and <<phone_registration_msg>> are placeholders within the file that we intend to replace with actual messages.

Next, let’s define shell variables GREETING_MSG and PHONE_REGISTRATION_MSG:

$ GREETING_MSG="Hello, Bob:"; 
$ PHONE_REGISTRATION_MSG="Your account is linked with 9999900000.";

We’ll use the message_template.txt sample file and shell variables in the following sections to replace the placeholder values using tools like sed and awk.

3. Using sed

Let’s use the -e switch available with sed to substitute the greeting and registration messages as separate substitution commands:

$ sed -e "s/<<greeting_msg>>/${GREETING_MSG}/g" \
-e "s/<<phone_registration_msg>>/${PHONE_REGISTRATION_MSG}/g" message_template.txt
Hello, Bob:
Thank you for registering with us.
Your account is linked with 9999900000.

We must note that we used double quotes instead of single quotes because we want the shell to extrapolate the values for the ${GREETING_MSG} and ${PHONE_REGISTRATION_MSG} variables. Otherwise, it treats them as literal strings.

So far, we have identified the target lines by matching them with placeholder patterns <<greeting_msg>> and <<phone_registration_msg>>. However, we can specify the line numbers exclusively while supplying the substitute text as variables. So, let’s go ahead and define these variables:

$ LINE_NAME=1; LINE_PHONE=3;

Finally, let’s modify the substitution commands to use these variables instead:

$ sed -e "${LINE_NAME}s/.*/${GREETING_MSG}/g" \
-e "${LINE_PHONE}s/.*/${PHONE_REGISTRATION_MSG}/g" message_template.txt
Hello, Bob:
Thank you for registering with us.
Your account is linked with 9999900000.

It looks like we’ve got this right!

4. Handling Special Characters With sed

Let’s say we want to use the same message template for a joint account. So, let’s update the GREETING_MSG variable with the new values to use the names of two persons:

$ GREETING_MSG="Hello, Alice & Bob:";

Next, let’s execute the same substitution commands as earlier and see if it works as expected:

$ sed -e "${LINE_NAME}s/.*/${GREETING_MSG}/g" \
-e "${LINE_PHONE}s/.*/${PHONE_REGISTRATION_MSG}/g" message_template.txt
Hello, Alice <<greeting_msg>> Bob:
Thank you for registering with us.
Your account is linked with 9999900000.

We can see that the replaced text isn’t what we were looking for. That’s because & is a special character denoting the whole part of the input string that matched the search pattern.

Moving on, let’s solve this issue by doing a post-processing of the $GREETING_MSG variable to escape the known special characters for the substitution string:

$ ESCAPED_GREETING_MSG=$(printf '%s\n' "$GREETING_MSG" \
| sed 's:[\\/&]:\\&:g; $!s:$:\\:')
$ echo $ESCAPED_GREETING_MSG
Hello, Alice \& Bob:

We must note that we’ve used colon(:) as a separator for the sed command *and escaped special characters such as &, backslash, forward-slash, and newlines*.

Finally, let’s use the ESCAPED_GREETING_MSG variable with the substitution commands while keeping other things the same:

$ sed -e "${LINE_NAME}s/.*/${ESCAPED_GREETING_MSG}/g" \
-e "${LINE_PHONE}s/.*/${PHONE_REGISTRATION_MSG}/g" message_template.txt
Hello, Alice & Bob:
Thank you for registering with us.
Your account is linked with 9999900000.

It looks like we’ve handled this edge case correctly.

5. Using awk

As awk supports programming constructs such as if-else blocks, we can use it to write a short script that uses the shell variables LINE_NAME, LINE_PHONE, GREETING_MSG, and PHONE_REGISTRATION_MSG.

Before using these shell variables within the .awk script, we need to make them available through export:

$ export LINE_NAME LINE_PHONE
export GREETING_MSG PHONE_REGISTRATION_MSG

Next, let’s write our replace-specific-line.awk script that uses these shell variables through the ENVIRON associative array:

$ cat replace-specific-line.awk
{
    if(NR==ENVIRON["LINE_NAME"]) { 
        print ENVIRON["GREETING_MSG"]
    } else if(NR==ENVIRON["LINE_PHONE"]) {
    print ENVIRON["PHONE_REGISTRATION_MSG"]
    } else {
        print $0
    }  
}

We must note the use of a built-in NR variable that holds the value of the current line number.

Finally, let’s execute the replace-specific-line.awk script and see it in action:

$ awk -f script.awk message_template.txt
Hello, Alice & Bob:
Thank you for registering with us.
Your account is linked with 9999900000.

We can see that the replaced text is as we expected it to be. Moreover, we don’t need to escape the & character in the GREETING_MSG variable when using awk.

6. Conclusion

In this article, we learned to use sed and awk for replacing specific lines with shell variables. Additionally, we trained ourselves to handle the edge case of special characters while working with sed.