1. Overview
The sed command is a versatile tool designed for stream editing and text processing tasks. Among its many capabilities, sed allows us to perform multiple substitutions on a file with a single command.
In this tutorial, we’ll explore various methods to replace different parts of a file with one sed line, enabling us to efficiently manipulate text files and automate editing tasks.
2. Sample Task
Let’s suppose we have a file named input_file, and we want to modify its contents:
$ cat input_file
The executable is located at /usr/bin/program.
Visit http://example.com for more information.
In particular, we’d like to implement several changes to the contents of the file:
- remove leading spaces from each line
- replace occurrences of /usr/bin/ with /usr/local/bin/
- change http links to https
Let’s explore different approaches to implement these changes efficiently using sed.
3. Sequential Substitutions Using the -e Flag
One approach to carrying out multiple substitutions with sed involves using the -e flag. In particular, by providing multiple -e flags followed by substitution commands, each operation is sequentially applied to the file:
$ sed -e 's/^[[:space:]]*//g' -e 's:/usr/bin/:/usr/local/bin/:g' -e 's/http/https/g' input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.
Each of the substitution commands takes the form s/pattern/replacement/g where g indicates global substitution.
In the first substitution command, the pattern ^[[:space:]]* matches zero or more whitespace characters at the beginning of each line, which gets replaced with an empty string.
In the second substitution command, we use colons as delimiters (:) since the pattern and replacement both contain forward slashes. Alternatively, we could escape the forward slashes within a forward slash pattern. In any case, we replace /usr/bin/ instances with /usr/local/bin/.
Finally, in the third substitution command, we replace any http instance in the file with https.
By combining these substitution commands, we can effectively perform the desired modifications on the file using the sed utility.
4. Substitution Separators
When multiple -e statements are inconvenient, we can also turn to separators within a single statement.
4.1. Semicolon Separator
We can separate multiple substitution commands using semicolons within a single sed command:
$ sed 's/^[[:space:]]*//g; s:/usr/bin/:/usr/local/bin/:g; s/http/https/g' input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.
In this case, we apply our last solution but place a semicolon between each operation. As a result, the substitution commands execute sequentially giving the desired outcome.
4.2. Newline Separator
Alternatively, we can separate multiple substitution commands via newlines:
$ sed 's/^[[:space:]]*//g
> s:/usr/bin/:/usr/local/bin/:g
> s/http/https/g' input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.
Here also, the substitution commands execute sequentially as usual.
5. Script File and the -f Flag
For more complex substitution scenarios, creating a separate script file can streamline the process. For example, we place each substitution command without quotations on a new line within the script.sed script file:
$ cat script.sed
s/^[[:space:]]*//g
s:/usr/bin/:/usr/local/bin/:g
s/http/https/g
We then pass the script file to sed using the -f flag:
$ sed -f script.sed input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.
Additionally, the file for processing, input_file, is passed as an argument at the end of the sed command.
6. Script File With Shebang
Another way to use a sed script file is to incorporate a shebang directive (also known as a hashbang) into the script. This directive enables the script to execute directly with the sed executable.
$ cat script.sed
#!/usr/bin/sed -f
s/^[[:space:]]*//g
s:/usr/bin/:/usr/local/bin/:g
s/http/https/g
By adding a shebang header, we indicate that the script executes by running /usr/bin/sed -f script.sed followed by any additional arguments. To ensure the script can be executed directly, we need to grant execute permission to script.sed using the chmod command. Then, we can pass input_file as an argument to the script:
$ chmod u+x script.sed
$ ./script.sed input_file
The executable is located at /usr/local/bin/program.
Visit https://example.com for more information.
As before, sed runs each of the substitution commands within the script and returns the desired output.
7. Conclusion
In this article, we’ve seen various ways to perform multiple substitutions using a single sed command. In particular, we’ve seen that we can use sequential substitutions with the -e flag, semicolon or newline separation, or sed script files. This way, we can streamline text editing tasks and automate repetitive operations when manipulating files.