1. Overview
sed (Stream Editor) is a tool that lets us transform and process text content. In the Linux userland, it’s one of the go-to tools for power users to manipulate text data quickly. However, some sed variants behave differently under certain use cases.
In this tutorial, we’ll discuss the differences between the two most used sed variants: GNU sed and BSD sed. We’ll learn the essential differences between the two with hands-on examples.
2. GNU sed vs. BSD sed
BSD sed is used on UNIX-based operating systems like FreeBSD, OpenBSD, NetBSD, and macOS. In addition, the macOS sed is derived from the FreeBSD sed.
In this section, we go through the differences between the GNU sed and BSD sed flavors. Most of the differences should also apply to other BSD variants of sed as well.
2.1. Extended Regular Expression (ERE)
ERE extends the Basic Regular Expression (BRE) by allowing additional pattern-matching abilities. In ERE, we don’t need to escape certain characters which carry out special operations.
In BSD sed, we enable the ERE option with -E:
$ sed -E 's/(apple|banana)/REMOVED/g' fruits.txt
REMOVED
REMOVED
orange
mango
tomato
guava
Before GNU sed 4.2, we enable the same option using the -r option:
$ gsed -r 's/(apple|banana)/REMOVED/g' fruits.txt
REMOVED
REMOVED
orange
mango
tomato
guava
However, in GNU sed 4.2+, we can use -E as well, which is an alias for -r. In the same way, in the newer versions of BSD sed (except OpenBSD sed), -r is an alias for -E.
2.2. The -i Option
-i lets us make modifications in place in a file. For instance, in GNU sed, we simply specify -i to replace text without having to create a backup file:
$ gsed -i 's/^apple$/avocado/' fruits.txt && cat fruits.txt
avocado
banana
orange
mango
tomato
guava
Conversely, -i in BSD sed always expects an argument. The argument is a backup file extension for the specified input file:
$ /usr/bin/sed -i '.bak' 's/^avocado$/apple/' fruits.txt && ls -lH
total 16
-rw-r--r--@ 1 hey staff 39 Jan 21 17:40 fruits.txt
-rw-r--r--@ 1 hey staff 39 Jan 21 17:39 fruits.txt.bak
In this case, “*.bak*” is appended to the backup filename. However, if we don’t need to create the backup file, we simply supply an empty string:
$ sed -i '' 's/^avocado$/apple/' fruits.txt
In addition, if we want to achieve cross-compatibility, we need to supply -e alongside -i:
$ sed -i -e 's/^banana$/lime/' fruits.txt
The -e option specifies the expression. It should work the same in both GNU sed and BSD sed except FreeBSD sed. In FreeBSD sed, -i still expects an argument even if we pass it the -e option. So, once we run the command with FreeBSD sed, it creates a backup file with -e appended to its filename.
2.3. Escaping Sequences
GNU sed can interpret and process escape sequences like \t, \s, \n, \w, and so on. These sequences have special meanings. For instance, \t specifies the tab character:
$ echo "apple&lime" | gsed 's/&/\t/'
apple lime
On the other hand, BSD sed has limited support for interpreting escape sequences:
$ echo "apple&lime" | sed 's/&/\t/'
apple\tlime
It only interprets \n (newline character) in the pattern and replacement parts of the expression. macOS sed is an exception in this case.
2.4. Newlines Between Expressions
In GNU sed, we can separate the expressions with a newline when surrounded by braces:
$ echo "apple" | gsed '{ s/apple/lemon/
s/lemon/lime/ }'
lime
In contrast, it’s not permissive in BSD sed:
$ echo "apple" | sed '{ s/apple/lemon/
s/lemon/lime/ }'
sed: 2: "{ s/apple/lemon/
s/Hi/Hey/ }": bad flag in substitute command: '}'
2.5. The i, a, and c Commands
The i, a, and c commands specify how to add text:
- i inserts text before a specified line
- a appends text after the specified line
- c changes the entire line(s) matching a pattern with new text
In BSD sed, we need to add a backslash and a newline after the i, a, and c commands:
$ echo "apple" | sed '1i\
lime'
lime
apple
Alternatively, we can achieve the same result in a single-line expression:
$ echo "apple" | sed $'1i\\\nlime\n'
lime
apple
In GNU sed, we don’t need anything like that:
$ echo "apple" | gsed '1ilime'
lime
apple
GNU sed automatically inserts a newline after the text is inserted with the i, a, and c commands.
2.6. The I and M Modifiers
In regular expressions, modifiers change the behavior of pattern matching. The I modifier disables case sensitivity and the M modifier does multi-line mode matching.
In older macOS sed, the I and M modifiers aren’t supported. However, they’re supported by FreeBSD sed as well as GNU sed:
$ echo "Apple" | gsed -E '/apple/I s/a/A/'
Apple
Similarly, we can use the M modifier in GNU sed as well:
$ echo -e "apple\nlime" | gsed -n '/^apple$/M{N; s/\n/ /p;}'
apple lime
2.7. The -s, -u, and -z Options
macOS sed doesn’t support -s, -u, and -z options:
- -s or –separate treats input files as separate streams
- -u or –unbuffered flushes the output buffer after each line
- -z or –null-data allows the processing of null-terminated lines
While they work in GNU sed, we’ll encounter an error when we specify one of these options in BSD sed:
$ sed -s 's/file/replaced/' file1.txt file2.txt
sed: illegal option -- s
2.8. The -a Option
In BSD sed, -a makes the w command append to a file:
$ echo "plum" | sed -a 'w fruits.txt' && cat fruits.txt
apple
banana
orange
mango
tomato
guava
plum
But, it’s not supported by macOS sed. However, we can achieve the same result with the append to file redirect operator >>.
3. Additional Differences
In addition to the core differences, there are extra features that are solely implemented by GNU sed:
- Deleting lines using the + symbol
- Printing every nth line
- Quitting after the first occurrence of a pattern with Q
- Changing the case of the first letter
- Executing a shell command before processing a file
- Turning off line wrapping
While we can achieve the same result with BSD sed, it’s much more difficult compared to the built-ins provided in GNU sed. For that reason, we should use alternatives like awk or Perl.
4. Using GNU sed on macOS
Sometimes, it’s harder to keep track of the differences between the variants. Fortunately, we can install GNU sed on macOS. For that purpose, we can use Homebrew:
$ brew install gnu-sed
Once installed, we can use it by prefixing it with g:
$ gsed --version
gsed (GNU sed) 4.9
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
5. Conclusion
In this article, we learned the core differences between the GNU sed and BSD sed. In addition to the core differences, we also briefly discussed the additional minor differences that are specific to GNU sed.
Finally, we saw how to install and use GNU sed on macOS.