1. Overview
Trailing newlines can end up in our files due to various reasons, such as saving the output of a buggy command that emits extra newlines. In this tutorial, we’ll learn how to remove trailing newline characters from a file.
2. Removing a Single Trailing Newline
Let’s look at the commands used to remove a single trailing newline from a file.
In our examples below, we use the wc -l command to count the number of newlines.
2.1. Using perl
First off, let’s check the contents of our sample file:
$ cat bad.txt
one newline
$ wc -l bad.txt
1 bad.txt
Here, wc -l outputs 1 as the file contains a single trailing newline.
Now, we can use *perl*‘s chomp function to remove the last character if it’s a newline:
$ perl -p -i -e 'chomp if eof' bad.txt
$ wc -l bad.txt
0 bad.txt
As we can see, wc -l outputs 0 after removing the trailing newline. We use the -p flag to print the output and the -i flag to write the output to the file, while -e allows us to pass the Perl code as an argument.
The if eof statement indicates that the character is the last character, i.e., at the end of the file. This method is quite efficient since we only invoke a single perl command.
2.2. Using head and tail
We can use the tail command to test whether the last character is a newline and then use the head command to remove it:
$ cat bad.txt
one newline
$ wc -l bad.txt
1 bad.txt
$ if [ "$(tail -c 1 bad.txt | wc -l)" = 1 ]; then head -c -1 bad.txt > tmp; mv tmp bad.txt; fi
$ wc -l bad.txt
0 bad.txt
Here, the output of wc -l is similar to the previous example. The -c 1 flag gets the last character of the file. We pipe this character into wc to check whether it’s a newline. If it is, we use head -c -1 to print the whole file except the last character, thus removing the newline.
This method uses a temporary file and overwrites the original file with it. Therefore, it is less efficient than the method using perl, as we invoke two commands instead of one.
3. Removing All Trailing Newlines
Instead of just removing the last trailing newline, we can remove all trailing newlines from a file.
First, let’s look at the file contents:
$ cat bad.txt
many newlines
$ wc -l bad.txt
4 bad.txt
Here, wc -l outputs 4 as the file contains four trailing newlines.
We can use Bash’s built-in printf command for removing the trailing newlines:
$ printf "%s" "$(< bad.txt)" > tmp
$ mv tmp bad.txt
$ wc -l bad.txt
0 bad.txt
We can see that wc -l outputs 0 after removing the trailing newlines. We load the file into memory with the $(< ) operator and pass it to printf. Bash automatically strips trailing newlines when reading the file.
4. Conclusion
In this article, we learned to use various commands for removing single as well as multiple trailing newlines from the end of a file.