1. Overview
Besides the Linux file permissions that protect our files from misuse, most Linux shells have a built-in protection mechanism to prevent accidental file overwrites. In this article, we’ll discuss the usage of these protections.
2. Protecting Files With noclobber
Most POSIX shells, if not all, implement a noclobber option. Meaning the shell will complain if a shell redirect is trying to overwrite an existing file.
By default, mainly due to tradition, the noclobber option is disabled. To turn it on in bash or ksh, we’ll use the following command:
set -o noclobber
When using csh or tcsh, we’ll set the option as follows:
set noclobber
Once we set the noclobber option, if we try to overwrite a file, bash will complain:
set -o noclobber
touch temp.txt # Create temp.txt file
echo "Hello" > temp.txt # Try to overwrite file contents
-bash: temp.txt: cannot overwrite existing file
Using csh/tcsh, the error message is a bit more cryptic:
set noclobber
touch temp.txt # Create temp.txt file
echo "Hello" > temp.txt # Try to overwrite file contents
temp.txt: File exists
We should note that this will only protect us from file overwrites using redirection. Removing the file via rm, appending to the file via “*>>*” redirection, or simply writing to the file from within a process will continue to work normally.
3. Overriding Protections
To override the noclobber protection, we can either return to the default behavior by disabling the shell option or temporarily overriding them. To disable the noclobber restriction in bash or ksh, we use the bash options semantics:
set +o noclobber
In tcsh/csh, this translates to:
unset noclobber
To temporarily override the noclobber behavior, our shell processes provide special redirection operators: “*>!” and “>|*” respectively. Let’s show our original example using bash:
set -o noclobber
touch temp.txt # Create temp.txt file
echo "Hello" > temp.txt # Try to overwrite file contents
-bash: temp.txt: cannot overwrite existing file
echo "Hello" >| temp.txt # Overwrite file contents using override operator
When using tcsh, we’ll just replace “>|” with “>!”:
set noclobber
touch temp.txt # Create temp.txt file
echo "Hello" > temp.txt # Try to overwrite file contents
temp.txt: File exists
echo "Hello" >! temp.txt # Overwrite file contents using override operator
4. Example: Truncating a Logfile
A classic example where we might need this feature is when truncating a logfile. Hence, log files tend to be kept open by the service logging the data. As such, we usually are unable to delete them as the operating system is keeping a tab on open file handles. To truncate the log file, we redirect /dev/null to the file:
/dev/null >| my_logfile.log
This solution has as an added bonus the property of not updating the modification time if the file is unchanged. This means that if we run the redirect in crontab and the file stayed empty, the modification time will reflect the last time the file was truncated.
Another option that can be used in such a case without using the command line redirect is truncate. The following example will have the same effect as the previous one, effectively resizing the file to 0:
truncate -s 0 my_logfile.log
The truncate command has an additional bonus, as it allows us to resize our file to any size. The following example will resize our log file to 50MB:
truncate -s 50M my_logfile.log
5. Conclusion
In this article, we discussed the noclobber option and its usages. We also introduce the usage of truncate as an optional tool for resizing files.