1. Overview
In this tutorial, we’ll be looking at the logrotate tool. Specifically, we’ll learn how to automate log rotation using logrotate in Linux.
2. logrotate
In Linux, applications and background processes are constantly generating logs. It’s important to keep these logs in check by trimming them on a specific schedule. However, manually doing that is labor-intensive. To reduce the manual intervention, we can automate the process using logrotate.
logrotate is a log managing command-line tool in Linux. The administrators write the rules and policies for handling different log files into configuration files. Through the configuration file, logrotate will execute the appropriate function to manage the matching log files.
Although logrotate works for any kind of file, it’s generally used for managing the log files.
2.1. Installation
To install logrotate, we can use the package manager in different Linux distros.
On Ubuntu, we can install the package using apt-get:
$ sudo apt-get update
$ sudo apt-get install -y logrotate
For RHEL based Linux such as CentOS, we can use the yum package manager:
$ sudo yum update
$ sudo yum install -y logrotate
Finally, we can install logrotate in alpine using the apk package manager:
$ sudo apk update
$ sudo apk add logrotate
2.2. Command Syntax
Generally, we run the logrotate command by specifying a list of options followed by the paths to each config file:
logrotate [OPTION...] <config file1> <config file2>...
When there are multiple configuration files, the latter definition will override the earlier definition. In other words, the first configuration file has the lowest precedence.
2.3. Directives
The directives are a basic building block of logrotate configuration, and they define different functions. They are then combined to form a configuration file that applies to different log files. Furthermore, each of these directives allows us to configure different aspects of logrotate. For example, the copy directive alters the rotation process. On the other hand, the compress directive changes how the rotated files are treated.
Additionally, most of these directives have a negation directive counterpart. These negation directives cancel their counterpart effect when specified. For instance, specifying a nocompress directive within a configuration block cancels a global compress directive.
For a comprehensive list of directives, we can find it on the official man page.
2.4. Configuration File
Using the directives, we can compose a set of configuration files that can be consumed by logrotate. Generally, the configuration file is composed in the following format:
<global directive 1>
<global directive 2>
<file path matchers 1> {
<directive 1>
<directive 2>
...
<directive n>
}
<file path matchers 2a> <file path matchers 2b> {
<directive 1>
<directive 2>
...
<directive n>
}
At the top of the configuration files, we can specify 0 or more global directives. The global directives affect all the log files that are managed by the logrotate.
Then, we specify multiple glob patterns to match the files we want to rotate. Additionally, we define a set of directives within the curly braces that are specific for the files that match the pattern.
Let’s look at an example of a configuration file:
compress
/var/log/nginx/* {
rotate 3
daily
}
/var/log/nginx/error.log {
rotate 3
size 1M
lastaction
/usr/bin/killall -HUP nginx
endscript
nocompress
}
In this example, we can see that a global compress directive at the top. Specifically, any log files that are matched will be compressed upon rotation.
Then, we can see 2 patterns defined: /var/log/nginx/* and /var/log/nginx/error.log. The first matcher uses a wildcard to match every file within the /var/log/nginx directory. On the other hand, the 2nd matcher only matches the specific error.log file in /var/log/nginx.
Within the 2nd configuration block, a nocompress directive is specified to override the global compress directive.
2.5. Default Configuration
When we install logrotate, it creates a default configuration at /etc/logrotate.conf. This configuration file contains several global directives that define some of the default behavior for logrotate. Since the configuration files in logrotate are additive, we must keep in mind the default configuration so we can override if needed.
For example, logrotate installation on Ubuntu comes with global directives like weekly, rotate 4, and create.
2.6. Applying Configuration Files
Once we’ve written the rotation policy on a configuration file, we’ll need to run the logrotate command for it to take effect. Concretely, to apply the log-rotation.conf configuration file, we can run the logrotate command followed by the configuration file path:
$ logrotate log-rotation.conf
If we want logrotate to evaluate the condition immediately, we can pass the -f option to force run it:
$ logrotate -f log-rotation.conf
3. Rotating Condition
Using logrotate, we can rotate the log files under different conditions. Particularly, we can rotate log files on a fixed duration or if the file has grown to a certain size.
3.1. Rotating Log Files Based on Duration
To rotate a file based on duration, we can specify 1 of the 4 directives: daily, weekly, monthly, and yearly. For example, we can rotate the /var/log/nginx/access.log daily by specifying the daily directive in the configuration file:
/var/log/nginx/access.log {
daily
}
Alternatively, to rotate it weekly, we simply specify the weekly directive:
/var/log/nginx/access.log {
weekly
}
3.2. Rotating Log Files Based on Size
Most of the time, we would want to rotate the log file once it hit a certain size. To achieve this, we can specify the size directive followed by the size limit. For instance, to rotate the /var/log/nginx/access.log once it hits 1 megabyte, we can use the size 1M directive:
/var/log/nginx/access.log {
size 1M
}
Besides the suffix M as megabytes, the directive recognizes suffix k as kilobytes and G as gigabytes. Concretely, specifying size 1000k will have the same effect as specifying size 1M.
4. Handling Rotated Files
Once a log file is rotated, we can specify what should happen to them through a different set of directives. For example, we could send the rotated files to an email or simply discard them.
4.1. Keeping Only a Few Rotated Log Files
We could limit the number of rotated files we keep in our system through the rotate n directive where n is an integer value. Once the limit n is reached, any further rotation will remove the oldest rotated log file such that the limit is adhered to. For example, to keep only 3 rotated /var/log/nginx/access.log files:
/var/log/nginx/access.log {
size 1M
rotate 3
}
Upon the rotation, logrotate will check if there are already 3 rotated access.log in the system. If there is, it will discard the oldest to make space for the new rotated file. Ultimately, this configuration makes sure that there will be 3 or fewer rotated /var/log/nginx/access.log in the system.
4.2. Removing Rotated Files by Age
Furthermore, we can remove the rotated files by their age. Particularly, we can use the maxage directive to discard files that are older than a specific number of days. The directive takes an additional integer argument that specifies the number of days.
For instance, we can discard rotated logs that are already in the system for more than 5 days:
/var/log/nginx/access.log {
size 1M
maxage 5
}
4.3. Compressing the Rotated Files
The logrotate tool would optionally compress the old log files after each rotation. Concretely, we could make logrotate compress the rotated log files using the compress directive:
/var/log/nginx/access.log {
size 1M
compress
}
In the example above, logrotate will rotate the log file once it hits 1 megabyte in size. Subsequently, it will compress the older log file using the gzip command. To negate a globally applied compress directive, we can use the nocompress directive:
compress
/var/log/nginx/access.log {
size 1M
}
/var/log/nginx/error.log {
size 1M
nocompress
}
In the example above, logrotate would compress the access.log file after each rotation. However, it will exempt error.log from compression due to the nocompress directive.
4.4. Mailing the Rotated Files
After every rotation, we can have logrotate email the rotated files to some recipients using the mail directive. To enable that, we’ll first have to configure a working email client on the system. Then, we can specify the mail directive to mail the log files that are about to be discarded due to rotation:
/var/log/nginx/error.log {
size 1M
rotate 2
mail
}
With the configuration above, the logrotate would keep a maximum of 2 copies of the rotated log file for /var/log/nginx/error.log. Additionally, right before it discards any rotated logs, logrotate will mail it to the specified recipients.
Instead of mailing the expired log, we can mail the freshly rotated log file using the mailfirst directive:
/var/log/nginx/error.log {
size 1M
rotate 2
mail
mailfirst
}
When a rotation happens, logrotate will email the newest rotated log files instead of the oldest one.
5. Handling Active Log File
In the default behavior, the rotation process beings by logrotate renaming the active log file into a different name. Then, it creates a new log file with the same name. Finally, it invokes other logic such as compress and mail to complete the rotation process.
For some applications, this default behavior is too disruptive. Particularly, an application might crash if it’s not flexible enough to handle the transition to a new file. In those cases, we can alter the behavior using 1 of the 3 directives: create, copy, and copytruncate.
5.1. Creating a New Log File
The create directive is how logrotate handles the rotation by default. With this policy, it simply renames the log file and recreates the log file with the same name. On the surface, it looks like the log file is still the same after the rotation. However, we’ll get different inode values for the same file after the rotation.
Let’s configure the create directive for the /tmp/logs/error.log:
/tmp/logs/error.log {
size 100k
rotate 2
create
}
Then, we enter 200 kilobytes worth of data into the error.log file and note down its inode value:
$ ls -hil /tmp/logs
total 200K
402882 -rw-r--r-- 1 user user 200K Sep 5 12:45 error.log
Next, we run the logrotate command to trigger the rotation:
$ logrotate logrotate.conf
$ ls -hil /tmp/logs
total 200K
402878 -rw-r--r-- 1 user user 0 Sep 5 13:00 error.log
402882 -rw-r--r-- 1 user user 200K Sep 5 12:59 error.log.1
As we can see, the file with inode value 402882 is renamed to error.log.1. On the other hand, the error.log file has a new inode value. In other words, the logrotate command first renames the log file to error.log.1. Subsequently, it creates a new error.log.
5.2. Making a Copy of the Active Log File
To make the rotation process less disruptive, we can rotate the log by copying the old log into a different file and clearing the content.
This flavor of rotation comes in the form of 2 different directives: copytruncate and copy. Contrary to create, both copy and copytruncate make a copy of the main log file instead of renaming it.
However, there’s a difference between copy and copytruncate when it comes to what happens to the content on the main file. On the one hand**, the copytruncate directive clears the content in the original log file. On the other hand, the copy directive will not clear off the content of the original log file.**
Let’s look at an example to learn their exact behavior in depth. Firstly, we create the /tmp/logs/error.log as a 200 kilobytes file:
$ ls -hil /tmp/logs
total 200K
402878 -rw-r--r-- 1 user user 200K Sep 5 13:10 error.log
Next, we create a configuration file for /tmp/logs/error.log to rotate on size 100 kilobytes with the copytruncate directive:
/tmp/logs/error.log {
size 100k
rotate 2
copytruncate
}
Finally, we run the logrotate command to trigger the rotation:
$ logrotate logrotate.conf
ls -hil /tmp/logs
total 200K
402878 -rw-r--r-- 1 user user 0 Sep 5 13:13 error.log
402882 -rw-r--r-- 1 user user 200K Sep 5 13:13 error.log.1
Upon inspection, we can observe that the error.log is still the same file with inode value 402878. Additionally, the logrotate has cleared off the content of error.log during the rotation, and the size is now 0 bytes.
Let’s repeat the experiment using the copy directive:
$ cat logrotate.conf
/tmp/logs/error.log {
size 100k
rotate 2
copy
}
$ logrotate logrotate.conf
$ ls -hil /tmp/logs
total 400K
402878 -rw-r--r-- 1 user user 200K Sep 5 13:16 error.log
402882 -rw-r--r-- 1 user user 200K Sep 5 13:16 error.log.1
Similar to copytruncate, the error.log is the same file. However, instead of clearing off the content of error.log, the log file is still 200 kilobytes in size after the rotation.
The copy directive should be used with care, especially for chatty logs. This is because the non-deleting nature of the copy directive causes both the main log and rotated logs to keep growing in size. Ultimately, it might result in the system running out of disk space.
6. Running Scripts During Rotation
In addition to the predefined directives, logrotate offers the possibility to run custom scripts at different hook points during the rotation. Concretely, we can run attach scripts on the 4 different hook points during rotation:
- Before any rotation
- Before each rotation
- After each rotation
- After all rotation
6.1. Before or After All Rotation
Using the firstaction directive, we can run some custom scripts before any of the rotations take place. On the other hand, logrotate runs the scripts in the lastaction directive right after all the rotation has taken place. In both cases, the logrotate will pass as the first argument the file path pattern to the script.
For example, we can log the timestamp before and after the rotations using these directives:
/tmp/logs/*.log {
size 100k
firstaction
echo "start rotation $(date)" >> /tmp/rotation.log
endscript
lastaction
echo "complete rotation $(date)" >> /tmp/rotation.log
endscript
}
In the configuration above, we’ll write the timestamp into /tmp/rotation.log right before any rotation happens. Once all the rotations are completed, we’ll again append the timestamp to the /tmp/rotation.log file.
6.2. Before or After Each Rotation
To run a script right before each rotation, we can write the script with the prerotate directive. Similarly, we can use a postrotate directive to run scripts after each rotation. When logrotate runs the script, it will pass as the first argument the file path for each log file to the scripts.
For instance, we can write a script to log the moment each log file rotation takes place using the prerotate and postrotate directives:
/tmp/logs/*.log {
size 100k
prerotate
echo "start rotation for $1 $(date)" >> /tmp/rotation.log
endscript
postrotate
echo "complete rotation for $1 $(date)" >> /tmp/rotation.log
endscript
}
The configuration above will produce 2 log lines in the /tmp/rotation.log files for each log file rotated.
6.3. Differences Between firstaction/lastaction and prerotate/postrotate
On the surface, both firstaction/lastaction and prerotate/postrotate seem to be doing the same thing: run scripts before and after the rotation. However, the difference between the pairs of directives is the number of times they are run during the rotations for 1 matcher.
The scripts in the firstaction/lastaction directive pair will only run once for each matcher. On the other hand, logrotate will run the scripts in prerotate/postrotate for each of the matched file paths.
For example, if the pattern matches 3 files, firstaction/lastaction will only run once before and after the 3 files are rotated. On the other hand, the scripts in prerotate/postrotate will execute 3 times, because there are 3 files to be rotated.
To understand them better, let’s look at an example. Firstly, we create a logrotate configuration to demonstrate the differences:
$ cat logrotate.conf
/tmp/logs/*.log {
size 100k
firstaction
echo "on first action. first argument: $1" >> /tmp/event-hooks.log
endscript
lastaction
echo "on last action. first argument: $1" >> /tmp/event-hooks.log
endscript
prerotate
echo "on prerotate. first argument: $1" >> /tmp/event-hooks.log
endscript
postrotate
echo "on postrotate. first argument: $1" >> /tmp/event-hooks.log
endscript
}
Through the configuration, we will rotate all the files that match the path /tmp/logs/*.log once their size hits 100 kilobytes. Then, we define scripts that print the event name and first argument to the file /tmp/event-hooks.log.
Next, we create 3 different files inside the /tmp/logs directory. Furthermore, each of the files is created with 977 kilobytes of data to trigger the rotation:
ls -hl
total 2.9M
-rw-r--r-- 1 user user 977K Sep 5 14:57 access.log
-rw-r--r-- 1 user user 977K Sep 5 14:57 auth.log
-rw-r--r-- 1 user user 977K Sep 5 14:57 error.log
Finally, we can run the command to rotate our files inside the logs directory:
$ logrotate logrotate.conf
After that, we can inspect the /tmp/event-hooks.log to verify the sequence of execution for each script:
$ cat /tmp/event-hooks.log
on first action. first argument: /tmp/logrotate/logs/*.log
on prerotate. first argument: /tmp/logrotate/logs/access.log
on postrotate. first argument: /tmp/logrotate/logs/access.log
on prerotate. first argument: /tmp/logrotate/logs/auth.log
on postrotate. first argument: /tmp/logrotate/logs/auth.log
on prerotate. first argument: /tmp/logrotate/logs/error.log
on postrotate. first argument: /tmp/logrotate/logs/error.log
on last action. first argument: /tmp/logrotate/logs/*.log
7. Summary
In this tutorial, we’ve looked at the logrotate command-line tool in Linux. We started with a brief introduction to logrotate, followed by the installation guide. Then, we’ve learned the different aspects of logrotate, such as directives and the structure of its configuration files.
We then proceed to look at several directives in action. For instance, we see how we can rotate logs based on time or size conditions. Besides that, we’ve also looked at a set of directives that allows us to manipulate the rotated files. After that, we’ve discussed the disruptive nature of the default rotating process. Then, we looked at the alternative copy and copytruncate.
Finally, we’ve explored the firstaction/lastaction and prerotate/postrotate directives pair that allows us to execute scripts in between rotations.