1. Introduction
In this article, we’ll use a simple systemd service and timer to demonstrate how we can redirect the output of a systemd service to a given file.
2. Our Example Service and Timer
We define systemd services in special files called unit files, which end in the .service suffix. These files are loaded from several locations, the two main ones being:
- /usr/lib/systemd/system/ (units provided by packages)
- /etc/systemd/system/ (where units will be added by sysadmins)
systemd timers are unit files that end in the .timer suffix. We can use timers to control systemd service execution and scheduling. Timers are often used as an alternative to cron due to the ease of integration they offer.
For our demonstration, we will have three files placed in /etc/systemd/system/:
- bird_watching.service
- bird_watching.timer
- bird_watcher.sh
Only the shell script will need to be made executable:
$ chmod +x /etc/systemd/system/bird_watcher.sh
2.1. Service Unit File
Let’s take a look at our service unit file:
[Unit]
Description=Bird watching service
[Service]
Type=oneshot
ExecStart=/etc/systemd/system/bird_watcher.sh
[Install]
WantedBy=multi-user.target
This is a very basic service unit file. When made active, it will execute a mock shell script specified in the ExecStart field, which will output the name of a bird species at random.
2.2. Timer Unit File
Now let’s take a look at the timer unit file for our service:
[Unit]
Description=Activate bird_watching.service every minute
[Timer]
OnUnitInactiveSec=1m
[Install]
WantedBy=timers.target
As we see from the content of our timer file above, our timer activates our service every minute based on the time that it has been inactive.
After we create the files, we force systemd to load the units, and then we enable and start the timer. Note that we don’t enable or start the service itself, only the timer associated with it:
$ systemctl daemon-reload
$ systemctl --now enable bird_watching.timer
3. Redirecting Service Output
When our service runs, we can view its output with journalctl, which is the utility used to interact with systemd’s logging component journald:
$ journalctl -u bird_watching -f
-- Logs begin at Wed 2021-01-06 00:45:26 UTC. --
Feb 01 03:07:15 localhost systemd[1]: Starting Bird watching service...
Feb 01 03:07:15 localhost bird_watching[2184673]: Baltimore oriole
Feb 01 03:07:15 localhost systemd[1]: bird_watching.service: Succeeded.
Feb 01 03:07:15 localhost systemd[1]: Finished Bird watching service.
Feb 01 03:08:17 localhost systemd[1]: Starting Bird watching service...
Feb 01 03:08:17 localhost bird_watching[2184726]: Black-capped chickadee
Feb 01 03:08:17 localhost systemd[1]: bird_watching.service: Succeeded.
Feb 01 03:08:17 localhost systemd[1]: Finished Bird watching service.
Feb 01 03:09:41 localhost systemd[1]: Starting Bird watching service...
Feb 01 03:09:41 localhost bird_watching[2184985]: House sparrow
Feb 01 03:09:41 localhost systemd[1]: bird_watching.service: Succeeded.
Feb 01 03:09:41 localhost systemd[1]: Finished Bird watching service.
However, as is the topic of this article, we may want our service to log to a specific file as well. We’ll consider two options for achieving this. The first option will utilize integration with rsyslog, while the second uses systemd directly but requires a newer version of systemd.
3.1. Integration With rsyslog
With rsyslog installed and running on our host, we add some additional lines to our service unit file:
[Unit]
Description=Bird watching service
[Service]
Type=oneshot
ExecStart=/etc/systemd/system/bird_watcher.sh
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=bird_watching
[Install]
WantedBy=multi-user.target
The three new lines will instruct systemd to use our rsyslog service to manage the stdout and stderr of the script executed by our service. Again, whenever our service file changes, we need to have systemd reload the unit:
$ systemctl daemon-reload
Now, we need to add configuration in rsyslog so that it knows how to handle the logs we’ll be sending it. Let’s add the following to a new file /etc/rsyslog.d/bird_watching.conf:
if $programname == 'bird_watching' then /var/log/bird_watching.log
& stop
Now, let’s create the file and make sure the syslog user can access and write to it:
$ touch /var/log/bird_watching.log
$ chown syslog /var/log/bird_watching.log
$ ls -l /var/log/bird_watching.log
-rw-r----- 1 syslog adm 122664 Feb 1 04:17 /var/log/bird_watching.log
With this done, let’s cycle the rsyslog service and check our log file:
$ systemctl restart rsyslog
$ tail -f /var/log/bird_watching.log
Feb 1 04:12:41 localhost bird_watching[2188458]: Black-capped chickadee
Feb 1 04:13:42 localhost bird_watching[2188511]: Baltimore oriole
Feb 1 04:14:56 localhost bird_watching[2188564]: House sparrow
Feb 1 04:15:57 localhost bird_watching[2188620]: Northern cardinal
Feb 1 04:17:11 localhost bird_watching[2188709]: Blue jay
3.2. Direct Integration for Newer systemd Versions
If our host is running a systemd version of 240 or newer, we can take advantage of a feature that will allow us to redirect our stdout and stderr without invoking the services of a syslog daemon:
$ systemctl --version
systemd 245 (245.4-4ubuntu3.13)
[redacted]
By running the command above, we can verify that we have a systemd version that contains this feature.
Now that we know the feature is available, let’s configure our service unit file to use it:
[Unit]
Description=Bird watching service
[Service]
Type=oneshot
ExecStart=/etc/systemd/system/bird_watcher.sh
StandardOutput=append:/var/log/bird_watching.log
StandardError=append:/var/log/bird_watching.log
[Install]
WantedBy=multi-user.target
This configuration will create the file if it doesn’t exist and open it in append mode.
Next, let’s ask systemd to reload the unit, as we’ve made changes to it:
$ systemctl daemon-reload
Now, we can watch the file for our service’s stdout and stderr:
$ tail -f /var/log/bird_watching.log
American robin
House sparrow
Baltimore oriole
American robin
Blue jay
This option lacks the additional details (timestamps) of the rsyslog approach from above but has the benefit of not requiring an additional syslog daemon.
4. Conclusion
In this article, we covered the basics of systemd services and timer files. We also discussed two ways of redirecting the stdout and stderr of a systemd service to a specified file. We hope this has been useful and that it may help you to better control the logging of your systemd services.