1. Overview
In this tutorial, we’ll get a deeper understanding of monitoring device events in Linux. First, we’ll overview the key terms like kernel, udev, and udevadm, followed by hands-on exercises to solidify our knowledge.
2. Basics of Device Events in Linux
In Linux, devices are files in the /dev/ directory. Let’s run a command to see a partial listing of those particular files:
$ ls -lrt /dev/
total 0
drwxrwxrwt 2 root root 40 Jul 23 11:24 mqueue
drwxr-xr-x 2 root root 60 Jul 23 11:24 mapper
drwxr-xr-x 2 root root 60 Jul 23 11:24 dma_heap
drwxr-xr-x 2 root root 60 Jul 23 11:24 vfio
drwxr-xr-x 2 root root 60 Jul 23 11:24 net
...
But how are these files created? To answer this, we can insert a flash drive into our system. As a result, a file for the device is either dynamically created or becomes accessible in the /dev/ directory.
In detail, this is made possible by a userspace program called udev, a dynamic device manager for the Linux kernel. Whenever a device is added or removed, the kernel sends a notification of the change to udev.
So, udev listens for these uevents from the kernel. For every event, systemd-udevd (the udev daemon) runs specific instructions defined for handling each device. In fact, the /etc/udev/rules.d/ and /lib/udev/rules.d/ directories define this set of rules.
Below are the contents of /etc/udev/rules.d/:
$ ls -l /etc/udev/rules.d/
total 84
-rw-r--r-- 1 root root 10302 Jul 11 22:37 70-snap.firefox.rules
-rw-r--r-- 1 root root 1389 Jul 11 22:37 70-snap.snapd-desktop-integration.rules
-rw-r--r-- 1 root root 63312 Jul 11 22:37 70-snap.snapd.rules
-rw-r--r-- 1 root root 3121 Jul 11 22:37 70-snap.snap-store.rules
The above output displays the custom rules for udev.
Next, let’s see a partial listing of /lib/udev/rules.d/:
$ ls -l /lib/udev/rules.d/
total 624
-rw-r--r-- 1 root root 991 Mac 25 12:53 39-usbmuxd.rules
-rw-r--r-- 1 root root 998 Jan 20 2019 40-usb-media-players.rules
-rw-r--r-- 1 root root 42861 Sep 6 2021 40-usb_modeswitch.rules
-rw-r--r-- 1 root root 655 Apr 7 22:28 40-vm-hotadd.rules
-rw-r--r-- 1 root root 165 Okt 26 2021 50-apport.rules
...
The output above shows the default rules for udev.
3. Monitor Device Events with udevadm
In this section, we’ll monitor the handling of devices. Importantly, we can manipulate the behavior of the udev daemon using the udevadm tool.
3.1. Using the udevadm monitor Command
The udevadm tool monitors a device from when it’s plugged into the system to when it’s removed. So, to display this activity, we can use the udevadm monitor command:
$ udevadm monitor
monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent
We can see that the udevadm tool has now started monitoring device events. So now, let’s see what happens when we plug a disk into our system:
$ udevadm monitor
monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent
KERNEL[40495.431179] change /devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:0/1:0:0:0/block/sr0 (block)
UDEV [40501.736535] change /devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:0/1:0:0:0/block/sr0 (block)
KERNEL[40503.777729] change /devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:0/1:0:0:0/block/sr0 (block)
UDEV [40506.864856] change /devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:0/1:0:0:0/block/sr0 (block)
We can see some activity. In particular, the device named sr0 at /block/sr0 is the disk we just inserted. First, the kernel sends a notification to udev for a change in that device. Next, udev updates the device’s state.
3.2. Displaying Device Properties
Next, let’s take a look at the device properties by examining the udev database. For this, we’ll need the info command with a path (-p):
$ udevadm info -a -p /block/sr0
Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.
looking at device '/devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:0/1:0:0:0/block/sr0':
KERNEL=="sr0"
SUBSYSTEM=="block"
DRIVER==""
ATTR{alignment_offset}=="0"
ATTR{capability}=="109"
From the output, we get to know the device attributes in a key-and-value format due to the -a flag. If we look at the attribute SUBSYSTEM, we can see that its value is block, meaning a block device.
4. Monitor Device Events with evtest
evtest is a tool that keeps track of an input device by showing generated events.
First, let’s install the tool:
# apt-get install evtest
Before we continue, it’s important to note that we’ll need to switch to a virtual terminal. But why do that? Well, when a device is active, a process takes it over. This hinders evtest from displaying events. To emphasize, switching to a virtual terminal breaks this hold on the device.
Next, let’s look at the input devices file in the /proc pseudo-system:
$ cat /proc/bus/input/devices
I: Bus=0011 Vendor=0002 Product=0007 Version=01b1
N: Name="SynPS/2 Synaptics TouchPad"
P: Phys=isa0060/serio1/input0
S: Sysfs=/devices/platform/i8042/serio1/input/input5
U: Uniq=
H: Handlers=mouse0 event4
B: PROP=9
B: EV=b
B: KEY=6420 30000 0 0 0 0
B: ABS=260800011000003
...
The output shows a selected device file which we’ll analyze. In particular, we’ll focus on the /dev/input/event4 file (from the Handlers section above), responsible for mouse input:
# evtest /dev/input/event4
Input driver version is 1.0.1
Input device ID: bus 0x11 vendor 0x2 product 0x7 version 0x1b1
Input device name: "SynPS/2 Synaptics TouchPad"
Supported events:
Event type 0 (EV_SYN)
Event type 1 (EV_KEY)
Event code 272 (BTN_LEFT)
Event code 273 (BTN_RIGHT)
Event code 325 (BTN_TOOL_FINGER)
Event code 330 (BTN_TOUCH)
Event code 333 (BTN_TOOL_DOUBLETAP)
Event code 334 (BTN_TOOL_TRIPLETAP)
...
evtest* shows data received from the *kernel. The above snippet is just a part of the long evtest output. It shows basic device data, as well as generated events.
5. Conclusion
In this article, we saw that the udevadm and evtest tools are instrumental when working with device events in the Linux system. Furthermore, we also reviewed important key topics like kernel and userspace udev components. With this knowledge, we can keep track of device events with ease.