1. Introduction
Docker is a ubiquitous tool and platform that enables easier development and containerization of applications. With it, we can encapsulate a program with its dependencies, so it’s more convenient to handle, ship, and run, regardless of the underlying hardware or operating system (OS). Still, there are times when we might want to expose a physical component from the host to the container.
In this tutorial, we explore ways to enable a containerized application to access a host device. First, we briefly refresh our knowledge about privileged containers. After that, we show how using privileges for a container can enable shared device access. Next, we go over two methods to expose host devices within containers without full privileges. Finally, we explain how to simulate hotplug functionality and device passthrough.
Importantly, we consider all environments that we work with have successfully executed two commands:
$ apt-get update && apt-get install tree usbutils udev
To indicate commands that should run within a container, we use the entire prompt of that specific container. Further, we use a USB device for our examples, but the concepts can mostly be applied to serial devices as well.
We tested the code in this tutorial on Debian 12 (Bookworm) with GNU Bash 5.2.15. It should work in most POSIX-compliant environments unless otherwise specified.
2. Privileged Container Mode
Since any raw machine or component access may require it, let’s first discuss privileged mode in relation to containers.
In practice, one of the main ideas of a container is isolation from the host. So, it usually achieves this by separating several namespace categories:
- PID Namespace (pid): processes
- Network Namespace (net): networks
- Mount Namespace (mnt): mounts and filesystems
- UTS Namespace (uts): UNIX time sharing, hostname and domain name
- IPC Namespace (ipc): inter-process communication
- User Namespace (user): user and group identifiers
- Cgroup Namespace (cgroup): control groups
Collectively, these isolated components should comprise a full environment without any control over the host.
Using –privileged to allow a container access to the host in any category can result in a complete takeover. For example, in perhaps one of the most critical scenarios, containers can be permitted to change the cgroup options. In this case, any container can take control over others. In another example, the same applies to privileged mnt operations: we could gain full access to the host filesystem.
Because of this, running a privileged container is generally discoured. If we do decide to use the option, best practices dictate that we should isolate it via other means, such as SELinux.
3. Privileged Container Device Sharing
When it comes to raw devices and access, we can directly use the –privileged option:
$ docker run --privileged --tty --interactive debian /bin/bash
With this fairly basic command, we perform several actions:
- create new container
- make container –privileged
- base contaner on debian DockerHub image
- name container debian (implicitly)
- start –interactive session with —tty within container
- run /bin/bash within container
Thus, we should be able to interact with the resulting instance directly.
root@6660deadbeef:/# tree /dev/bus/usb
/dev/bus/usb
`-- 001
|-- 001
`-- 002
root@6660deadbeef:/# lsusb
Bus 001 Device 002: ID 0666:4301 X Corp. Mass Storage Device
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Here, we use both tree on /dev/bus/usb/ (USB bus devices) and the lsusb command to identify USB devices. Consequently, we see a generic USB host controller as Device 001 of Bus 001. On the other hand, Device 002 of the same bus is a Mass Storage Device.
To get more information, we can also employ usb-devices:
root@6660deadbeef:/# usb-devices
T: Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#= 1 Spd=480 MxCh= 1
D: Ver= 2.00 Cls=09(hub ) Sub=00 Prot=01 MxPS=64 #Cfgs= 1
P: Vendor=1d6b ProdID=0002 Rev=06.05
S: Manufacturer=Linux Foundation 2.0
S: Product=root hub
S: SerialNumber=6060100101
C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub
E: Ad=81(I) Atr=03(Int.) MxPS= 4 Ivl=256ms
T: Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 3 Spd=480 MxCh= 0
D: Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1
P: Vendor=0666 ProdID=4301 Rev=01.00
S: Manufacturer=X Corp.
S: Product=Mass Storage Device
S: SerialNumber=6660430100
C: #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=2mA
I: If#= 0 Alt= 0 #EPs= 2 Cls=08(stor.) Sub=06 Prot=50 Driver=usb-storage
E: Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=125us
E: Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
Since one of these devices is a mass storage medium, we can attempt to identify which of the block devices is associated with it:
$ udevadm info --path=/sys/block/sdb --query=env
DEVPATH=/devices/platform/hub.0/usb1/1-1/1-1:1.0/host4/target4:0:0/4:0:0:0/block/sdb
DEVNAME=/dev/sdb
DEVTYPE=disk
DISKSEQ=32
MAJOR=8
MINOR=16
SUBSYSTEM=block
In this case, we employ udevadm to –query the path of our suspect block device at /sys/block/sdb in the /sys pseudo-filesystem. Thus, we find out that it is indeed linked to the mass storage device we see from the host.
Let’s compare this output with the host output for the same device:
$ udevadm info --path=/sys/block/sdb --query=env
DEVPATH=/devices/platform/hub.0/usb1/1-1/1-1:1.0/host4/target4:0:0/4:0:0:0/block/sdb
DEVNAME=/dev/sdb
DEVTYPE=disk
DISKSEQ=32
MAJOR=8
MINOR=16
SUBSYSTEM=block
USEC_INITIALIZED=50765557396
ID_BUS=usb
[...]
ID_USB_VENDOR_ID=0666
ID_USB_REVISION=0605
ID_USB_TYPE=disk
ID_USB_INSTANCE=0:0
ID_USB_INTERFACES=:080650:
ID_USB_INTERFACE_NUM=00
ID_USB_DRIVER=usb-storage
ID_PATH_WITH_USB_REVISION=platform-hub.0-usbv2-0:1:1.0-scsi-0:0:0:0
ID_PATH=platform-hub.0-usb-0:1:1.0-scsi-0:0:0:0
ID_PATH_TAG=platform-hub-usb-0_1_1_0-scsi-0_0_0_0
ID_FS_UUID=4558-AA4F
ID_FS_UUID_ENC=4558-AA4F
ID_FS_VERSION=FAT16
ID_FS_TYPE=vfat
ID_FS_USAGE=filesystem
[...]
TAGS=:systemd:
CURRENT_TAGS=:systemd:
Notably, the host seems to have a more complete picture of the device, despite the –privileged startup and root permissions.
Since /dev/sdb is our USB mass storage device, we can attempt to mount it as such:
root@6660deadbeef:/# mkdir /mnt/usb
root@6660deadbeef:/# mount /dev/sdb /mnt/usb
root@6660deadbeef:/# ls /mnt/usb/
file1 file2
After creating a mount point and mounting the /dev/sdb device, we can see its contents.
4. Non-privileged Container Device Sharing
To see host devices in a non-privileged container, we can use two separate options to expose them in different ways.
For example, a USB device can be seen as individual entities:
- raw USB controller
- the mass storage device
- block device
- actual (mounted) contents
So, let’s explore each of these aspects.
4.1. Using –volume
The –volume option can mount a given path from the host to the same or a different path in the container:
$ docker run --tty --interactive --volume=/dev/bus/usb:/dev/bus/usb --volume=/dev/sdb:/dev/sdb debian /bin/bash
In this case, we expose both the /dev/bus/usb and /dev/sdb paths we used before to the same paths within the container:
root@6670beddeaf0:/# lsusb
Bus 001 Device 002: ID 0666:4301 X Corp. Mass Storage Device
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Consequently, we can perform all other checks as well.
However, we can’t mount /dev/sdb:
root@6670beddeaf0:/# mkdir /mnt/usb
root@6670beddeaf0:/# mount /dev/sdb /mnt/usb
mount: /mnt/usb: permission denied.
dmesg(1) may have more information after failed mount system call.
That’s because –volume exposes the path as is without allowing access to the underlying raw device. In other words, although it looks like one, /dev/sdb doesn’t behave like a block device as far as the container is concerned.
So, apart from metadata verification, the –volume option is usually best suited for sharing mounts with the host:
$ docker run --tty --interactive --volume=/mnt/usb:/mnt/usb debian /bin/bash
In this case, if /mnt/usb/ is the mount point of our device on the host, we allow the container access to its contents via a volume.
4.2. Using –device
On the other hand, the –device option can share a specific raw device with the container without breaking its general isolation:
$ docker run --tty --interactive --device=/dev/bus/usb --device=/dev/sdb debian /bin/bash
In this case, the paths are replicated between the host and container.
First, let’s verify we have access to the same USB device:
root@6680beaddeed:/# lsusb
Bus 001 Device 002: ID 0666:4301 X Corp. Mass Storage Device
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Now, we can attempt a mount of /dev/sdb:
root@6680beaddeed:/# mkdir /mnt/usb
root@6680beaddeed:/# mount /dev/sdb /mnt/usb
mount: /mnt: permission denied.
dmesg(1) may have more information after failed mount system call.
Again, we see a permission denied message. Yet, this time, the reason is different.
To understand it, let’s recreate the container in another way:
$ docker run --security-opt apparmor=unconfined --cap-add SYS_ADMIN --tty --interactive --device=/dev/bus/usb --device=/dev/sdb debian /bin/bash
In this case, we add two additional options with specific values:
- –security-opt removes the AppArmor confinement
- –cap-add permits all SYS_ADMIN activities, which include mounting
Notably, we can specify both of these options several times, for different security options and capabilities. For example, some systems use other means of security confinement as well:
- –security-opt seccomp=unconfined
- –security-opt label:disable
In any case, after reconfiguring the security and capabilities, we can retry the mount operation with success:
root@6690bed0deb:/# mkdir /mnt/usb
root@6690bed0deb:/# mount /dev/sdb /mnt/usb
root@6690bed0deb:/# ls /mnt/usb/
file1 file2
At this point, we removed just enough isolation, so we can expose and permit the mounting of a specific device.
5. Non-privileged Container cgroup Rules
The solutions we already covered address device sharing but don’t consider one case: plugging or replugging the device while a container is already running. In this special scenario, devices can get a different bus ID, thereby rendering the original share obsolete.
Notably, this isn’t a problem when sharing the whole /dev/bus/usb/ directory, but might present challenges for serial devices. While we can just use –volume=/dev:/dev to expose /dev, this basically makes the host isolation obsolete.
Instead, to account for this situation without using –privileged or breaking security, we can reconfigure cgroup rules.
To begin with, *we check the information about the tty devices via the [-l]ong mode of ls*:
$ ls -l /dev/ttyS*
crw-rw---- 1 root dialout 4, 64 Feb 2 10:00 /dev/ttyS0
crw-rw---- 1 root dialout 4, 65 Feb 2 10:00 /dev/ttyS1
crw-rw---- 1 root dialout 4, 66 Feb 2 10:00 /dev/ttyS2
crw-rw---- 1 root dialout 4, 67 Feb 2 10:00 /dev/ttyS3
In this case, we see 4 is the major group for all tty devices. To make our examples more universal, we assign this number to a variable:
$ MAJOR_TTY_GROUP=4
After that, we recreate a container with a new switch:
$ docker run --device-cgroup-rule='c '$MAJOR_TTY_GROUP':* rwm' --tty --interactive debian /bin/bash
The –device-cgroup-rule we define applies to :* all devices within $MAJOR_TTY_GROUP and permits all available actions:
- r (read)
- w (write)
- m (mknod)
However, to ensure plug events trigger reassignment, we also need to add a udev rule:
$ cat /etc/udev/rules.d/90-docker-tty.rules
ACTION=="add", SUBSYSTEM=="tty", RUN+="/usr/shr/bin/docker-tty.sh 'plugged' '%E{DEVNAME}' '%M' '%m'"
ACTION=="remove", SUBSYSTEM=="tty", RUN+="/usr/shr/bin/docker-tty.sh 'unplugged' '%E{DEVNAME}' '%M' '%m'"
This way, udev* calls /usr/shr/bin/docker-tty.sh on each add or remove operation within the tty *SUBSYSTEM.
Finally, we create the /usr/shr/bin/docker-tty.sh and make it executable:
$ cat /usr/shr/bin/docker-tty.sh
#!/usr/bin/env bash
CONTAINERNAME=<CONTAINER_NAME>
if [ -n "$(docker ps --quiet --filter name=$CONTAINERNAME)" ]; then
if [ "$1" == "plugged" ]; then
docker exec --user root env_dev mknod $2 c $3 $4
docker exec --user root env_dev chmod --recursive 777 $2
else
docker exec --user root env_dev rm $2
fi
fi
$ chmod +x /usr/shr/bin/docker-tty.sh
Thus, we can insert and remove tty* devices, and the container named CONTAINERNAME should react by executing the relevant commands to add or remove the device within its context.
6. Summary
In this article, we talked about exposing raw device types like USB or serial to enable device sharing between a container and its host.
In conclusion, although we can expose raw devices via the –privileged switch, there are more refined methods to allow just enough access to the container without fully breaking its isolation.