1. Overview
In this tutorial, we’ll look at the issue where the Docker container’s published ports ignore the firewall rules.
2. Dropping Incoming Packets Using UFW
It’s a common security practice to set drop-all rules for incoming packets. Then, the system admin opens up the ports selectively according to needs. It aligns with the principle of least privilege that’s commonly championed in the realm of security.
One simple way to set drop-all rules by default for all the incoming packets is to use the ufw command. Specifically, we run ufw default deny incoming to drop all incoming packets by default:
$ ufw default deny incoming
Behind the scene, the ufw command uses the iptables command to configure the firewall. In our example, the deny incoming default rules inserts network packet drop rules onto the FORWARD and INPUT chain of the iptables. When the system receives an incoming network packet, it’ll see the default drop rules and drop the packet.
Usually, this is good enough to block all the incoming packets to our server. However, if we’re running the Docker engine on our server and we’ve published ports in any of these Docker containers, this won’t be the case.
3. Docker Container’s Publish Ports Ignoring iptables Rules
One problem with the deny incoming default rule we saw in the previous section is that ports that are published by Docker containers ignore the rules. This means, if we have a container that exposes port 80, external traffic will be able to bypass the drop-all rules and reach the container process.
3.1. Demonstration
To demonstrate the issue, we first deny all the incoming network packets using the ufw command:
$ ufw default deny incoming
Then, we start an Nginx container and publish port 81 with the –publish option using docker run:
$ docker run --detach --rm --publish 81:80 nginx:latest
Additionally, we start a local Nginx process that listens on port 80:
$ service nginx start
Up to this point, what we’ve done is start the same Nginx process once in a Docker container exposed at port 81 and another as a local process serving at port 80. With the drop-all default rules, we’d expect both ports to be unreachable from outside.
Let’s send an HTTP request to both of the ports using the curl command from another host:
$ curl --connect-timeout 3 hostA:81
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
$ curl --connect-timeout 3 hostA:80
curl: (28) Failed to connect to hostA port 80 after 3001 ms: Timeout was reached
From the result above, we can see that connection to the port 80 timed out. This is because the packet has been dropped according to the rule. However, the port published by the Docker container, 81, is still accessible from outside.
To understand the phenomena we’ve observed, it’s important to understand the relationship between the Docker engine and the iptables.
3.2. Docker’s Custom iptables Chains
To provide for network isolation, the Docker engine manipulates the host’s iptables. Specifically, the Docker engine adds two custom chains, DOCKER, and DOCKER-USER to the iptables. Additionally, it configures the iptables such that all the incoming packets are evaluated by these two chains first.
As a result, all the incoming packets that are destined for the Docker container exposed ports will be evaluated by these two chains first. But the deny incoming rules are in the INPUT and FORWARD chains. Therefore, incoming network packets targeting the published ports won’t be evaluated by the drop-all rules, rendering them ineffective.
Let’s look at some workaround for this issue.
4. Solution 1: Don’t Publish Port Using -p
One way we can prevent the Docker container from accidentally exposing unnecessary ports is to not publish the ports in the first place. That means, we start the Docker containers without the -p or –publish options. This would ensure no incoming packets can bypass the rules we’ve set in the INPUT and FORWARD chains.
Of course, a lot of the Docker containers need to handle incoming traffic, perhaps from other containers in the same host, to be useful. If we want to allow network traffic from other Docker containers within the same host, we can create a Docker network and links the two containers together.
For example, first, we create an Nginx Docker container without publishing any ports:
$ docker --name server -d nginx:latest
Then, we’ll create another container that acts as the client attempting to connect to the Nginx container:
$ docker --name client -d ubuntu:latest
Now, we’ll create a Docker network internal-link and connect both of the containers to this network:
$ docker network create internal-link
$ docker network connect internal-link server
$ docker network connect internal-link client
By connecting these two containers to the same network, we are enabling network packets to be exchanged between them.
To test it out, we can drop it into the shell of the client container. Then, we send an HTTP request to the server container:
$ docker exec -it client bash
$ curl server:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
5. Solution 2: Bind Container Locally
Alternatively, if we require the Docker containers to accept network traffic from other processes in the same host, we can bind the ports to the local interface. By binding the exposed ports to the local loopback interface, it prevents the process from accepting network packets from the external network.
To bind a published port to the local loopback interface, we prefix the host’s port in the argument pair with the local loopback IP address 127.0.0.1:
$ docker run --detach --name nginx --publish 127.0.0.1:80:80 nginx
The command above runs an Nginx container and exposes port 80 on the local loopback interface only. In other words, only network packets originating from the local using the local loopback interface get a response.
6. Solution 3: Disable iptables in daemon.json
This solution prevents the Docker engine from modifying the iptables, thereby preventing the Docker engine from creating and prioritizing the custom chains.
To stop the Docker engine from modifying the iptables, we first create a file /etc/docker/daemon.json if it doesn’t already exist. Then, we add the attribute iptables and set it to false:
$ cat /etc/docker/daemon.json
{
"iptables": false
}
Then, we have to restart the Docker engine using the service command for it to take effect:
$ service docker restart
Notably, this solution comes with side effects. For example, it causes the Docker containers to lose outbound network access. Besides that, the bridge network might not work correctly as it relies on manipulating the iptables to facilitate it.
7. Conclusion
In this article, we first looked at the ufw command and learned how to set a default policy to drop all incoming packets.
Then, we learned that Docker containers’ exposed ports won’t respect the drop-all rules.
Finally, we’ve looked at several solutions, including not exposing the port at all, binding the port locally, and disabling the iptables manipulation on Docker entirely.