1. Introduction
One of the more versatile tools we have at our disposal is ssh. More widely known as the remote access tool of choice for administering remote hosts, it has so many additional handy functionalities that are really useful to know. For example, we can use ssh to transfer files, mount remote hosts as network filesystems, and generate keys.
*In this tutorial, we’ll dig a little further into ssh‘s proxying and tunneling capabilities*.
2. Concepts
From the earliest days of the TCP/IP stack, remote node access was a must. However, the protocols designed for it – telnet for interactive access and FTP for file transfers – lacked a very basic security measure: The data flow was not encrypted.
As a result, the data transferred using these protocols can be captured and scrutinized by network traffic analysis tools like tcpdump and Wireshark. As the username and password were exchanged in clear text, it’s easy for an attacker to gather them.
To address this issue and other limitations, ssh implements encryption in its connections. That way, provided it is using adequate configuration, strong encryption, and key exchange schemas, we can be sure that our communication won’t be intercepted on the way.
One of the more useful features, and not widely known, is the ability to establish tunnels and proxies. This way, we can use an ssh session to connect other remote services that are not visible to us otherwise, like the ones protected by a firewall:
As seen in the picture, a client machine, while opening the ssh session, instructs the SSH server to open tunnels that can work in both directions.
It can also work as a proxy for XWindows protocol, allowing us to locally open remote GUI applications, or as a SOCKS 4 or 5 compliant proxy server, allowing the client to access multiple destinations from the remote site impersonating the SSH Server.
Even more impressive and dangerous, as it enables a full VPN-like experience, is the ability to tunnel level 2 or level 3 packets using tun devices.
3. Server Configuration
First of all, we might note that allowing tunnels through ssh connections might be a security liability, as it can pierce our firewall policies. As such, care must be taken to not expose services and hosts that we might prefer to keep closed to unwanted access. For this reason, some Linux distributions disable tunneling and proxying by default.
3.1. SSHD Options
The enablement of sshd, the daemon that serves ssh sessions, is done by editing the sshd_config file. Its location varies a little but is usually on /etc/ssh or /etc/openssh. The relevant configuration keys are:
- AllowStreamLocalForwarding: Allows Unix domain sockets to be forwarded. The default, when omitted, is yes
- AllowTcpForwarding: Allows TCP port forwarding. The default, when omitted, is to allow. It enables single TCP port forwards and socks proxying
- DisableForwarding: Disables all kinds of forwarding. Override, if enabled, all other related configurations options
- GatewayPorts: Allows other hosts to use the ports forwarded to a client (reverse tunnels). By default, only the hosts running the SSH server can use reverse tunnels. Disabled by default
- PermitListen: Specifies the addresses and ports that can be bound to allow port-forwarding to clients. It provides more fine control if we enable GatewayPorts. The default is localhost (‘127.0.0.1’ and ‘::1’)
- PermitOpen: Specifies the address and ports a TCP forwarding may point to. By default, any destination is enabled
- PermitTunnel: Specifies whether tun device forwarding is allowed. Default is no
- X11Forwarding: Specifies whether X11 forwarding is allowed. Default is no
- X11UseLocalhost: Forces the X11 forwarding to be only allowed from the SSH server host loopback address. If disabled, other hosts on the SSH server network might use it. Default is true
3.2. Other Configurations
Other configurations on the host might affect the ssh‘s ability to forward and proxy. AppArmor and SELinux might inhibit some of these options. Also, some host firewall configurations might limit the ability to connect to and from external services (see our tutorial on iptables). Note that binding listening ports under 1024, by default, requires root privileges.
And, of course, the in-between firewalls must allow the SSH traffic, usually on port TCP/22, but we can use other ports by changing the default on the sshd_config file.
4. Forward TCP Tunnels
4.1. Single-Port
A forward or direct TCP tunnel is the one that follows the direction of the SSH connection from the client to the SSH server. Our introductory tutorial on SSH briefly describes this type of forwarding. To create a direct TCP forward tunnel, we have to use the -L option on the command line:
ssh -L [bind_address:]port:host:hostport [user@]remote_ssh_server
The optional bind_address assigns a client local interface to listen for connections. If we omit it, ssh binds on the loopback interfaces only. We can also use “0.0.0.0” or “::” to bind on all interfaces. So, if we issue the following command:
ssh -L 0.0.0.0:8022:10.1.4.100:22 [email protected]
We would have an SSH connection opened to the host on the 10.1.4.20 IP address and a tunnel, listening on the client port 8022, pointing to the SSH address on host 10.1.4.100.
That way, if a connection goes into client port 8022, it will be forwarded to the destination host and port, using the SSH server IP address, looking exactly like a regular local network between them.
Similarly, to forward local sockets (somewhat less usual), we can use:
ssh -L local_socket:host:hostport [user@]remote_ssh_server
Or we can use:
ssh -L local_socket:remote_socket [user@]remote_ssh_server
4.2. Dynamic or Multi-Port
A special case of the forward TCP tunnels is the Socks proxy capability. Using these options, the SSH client listens on a specified binding port and acts as a SOCKS 4 or 5 proxy server.
Any connections using SOCKS protocol to the binding port will be forwarded to the SSH server using its own IP address. To do that, we would use:
ssh -D [bind_address:]port [user@]remote_ssh_server
Note that we don’t even need to specify the destination host and port for the forwarding in this case. All SOCKS compliant incoming connections on the specified port will flow through the tunnel.
To use it, we must, of course, configure the application that will use the tunnel to use a proxy server on the bound address and port specified on the command line. For instance, after issuing:
ssh -D 8080 [email protected]
We can configure a browser on the client host to use our SOCKS proxy server on 127.0.0.1, port 8080. That way, we can navigate the web as if we were using a browser installed on the SSH server located at 10.1.4.100.
And what if the client application does not support SOCKS proxying? We can use solutions such as proxychains or tsocks that intercept sockets systems calls and force the connections to flow through a SOCKS proxy.
5. Reverse Tunnels
5.1. Single-Port
The reverse or callback proxies allow us to do tricks similar to the one above but in the reverse direction. We can open services on our own local networks to hosts on the remote side of the SSH session. The command syntax is quite similar to the direct forward:
ssh -R [bind_address:]port:host:hostport [user@]remote_ssh_server
This creates a reverse tunnel. It forwards any connection received on the remote SSH server to the bind_address:port to local client network host:hostport. If we omit the bind_address parameter, it binds to the loopback interfaces only.
Similarly, using sockets, we can use three different syntaxes:
ssh -R remote_socket:host:hostport [user@]remote_ssh_server
ssh -R remote_socket:local_socket [user@]remote_ssh_server
ssh -R [bind_address:]port:local_socket [user@]remote_ssh_server
5.2. Dynamic or Multi-Port
Finally, we can expose a SOCKS proxy server on the remote host directed to the client’s network as we can do with direct forwarding. We can do this only by omitting the local destination host and port:
ssh -R [bind_address:]port [user@]remote_ssh_server
This opens a port on the remote SSH server that’ll serve as a SOCKS server to the local client network, potentially piercing any outbound traffic rules that would otherwise apply to the remote SSH server.
6. X Windows Tunnels
One special case of reverse tunneling is the ability to tunnel X11 connections. That way, GUI applications running on the remote side of SSH connections can make use of the local side running X Servers to expose their user interfaces.
The SSH takes care of establishing the needed tunnels. Also, it sets the DISPLAY environment variables needed by the X client applications on the SSH server. That way, they’ll know how to correctly connect the local client’s X server.
There are two kinds of X11 forwarding, applying the X11 Security Extension restrictions (see xhost and xauth, for reference):
ssh -X [user@]remote_ssh_server
Or creating the tunnel assuming a trusted environment that will not enforce X11 Security Extension:
ssh -Y [user@]remote_ssh_server
7. Multiple Tunnels and Multiple Host Hopping
We can create as many tunnels as we need, mixing types and directions. We can accomplish this by adding more options to the command line:
ssh -X -L 5432:<DB server IP>:5432 -R 873:<local RSYNC server>:873 [user@]remote_ssh_server
This opens a direct forward to a remote PostgreSQL server, a reverse tunnel to a local rsync server, and allows GUI applications to flow to our local X Server.
And we can also use SSH tunnels to reach farther SSH servers, creating tunnels to them piercing through as many firewall layers as we need, creating the tunnel on each connection until we can reach the desired point:
ssh -L 8022:<server2>:22 user@server1
ssh -L 8023:<server3>:22 -p 8022 user@localhost
ssh -p 8023 user@localhost
That sequence creates, on each step, a tunnel to the next server, from server1, and server2, until the tunnel opened on local port 8023 allows us to reach server3.
8. Configuration Files
In complex scenarios, creating multiple tunnels on the command line might prove to be tricky, as it can lead to really long command lines.
That’s why one of the most lovely features of ssh is allowing any command-line parameters in the config files. We can use the global ssh client config file (located on /etc/ssh/ssh_config or/etc/openssh/ssh_config) or use our user’s specific configuration file that is located at ~/.ssh/config. If it doesn’t exist, which is the default, we’ll have to create a new one.
In these files, we can specify default configurations to each commonly used endpoint, including forwarding tunnels and proxies:
host 10.1.4.100
ForwardX11 yes
LocalForward 0.0.0.0:5432 10.1.4.200:5432
RemoteForward localhost:8022 localhost:22
user baeldung
This will connect to the remote SSH server on 10.1.4.100, using user ‘baeldung‘, allowing:
- X Windows reverse tunnel
- Direct tunneling from the local port 5432 to remote host 10.1.4.200 port 5432
- Reverse/callback tunnel on port 8022 in the loopback interfaces of the SSH server to our local client host
A lot of other options are available, like compression, Kerberos authentication forwarding, and many others. Also, the host specification allows wildcards.
9. Persistent Tunnels
By the way, an SSH tunnel only exists as long as the SSH connection holds. Even if we can even configure the frequency and timeout for the session keepalives to facilitate the connection-loss detections, it would be nice to fully automate the SSH session creation and reconnection.
For that, a handy piece of software is autossh. This utility can automatically create and recreate SSH sessions. If we add authentication keys, as shown on our SSH keys tutorial, the tunnels will open without user intervention, as long as autossh is running. Its syntax is:
autossh [-V] [-M port[:echo_port]] [-f] [SSH_OPTIONS]
- -V: Show autossh version
- -M: Creates a direct tunnel on a port, loop-backed to a reverse one, echo_port. It provides an alive checking mechanism. However, with recent OpenSSH, we can achieve similar results using ServerAliveInterval and ServerAliveCountMax options in the sshd_config file
- -f: Forces autossh to run in the background before running ssh
- SSH_OPTIONS: The options we would use to start ssh
That way, to start a persistent connection, we can use:
autossh -X -L 5432:<DB server IP>:5432 -R 873:<local RSYNC server>:873 [user@]remote_ssh_server
If we’ve defined a host config, the command line is much simpler:
autossh -f [host]
10. Conclusion
This article showed some nice tricks we can do with ssh to improve our reachability to, or from, remote hosts using its tunneling capabilities.
However, we must always keep in mind that ssh access to a host widens its cyberattack surface. So, the use of any kind of tunneling might increase the risks, allowing easier horizontal movement through our SSH server network.
Finally, there’s an even more advanced type of tunneling: the tun device tunneling. That’s even more dangerous, as it acts like a full-blown VPN (Virtual Private Network). This type of tunnel binds the two remote network segments where the endpoints are located.
We can use it to create Layer 2 tunnels that behave as if both sides were on the local network or layer 3 tunnels. In this case, each side has its own IP subnetwork and the SSH endpoints have to be configured to route traffic between them.