1. Introduction
When it comes to Nginx, it’s one of the most popular servers out there. It’s fast, lightweight and responsible for hosting some of the biggest sites on the internet. Nginx is often used as a load balancer, a reverse proxy, and an HTTP Cache, among other uses.
In this tutorial, we are focusing on learning how to use it as a forward proxy for any requested location.
2. The Motivation for a Forward Proxy
Proxy servers are entities that act as middlemen between a client and the host of the requested resource. This means the traffic goes through an additional machine in order to get to the destination (host server). The proxy continues the request on behalf of the client, so when the host server accepts the request, they only see the IP of the proxy. In comparison, a reverse proxy sits directly in front of the web and routes the request coming from the client to the correct web server (inside a network of multiple servers).
The only downside of using forward proxies is that they work on the application level, so we have to set up the proxy for each app we’re planning to route the traffic for.
Some use cases for using a Forward Proxy are:
- Masking the IP and location to gain access to location-restricted services
- For isolated internal networks that need to connect to specific resources on the internet
- For caching requests to specific servers for content that rarely changes in order to save resources
It’s worth noting that proxies do not encrypt traffic, whereas VPNs redirect the traffic through secure and encrypted tunnels.
3. Implementing a Forward Proxy with Nginx
In order to implement a forwarding proxy, we’re going to use a Linux machine with Nginx installed. For the sake of this tutorial, we’ll be using VirtualBox with a Linux distro server that’s up and running, together with Nginx installed, but you can use whatever is more convenient to you like Docker or even the old PC that’s been lying in the corner for years.
First, we locate the default Nginx configuration file and comment out the server part in order to save it as an archived copy. Usually, we can find it in /etc/nginx/sites-enabled/default:
# Default server configuration
#server {
#listen 80 default_server;
#listen [::]:80 default_server;
#root /var/www/html;
# Add index.php to the list if you are using PHP
#index index.html index.htm index.nginx-debian.html;
#server_name _;
#location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
#try_files $uri $uri/ =404;
#}
#}
Next, let’s create a new file called forward and add all the needed configurations to turn Nginx into a working forward proxy:
server {
listen 8888;
location / {
resolver 8.8.8.8;
proxy_pass http://$http_host$uri$is_args$args;
}
}
With the first configuration ‘listen 8888;’ – we’re basically telling the server that all the requests going to this port must be handled with the following configurations. The location argument is responsible for specific server sub-division block configurations and basically tells the server how to handle requests for specific URIs.
The ‘resolver 8.8.8.8’ directive specifies which nameservers should be used to resolve the names of upstream servers into addresses, in this case 8.8.8.8 corresponds to Google’s nameservers.
The variable $http_host contains the host in the original request, whereas $uri contains the path after the domain or IP. The last two variables $is_args and $args check for any additional arguments in the initial request, and they add them automatically to the proxied request.
After we update all the necessary configurations, we need to restart the nginx.service for them to take effect:
sudo systemctl restart nginx.service
4. Using the Forward Proxy
As we mentioned before, forward proxies work on the application level, so naturally, depending on the client, there are multiple ways that we can configure the forward proxy. For this step, we’re going to create a simple client in JavaScript and trace the request.
Before we begin, let’s make sure that the latest node.js and npm are installed on our local machine. Next, we create the directory and the file for the client. Let’s call the directory Proxy Test and the file proxytest.js accordingly.
Next, we need to initialize NPM’s package.json so we can install all the needed libraries. We do this by running the npm init command on the terminal inside our project’s directory:
npm init
After we successfully initialize the repository, we need to install the request library that we’ll use for building the custom request with the proxy configuration:
npm install request
Finally, let’s open an IDE and paste the below code into our proxytest.js file:
var request = require('request');
request({
'url':'http://www.google.com/',
'method': "GET",
'proxy':'http://192.168.100.40:8888'
},function (error, response, body) {
if (!error && response.statusCode == 200) {
console.log(body);
}
})
Now, let’s run this code snippet:
node proxytest.js
Let’s take a step back and look at each line. The first line imports the library into the request object, which we’ll use later on.
Inside the request object, we specify the URL of the destination server, the HTTP Method, and the proxy as URL and port keypair. Inside the callback function, if the request is successful, we log the response body in the console.
Next, let’s have a look at Nginx’s debug logs :
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script copy: "http://"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script var: "www.google.com"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script var: "/"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script var: ""
2022/02/20 13:46:13 [debug] 1790#1790: *1 http init upstream, client timer: 0
2022/02/20 13:46:13 [debug] 1790#1790: *1 epoll add event: fd:7 op:3 ev:80002005
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script copy: "Host"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script var: "www.google.com"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script copy: "Connection"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script copy: "close"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script copy: ""
2022/02/20 13:46:13 [debug] 1790#1790: *1 http script copy: ""
2022/02/20 13:46:13 [debug] 1790#1790: *1 http proxy header:
"GET / HTTP/1.0
Host: www.google.com
Connection: close
"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http cleanup add: 0000560CE3CF5E30
2022/02/20 13:46:13 [debug] 1790#1790: *1 http finalize request: -4, "/?" a:1, c:2
2022/02/20 13:46:13 [debug] 1790#1790: *1 http request count:2 blk:0
2022/02/20 13:46:13 [debug] 1790#1790: *1 http run request: "/?"
2022/02/20 13:46:13 [debug] 1790#1790: *1 http upstream check client, write event:1, "/"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http upstream resolve: "/?"
2022/02/20 13:46:14 [debug] 1790#1790: *1 name was resolved to 142.250.184.100
2022/02/20 13:46:14 [debug] 1790#1790: *1 name was resolved to 2a00:1450:4002:406::2004
2022/02/20 13:46:14 [debug] 1790#1790: *1 get rr peer, try: 2
2022/02/20 13:46:14 [debug] 1790#1790: *1 get rr peer, current: 0000560CE3CF5EB8 -1
2022/02/20 13:46:14 [debug] 1790#1790: *1 stream socket 12
2022/02/20 13:46:14 [debug] 1790#1790: *1 epoll add connection: fd:12 ev:80002005
2022/02/20 13:46:14 [debug] 1790#1790: *1 connect to 142.250.184.100:80, fd:12 #3
As we can see, our initial request goes through the proxy. Immediately after, the proxy server launches the new request, containing all the data from the initial request, to the destination resource. After that, it takes the response from the resource and returns it to our client:
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy status 200 "200 OK"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Date: Sun, 20 Feb 2022 12:46:15 GMT"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Expires: -1"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Cache-Control: private, max-age=0"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Content-Type: text/html; charset=ISO-8859-1"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info.""
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Server: gws"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "X-XSS-Protection: 0"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "X-Frame-Options: SAMEORIGIN"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Set-Cookie: 1P_JAR=2022-02-20-12; expires=Tue, 22-Mar-2022 12:46:15 GMT; path=/; domain=.google.com; Secure"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Set-Cookie: NID=511=IkyJTmMt6I2b3fHpGNUwdfCkv1q9cjzyeUaxC-cxMZWcbmSi4sVlRlwXJUTRA9ujqQnK2v6DNyhitL3zPRSf7RSIHDCv8aYcUD7jp3vX4sE7ZkiprAWmJo9FlnUJtV9H0IzOFyPck15Jfs0zb1VeOMOjKZk0BZ0XRQ3gNptMOl8; expires=Mon, 22-Aug-2022 12:46:15 GMT; path=/; domain=.google.com; HttpOnly"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Accept-Ranges: none"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header: "Vary: Accept-Encoding"
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy header done
2022/02/20 13:46:14 [debug] 1790#1790: *1 xslt filter header
2022/02/20 13:46:14 [debug] 1790#1790: *1 HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Sun, 20 Feb 2022 12:46:14 GMT
Content-Type: text/html; charset=ISO-8859-1
Transfer-Encoding: chunked
Connection: close
Expires: -1
Cache-Control: private, max-age=0
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2022-02-20-12; expires=Tue, 22-Mar-2022 12:46:15 GMT; path=/; domain=.google.com; Secure
Set-Cookie: NID=511=IkyJTmMt6I2b3fHpGNUwdfCkv1q9cjzyeUaxC-cxMZWcbmSi4sVlRlwXJUTRA9ujqQnK2v6DNyhitL3zPRSf7RSIHDCv8aYcUD7jp3vX4sE7ZkiprAWmJo9FlnUJtV9H0IzOFyPck15Jfs0zb1VeOMOjKZk0BZ0XRQ3gNptMOl8; expires=Mon, 22-Aug-2022 12:46:15 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
2022/02/20 13:46:14 [debug] 1790#1790: *1 write new buf t:1 f:0 0000560CE3CF7AD0, pos 0000560CE3CF7AD0, size: 760 file: 0, size: 0
2022/02/20 13:46:14 [debug] 1790#1790: *1 http write filter: l:0 f:0 s:760
2022/02/20 13:46:14 [debug] 1790#1790: *1 http cacheable: 0
2022/02/20 13:46:14 [debug] 1790#1790: *1 http proxy filter init s:200 h:0 c:0 l:-1
2022/02/20 13:46:14 [debug] 1790#1790: *1 http upstream process upstream
When the request is successfully sent to the destination, we see on the logs a response “200 OK” meaning the request was accepted and the response was returned successfully. From our logs we can also see all the HTTP headers that the response returned, listed line by line. Whatever HTTP headers the destination server returns are automatically added to the Proxy return object.
5. Conclusion
In this tutorial, we’ve learned how to set up an easy and lightweight forward proxy using the Nginx server. We’ve learned an important difference between a forward proxy and a VPN. Finally, we’ve also learned how to connect a JavaScript-based client to our newly created forward proxy.
The full source code can be found over on GitHub.