通过命令行在Ubuntu中选择最快的镜像

最后修改:2023年12月17日

by Francesco Galgani

1. Overview

apt mirrors are server locations around the world that host software packages. By selecting a mirror closer to our geographic location and with a faster connection, we can significantly speed up the process of installing and updating software.

In this tutorial, we’ll explore several command-line methods for determining the fastest apt mirror for Ubuntu.

We’ll focus on a fresh installation of Ubuntu Server 22.04 LTS, although the methods we show work with other versions of Ubuntu as well.

2. Check, Backup, and Benchmark sources.list

First, let’s identify our current mirror by checking sources.list:

$ cat /etc/apt/sources.list

The problem is that the output may be overly verbose because of the comments. Let’s use grep to filter it by removing comments and empty lines:

$ grep -v '^#' /etc/apt/sources.list | grep -v '^$'
deb http://th.archive.ubuntu.com/ubuntu jammy main restricted
deb http://th.archive.ubuntu.com/ubuntu jammy-updates main restricted
[...]
deb http://th.archive.ubuntu.com/ubuntu jammy-security multiverse

This filtered output shows that our current mirror is th.archive.ubuntu.com. To be on the safer side, let’s make a backup:

$ sudo cp /etc/apt/sources.list /etc/apt/sources.list.backup

Let’s force apt to ignore the cache and download the package information again so we can estimate the current mirror performance:

sudo rm -rf /var/lib/apt/lists/* && sudo apt update
[...]
Fetched 31.0 MB in 26s (1189 kB/s)
[...]

This gave us an estimate of the current mirror speed, which is 1189 kB/s.

3. Find the Fastest Mirror

Here, we’ll look at several methods for selecting the fastest mirror.

3.1. netselect

netselect identifies the most efficient and responsive servers from a list using parallel tests. It analyzes each server for several parameters:

  • Approximate ping time
  • Number of network hops, which is the number of intermediate devices that data must pass through to get from the source to the destination
  • Success rate of ping requests

It then calculates a score for each server, where a lower score indicates better performance.

Its package is available in the Debian repository, has no dependencies, and is compatible with Ubuntu. Let’s download it with wget and install it by building its URL according to our architecture:

$ ARCH=$(dpkg --print-architecture) # system architecture (e.g., amd64)
$ VERSION='0.3.ds1-30.1'
$ URL="http://ftp.debian.org/debian/pool/main/n/netselect/netselect_${VERSION}_${ARCH}.deb"
$ wget $URL
[...]
2023-12-06 00:05:53 (1,16 MB/s) - ‘netselect_0.3.ds1-30.1_amd64.deb’ saved [23208/23208]

$ sudo apt install ./netselect_${VERSION}_${ARCH}.deb
[...]
The following NEW packages will be installed:
  netselect
[...]

Then, let’s fetch the Launchpad HTML list of Ubuntu mirrors and save it to mirrors.txt:

$ wget -q -O- https://launchpad.net/ubuntu/+archivemirrors > mirrors.txt

Now, let’s use mirrors.txt to extract the URLs of the mirrors that are up to date. Of course, if the Launchpad HTML page changes format in the future, we’ll need to update this command accordingly:

$ grep -P -B8 "statusUP" mirrors.txt | grep -o -P "(f|ht)tp://[^\"]*" > filtered_mirrors.txt

Finally, let’s use netselect to find the fastest mirrors from filtered_mirrors.txt. The -s10 option selects the top 10 fastest servers, and -t20 sets 20 pings for each server:

$ sudo netselect -s10 -t20 $(cat filtered_mirrors.txt)
   68 http://mirrors.aliyun.com/ubuntu/
   68 http://mirrors.aliyun.com/ubuntu/
   74 http://mirrors.layeronline.com/ubuntu/
   74 http://mirrors.aliyun.com/ubuntu/
   74 http://mirrors.aliyun.com/ubuntu/
   75 http://mirrors.aliyun.com/ubuntu/
   76 http://mirrors.aliyun.com/ubuntu/
   77 http://mirrors.layeronline.com/ubuntu/
   77 http://mirrors.aliyun.com/ubuntu/
   77 http://mirrors.nipa.cloud/ubuntu/

In this case, the fastest is http://mirrors.aliyun.com/ubuntu/. The repetition of URLs in these results could be because netselect tests each IP address independently when a DNS name resolves to multiple IP addresses. For example, it’s easy to verify via host mirrors.aliyun.com that this mirror resolves to up to 12 different IPs.

Let’s use sed to replace our current mirror with http://mirrors.aliyun.com/ubuntu/ and test it:

# Replace the mirror in sources.list
newMirror='http://mirrors.aliyun.com/ubuntu/'
sudo sed -i "s|deb [a-z]*://[^ ]* |deb ${newMirror} |g" /etc/apt/sources.list

# Test the mirror speed
sudo rm -rf /var/lib/apt/lists/* && sudo apt update
[...]
Fetched 31.0 MB in 1min 4s (487 kB/s)
[...]

Unfortunately, the result is worse than that of the initial mirror, whose speed was 1189 kB/s. This leads us to conclude that netselect may not be reliable for our purposes.

3.2. apt mirror://

The apt mirror:// method involves using a special URL in /etc/apt/sources.list that instructs apt to fetch mirrors located within the user’s country.

This method uses the http://mirrors.ubuntu.com/mirrors.txt file, which contains a list of geographic mirrors based on the client’s source IP:

$ wget -q -O- http://mirrors.ubuntu.com/mirrors.txt
http://mirror1.totbb.net/ubuntu/
http://mirror1.ku.ac.th/ubuntu/
https://mirrors.nipa.cloud/ubuntu/
http://mirrors.psu.ac.th/ubuntu/
http://mirror.thaidns.co.th/ubuntu/
https://mirror.kku.ac.th/ubuntu/
http://archive.ubuntu.com/ubuntu/

All we need to do is replace the mirror URLs in sources.list with mirror://mirrors.ubuntu.com/mirrors.txt:

# Replace the mirror in sources.list
newMirror='mirror://mirrors.ubuntu.com/mirrors.txt'
sudo sed -i "s|deb [a-z]*://[^ ]* |deb ${newMirror} |g" /etc/apt/sources.list

# Test the mirror speed
sudo rm -rf /var/lib/apt/lists/* && sudo apt update
[...]
Fetched 31.0 MB in 31min 58s (16.2 kB/s)
[...]

This result is much worse than the initial mirror, which had a speed of 1189 kB/s. The execution of apt update truly took 31min 58s, which is totally unacceptable.

So, while this apt mirror:// method sounds interesting, we cannot trust it. The underlying problem is that the auto-generated list of geographic mirrors in mirrors.txt doesn’t take into account speed or availability, which can lead to poor results, as in this case. By the way, our nation’s mirrors aren’t necessarily the fastest.

3.3. curl Transfer Speed

Previously, in the netselect section, we saw how to retrieve the list of Ubuntu mirrors and save it in filtered_mirrors.txt. Since the previous methods weren’t satisfactory, we can try another approach, which is to create a Bash script that measures the speed of each mirror in filtered_mirrors.txt and then lists the fastest ones.

Let’s save the following code as speed.sh and give it execute permissions. It requires wget, grep, curl, bc, sort, and head, which are typically preinstalled in Ubuntu. Instead, both mapfile and declare are built-in Bash commands:

#!/bin/bash

# Fetch the HTML list of Ubuntu mirrors and extract URLs of up-to-date mirrors
wget -q -O- https://launchpad.net/ubuntu/+archivemirrors > mirrors.txt
grep -P -B8 "statusUP" mirrors.txt | grep -o -P "(f|ht)tp://[^\"]*" > filtered_mirrors.txt

# Read the list of mirrors
mapfile -t mirrors < filtered_mirrors.txt
total_mirrors=${#mirrors[@]}

# Array to hold speeds
declare -A speeds

echo "Testing mirrors for speed..."

# Test each mirror with a 2-second timeout
for i in "${!mirrors[@]}"; do
    # Calculate the sequence number
    seq_num=$((i+1))
    
    # Get the speed in bytes per second and convert to kilobytes per second
    speed_bps=$(curl --max-time 2 -r 0-102400 -s -w %{speed_download} -o /dev/null "${mirrors[$i]}/ls-lR.gz")
    speed_kbps=$(echo "$speed_bps / 1024" | bc)
    
    # Save the speed with the mirror URL
    speeds["${mirrors[$i]}"]=$speed_kbps
    
    # Print the mirror and speed
    echo "[$seq_num/$total_mirrors] ${mirrors[$i]} --> $speed_kbps KB/s"
done

# Sort mirrors by speed and get the top 5
echo "Top 5 fastest mirrors:"
for mirror in "${!speeds[@]}"; do
    echo "$mirror ${speeds[$mirror]}"
done | sort -rn -k2 | head -5

Without going into implementation details, let’s look at the logic of this script:

  • Reads URLs from filtered_mirrors.txt into the array mirrors and iterates over each mirror URL
  • Uses curl with a timeout of 2 seconds to download the first 102,400 bytes of the ls-lR.gz file, which is a compressed file containing a recursively generated list of all files and directories available on the mirror
  • Measures the download speed in bytes per second and converts it to kilobytes per second
  • Stores the speed in the speeds array with the mirror URL as the key
  • Prints the mirror URL and its download speed, preceded by the sequence number
  • At the end of testing all mirrors, sorts the speeds array by download speed in descending order
  • Extracts and prints the URLs and speeds of the top five fastest mirrors

Let’s execute it:

$ ./speed.sh 
Testing mirrors for speed...
[1/706] http://mirrors.dc.clear.net.ar/ubuntu/ --> 37 KB/s
[2/706] http://ubuntu.zero.com.ar/ubuntu/ --> 37 KB/s
[3/706] http://ubuntu.unc.edu.ar/ubuntu/ --> 47 KB/s
[...]
[707/709] http://mirror.clearsky.vn/ubuntu/ --> 146 KB/s
[708/709] http://mirrors.nhanhoa.com/ubuntu/ --> 206 KB/s
[709/709] http://opensource.xtdv.net/ubuntu/ --> 88 KB/s
Top 5 fastest mirrors:
http://mirrors.nipa.cloud/ubuntu/ 597
http://mirror1.totbb.net/ubuntu/ 581
http://mirror1.ku.ac.th/ubuntu/ 506
http://linux.domainesia.com/ubuntu/ubuntu-archive/ 302
http://repo.huaweicloud.com/ubuntu/ 292

Out of curiosity, let’s check Launchpad to see who these top five mirrors correspond to:

  1. NIPA.CLOUD (Thailand)
  2. TOT Internet Data Center (Thailand)
  3. Office of Computer Services, Kasetsart University (Thailand)
  4. DomaiNesia (Indonesia)
  5. Huawei Cloud (China)

These results are consistent with the fact that we ran the test from a Thai IP, so our script seems reliable. The low speeds are affected by the low timeout and the small amount of data downloaded.

Let’s try the fastest mirror:

# Replace the mirror in sources.list
newMirror='http://mirrors.nipa.cloud/ubuntu/'
sudo sed -i "s|deb [a-z]*://[^ ]* |deb ${newMirror} |g" /etc/apt/sources.list

# Test the mirror speed
sudo rm -rf /var/lib/apt/lists/* && sudo apt update
[...]
Fetched 31.0 MB in 17s (1847 kB/s)
[...]

This time, we got a better result than the 1189 kB/s of the initial mirror.

4. Conclusion

The speed of apt mirrors can vary due to factors such as our machine’s network connection, the current load on each mirror server, and the physical distance between our machine and the servers. By choosing an optimal mirror, we can ensure efficient access to software packages and updates.

In this article, we’ve seen several command line methods for selecting the fastest apt mirror for Ubuntu:

  • netselect → uses several parameters to select mirrors but doesn’t measure bandwidth
  • apt mirror:// → selects geographical mirrors based on the client’s source IP
  • curl transfer speed → Bash script to measure the bandwidth of each mirror

The last method using the Bash script seems to be the only one we can trust.

« 上一篇： How to Quickly Trim Audio and Video in Linux | Baeldung on Linux
» 下一篇： How to Display a Spinner for Long Running Tasks in Bash | Baeldung on Linux