1. Overview
In this tutorial, we’ll learn how to replicate packages across similar Linux distributions. First, we’ll discuss how automating this process for different distros as a whole is difficult to achieve.
Next, we’ll declutter the system packages for a clean replication process. Afterward, we’ll discuss the replication process for each major distribution.
2. The Problem
Automating the process of replicating packages across multiple distributions is a difficult feat to achieve for several reasons:
- Different package managers
- Different software sets
- Different package names
However, we can easily replicate packages for similar Linux distributions. For instance, on Debian and derivatives, we can simply use apt to replicate packages. Similarly, we can replicate packages in OpenSUSE for other OpenSUSE machines using rpm.
Similarly, the method varies based on the package manager in use. Therefore, we need a different process for different distributions.
3. First Steps
Before we dive into the process, we first need to make sure that our system has no clutter, which means obsolete, orphaned, and unused packages. Therefore, we need to clean up the system.
For that purpose, we use the package managers that come with the distributions.
On Debian and derivatives, we use apt:
$ sudo apt autoremove --purge
On Fedora and derivatives, we’d use dnf:
$ sudo dnf autoremove
For openSUSE, we can use a script:
#!/bin/bash
zypper packages --unneeded | awk -F'|' 'NR==0 || NR==1 || NR==2 || NR==3 || NR==4 {next} {print $3}' > list
while read pkg; do
sudo zypper -n rm -y --clean-deps "$pkg";
done < list
rm -rf list
Next, we make the script executable using chmod and run it:
$ chmod +x ./declutter-pkgs.sh && ./declutter-pkgs.sh
On Arch and derivatives, we use pacman:
$ sudo pacman -Rns $(pacman -Qdtq)
4. Debian and Ubuntu
Debian, Ubuntu, and derivatives come with the apt and dpkg package managers. apt is a high-level front-end to dpkg that simplifies the many aspects of package management.
4.1. Manually Replicating Packages
We can replicate the packages on other Debian or Ubuntu systems just by using apt.
First, we list all the installed packages with apt-mark, which is a tool for marking packages to help manage package dependencies:
$ sudo apt-mark showmanual
apt-clone
bsdutils
cloud-init
curl
dash
diffutils
efibootmgr
ffmpeg
findutils
...
The showmanual option shows all the packages that were installed explicitly. It means that it won’t output any dependencies that were installed with the packages. We write this list into a file:
$ apt-mark showmanual > $BACKUP/packages.txt
Next, we need to copy the sources as well to include packages that were installed from 3rd-party repositories:
$ sudo cp -r /etc/apt/sources.list /etc/apt/sources.list.d/ $BACKUP/sources/
Lastly, we copy the GPG keys as well:
$ sudo cp -r /etc/apt/trusted.gpg/ $BACKUP/keys
In the commands, we specified the generic $BACKUP variable. It could be a shared directory, a temporary location, or an external storage device. We must make sure to set this to the correct location.
Once it’s all done, we can appropriately copy the sources and keys to a new machine:
$ sudo cp -r $BACKUP/sources/* /etc/apt/
$ sudo cp -r $BACKUP/keys/* /etc/apt/trusted.gpg.d/
Now, let’s update the system apt package cache:
$ sudo apt update
Finally, we go through the packages.txt file and install each package:
$ while read -r p; do sudo apt install -y "$p"; done < packages.txt
That’s it! In the next section, we automate this process.
4.2. apt-clone
Earlier, we learned how to manually transfer packages to another machine. However, to save us some time, there’s a convenient package apt-clone that can automate this process for us.
So, let’s install apt-clone first:
$ sudo apt install -y apt-clone
Next, we create a clone:
$ apt-clone clone ubuntu-baeldung
It generates a ubuntu.baeldung-apt-clone.tar.gz file, which we copy to a new machine. In addition, we also need to install apt-clone on the new machine to restore the backup:
$ sudo-apt restore ubuntu-baeldung.apt-clone.tar.gz
Once we run the command, it updates the cache and installs the packages.
5. Fedora, RHEL, and CentOS
On Fedora and derivatives, we can use the dnf package manager. We can list all the explicitly installed packages and put them in a file:
$ dnf repoquery --qf '%{name}' --userinstalled \
| grep -v -- '-debuginfo$' \
| grep -v '^\(kernel-modules\|kernel\|kernel-core\|kernel-devel\)$' > packages.txt
Let’s break this down:
- repoquery prints information about packages
- –qf ‘${name}’ sets the display format to show only package names
- –userinstalled flag enables dnf to print explicitly installed packages
- the subsequent grep commands exclude packages with the given keywords
- > packages.txt writes the filtered list to packages.txt
Afterward, we copy the list to a new Fedora machine and install the packages from packages.txt:
$ < pkgs_a.lst xargs dnf -y install
6. OpenSUSE
OpenSUSE uses rpm, which is a package management system. We can use this to generate a list of installed packages on OpenSUSE:
$ rpm -qa > packages.txt
Now, we use zypper to install these packages on another OpenSUSE machine. zypper is the default package manager for OpenSUSE. We cat the contents of packages.txt and feed it to zypper:
$ sudo zypper install -y $(cat packages.txt)
The -y option disables prompts.
7. Arch Linux and Manjaro
Manjaro is a derivative of Arch Linux. They both use the pacman package manager, which is one of the fastest and most well-designed package managers out there in the Linux world. On both of these distros, we can generate a nice list of explicitly installed packages:
$ pacman -Qqen > repo_packages.txt
Apart from that, we can also create a list of AUR packages:
$ pacman -Qqem > aur_packages.txt
Once we have these files, we can install them on a new Arch Linux or Manjaro machine. For that purpose, we should update the package index cache first:
$ sudo pacman -Syyu
Then, we install the package in repo_packages.txt:
$ sudo pacman -S --noconfirm --needed < repo_packages.txt
Similarly, for the AUR packages, we need to use a tool like yaourt, yay, or paru:
$ paru -S --needed --noconfirm < aur_packages.txt
paru is a powerful package manager with options almost the same as pacman.
7.1. Putting It Together
Not only that, but it can also install packages from the official package repositories. So, with this knowledge, we can reduce the above steps. We create the list of the official repository and AUR package and put them in a single file:
$ pacman -Qqen | tee packages.txt | pacman -Qqem
Then, we can feed the file to paru:
$ sudo pacman -Syyu && paru -S --needed --noconfirm < packages.txt
7.2. Extras
Sometimes, there are libraries and binaries that aren’t known to pacman. These files are probably installed through other methods like Steam, and Desura, and through their own install methods.
For that reason, we can use a script:
$!/bin/bash
find / -regextype posix-extended -regex "/(sys|srv|proc)|.*/\.ccache/.*" -prune -o -type f \
-exec bash -c 'file "{}" | grep -E "(32|64)-bit"' \; | \
awk -F: '{print $1}' | \
while read -r bin; \
do pacman -Qo "$bin" &>/dev/null || echo "$bin"; \
done
The script finds specific files that aren’t associated with packages installed through pacman. In that case, the paths are printed. Therefore, we can copy these files and directories to a newer machine as well:
$ ./print-other-packages.sh > other_packages.txt
Next, we use tools like rsync to copy these packages to other Linux machines:
$ rsync -av --files-from=other_packages.txt $BACKUP/other
These files will be copied to $BACKUP/other, which we can then transfer to a new Arch Linux system.
8. Conclusion
In this article, we explored how to replicate packages across Linux distributions. First, we looked at the problem in detail and then discussed the replication process for each major Linux distribution.