1. Overview
In our daily routine, we use many computer programs to perform our tasks. A program is an executable file that may depend on one or more software libraries. A software library is a set of compiled source code files packaged in a single archive. If these libraries aren’t available during execution time, then our program will fail with an error message. Our Linux machines attempt to locate the necessary libraries whenever we start a program.
In this article, we will focus on how our Linux machine loads libraries and how we can locate them.
2. Linux Libraries Types
There are three types of libraries: static, shared, and dynamically loaded. Nowadays, shared and dynamically loaded libraries are the most popular among Linux distributions. Static libraries historically existed first, but we are not using them so often anymore. This is mostly because they lack the features of the other two types.
As developers, we compile our source code with the shared libraries that the code depends on to produce an executable file. When we start a program, the Linux program loader checks if the program depends on shared libraries and tries to find them. If the program loader fails to find one or more libraries, then it stops and prints a relevant error message. In addition, there is a standard naming scheme for shared libraries. Versioning information is contained in this naming scheme. This is useful if we want to handle different library versions.
On the other hand, the system loads a dynamically loaded library on-demand after the program has started. There is an application programming interface that the source code of the program uses to load the library during the execution of the program.
3. The ldconfig Command
The ldconfig command configures the system so that programs can use existing shared libraries. To find out if a shared library is installed, we may use the command with the -p option. In this case, it outputs a list of the cached shared libraries:
$ ldconfig -p | grep libpthread.so.0
libpthread.so.0 (libc6,x86-64, OS ABI: Linux 3.2.0) => /lib/x86_64-linux-gnu/libpthread.so.0
Here we filter the output of the ldconfig -p command with grep to check if the libpthread.so.0 library is installed. Note that we may achieve similar results with the -v option, where the libraries are grouped by the folder they are in.
4. Standard Library Paths
The Filesystem Hierarchy Standard describes the filesystem conventions of a Linux system. In this standard, folders /lib, /usr/lib and /usr/local/lib are the default folders to store shared libraries. The /lib folder has libraries used during the boot time of the system but also used by programs in the /bin folder. Similarly, the*/usr/lib* folder has libraries used by programs in the /usr/bin folder. Finally, /usr/local/lib folder has libraries used by programs in /usr/local/bin folder. Therefore, we may first check these standard paths to find out if a specific shared library is installed on our system.
Note that we should use the local sub-folder for the local installation of software programs. System updates do not affect this folder.
5. How Shared Libraries Are Located
The program loader invokes the dynamic linker during the execution time of a program. The dynamic linker is itself a shared library with a name like ld-*.so. The linker will locate the shared libraries that a program is using.
5.1. The /etc/ld.so.conf Configuration File
The linker will search for shared libraries in the paths listed in the configuration file /etc/ld.so.conf. In an Ubuntu system, this file contains a single line:
$ cat /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf
This line points to the /etc/ld.so.conf.d folder, for configuration files:
$ ls /etc/ld.so.conf.d
fakeroot-x86_64-linux-gnu.conf libc.conf x86_64-linux-gnu.conf
This folder contains three configuration files. Note that there may be more configuration files than the ones listed above. We may have installed an application that added a configuration file with extra paths. Let’s check the contents of the libc.conf file:
$ cat /etc/ld.so.conf.d/libc.conf
# libc default configuration
/usr/local/lib
We observe that the file contains the /usr/local/lib path.
5.2. The /etc/ld.so.cache Configuration File
Searching in the paths defined in the */etc/*ld.so.conf may delay program execution. Therefore, Linux systems use the /etc/ld.so.cache configuration file, which caches a list of all shared libraries and their location in the system. The ldconfig command builds this cache. When we copy a shared library to one of the standard folders, we should execute the ldconfig command to update the cache. Do note that even if a library file exists in one of the paths of the ld.so.conf configuration file, it still isn’t used if the cache hasn’t recorded it. If this is the case, then we should execute the ldconfig command to fix the cache.
5.3. Symbolic Links and Shared Libraries
Another important note is about shared library names and how they are linked. A shared library has three names in total: the real name, the so-name, and the linker name. Most often, the so-name is a symbolic link to the real name. Similarly, the linker name is usually a symbolic link to the so-name. If one of these symbolic links breaks, then we may not be able to use the library. The ldconfig command creates the symbolic link for the so-name for us.
6. Finding the Dependencies of a Program
The ldd command prints the shared libraries that a program depends on:
$ ldd /usr/bin/ls
linux-vdso.so.1 (0x00007ffdddbf4000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f792668c000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f792649a000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007f792640a000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f7926404000)
/lib64/ld-linux-x86-64.so.2 (0x00007f79266e5000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f79263e1000)
In the above example, the ldd command printed the shared libraries that the ls command uses. Do note that the command prints the exact path location of each library. If a library is missing, then the output is different:
$ ldd printhelloclient
linux-vdso.so.1 (0x00007fff7149f000)
libprinthello.so.1 => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8465e75000)
/lib64/ld-linux-x86-64.so.2 (0x00007f8466075000)
Here we print the dependencies of the printhelloclient executable file. We see that this program depends on the missing shared library libprinthello.so.1.
7. Non-standard Paths With the LD_LIBRARY_PATH Environment Variable
We can use the LD_LIBRARY_PATH environment variable to add non-standard library paths. This environment variable contains a colon-separated list of folders. The linker will first search the paths defined in this variable. Then it will search in the paths of the configuration file /etc/ld.so.conf. Therefore, we may append library paths to this environment variable:
$ export LD_LIBRARY_PATH=/home/user/sharedlib:$LD_LIBRARY_PATH
Here the linker will also search the /home/user/sharedlib folder for shared libraries. Note that we should use this variable for development and testing purposes.
8. Conclusion
In this article, we focused on the ways a Linux system finds shared libraries. Important configuration files are the /etc/ld.so.conf and /etc/ld.so.cache. In terms of commands, we may use the ldd command to find out missing dependencies. Also, we can use the ldconfig command with the -p option to check if a shared library is installed. Finally, we should check the standard library paths like /usr/lib and /usr/local/lib as well as extra paths listed in the /etc/ld.so.conf configuration file.