1. Overview
When we monitor our Linux system’s performance, we may notice that the CPU usage reported by the top command sometimes exceeds 100%. Essentially, this phenomenon can be puzzling, especially for those new to Linux.
In this tutorial, we’ll delve into why CPU usage can go over 100%, provide detailed examples using bash scripts, and explain how to interpret the output of these commands. Moreover, we’ll also explore how to modify the visual representation in the top command to better understand CPU usage.
2. Understanding Multi-Core Processors
Single-core CPUs containing a single processing unit were the foundation of early computer systems. Furthermore, they handle one operation at a time, making them simpler but less efficient for multitasking compared to modern multi-core processors.
Despite their limitations, they played an important role in the evolution of computing technology and are still used in specific applications like embedded systems where simplicity and power efficiency are prioritized. Their legacy endures as the basis upon which contemporary multi-core architectures were built.
Modern CPUs have multiple cores, allowing them to perform various tasks simultaneously. Accordingly, each core can handle its stream of instructions, which significantly enhances the overall processing power. For example, a quad-core processor has four separate cores, each capable of executing its tasks independently.
To see the number of CPU cores on our system, we can use the nproc command:
# nproc --all
4
This output indicates that the system has four CPU cores. Precisely, the top command is a powerful utility for monitoring system performance, including CPU usage. By default, top shows the percentage of CPU usage for each process. However, when dealing with multi-core processors, top can report CPU usage that exceeds 100%.
When top reports CPU usage, it does so on a per-core basis. For instance, if a process uses 100% of one core on a quad-core system, top will report it as 100%. Accordingly, if a process uses 100% of all four cores, top will report it as 400%.
3. Practical Examples With Bash Scripts
In this section, we’ll create a bash script to stress sour CPU workloads and use it to test with the top command.
Let’s explore how to monitor CPU usage with some practical examples. First, we’ll create a CPU-intensive process and observe its behavior using the top command:
#!/bin/bash
# cpu_stress.sh
stress --cpu 4 --timeout 60
In theory, this script creates a CPU-intensive process. However, to utilize this script, we’ll need to have the stress utility installed.
Next, let’s install the stress utility:
$ sudo apt-get install stress
In the above snippet, we used the sudo command to grant us superuser privileges to be able to install the required package. Furthermore, we used the apt-get command to install the package on our Linux environment. Since we installed the stress utility, we are now eligible to run the script above:
$ bash cpu_stress.sh
Ideally, this script will create a CPU-intensive load on all four cores for 60 seconds. Now, we can observe our CPU utilization using the top command:
$ top
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1234 user 20 0 4100 1244 932 R 400.0 0.0 0:10.45 stress
Finally, to interpret the output of top, we need to understand the various columns it displays:
- PID: process ID
- USER: user who owns the process
- PR: priority of the process
- NI: nice value of the process
- VIRT: virtual memory used by the process
- RES: resident memory used by the process
- SHR: shared memory used by the process
- S: process status (e.g., R for running, S for sleeping)
- %CPU: percentage of CPU used by the process
- %MEM: percentage of memory used by the process
- TIME+: total CPU time used by the process
- COMMAND: name of the command
In this example, the stress command is using 400% CPU, indicating it’s utilizing all four cores fully.
4. Adjusting Visualizations in top
The top command offers several ways to adjust how CPU usage is visualized, making it easier to understand the data. Through different visualizations, we can customize the output of the command to fit our needs and have enhanced visibility on our processes utilizations.
4.1. Changing Display Mode
We can press I (uppercase i) to toggle between different modes of displaying CPU usage. By default, top shows the cumulative usage of all cores. Ideally, pressing I will switch to displaying the usage of individual cores:
%Cpu0 : 100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu1 : 100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu2 : 100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu3 : 100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
In the above output, we changed the visualization to an individual core view to see the usage of each core.
4.2. Changing Update Interval
We can also change the update interval for top by pressing d and then entering the desired interval in seconds. This can help us get a more real-time or more average view of CPU usage.
Let’s press d while top is running and enter 1 for a one-second update interval:
top - 15:25:36 up 1:23, 1 user, load average: 0.31, 0.47, 0.58
Tasks: 123 total, 2 running, 121 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4.0 us, 2.0 sy, 0.0 ni, 94.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 2048.0 total, 1024.0 free, 512.0 used, 512.0 buff/cache
MiB Swap: 2048.0 total, 2048.0 free, 0.0 used. 1536.0 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1234 user 20 0 4100 1244 932 R 100.0 0.0 0:00.45 stress
We notice that the update interval is set to one second, providing a more frequent refresh rate for monitoring CPU usage.
4.3. Limiting Displayed Processes
Finally, we can press k to kill a process directly from the top interface. Specifically, this can be useful if a specific process is causing high CPU usage.
To illustrate more, we press k while the top command is running and enter the PID of the process we want to kill:
PID to kill: 1234
After entering the PID, the specified process will be terminated and CPU utilization will decrease. However, notably, seeing CPU usage over 100% in top is normal for systems with multiple CPU cores. It indicates that the process is utilizing more than one core.
For example, on a quad-core system, a single process could use up to 400% CPU if it fully utilizes all four cores as we have seen earlier. Subsequently, this does not imply any issue with the system; rather, it reflects the multi-core nature of modern CPUs.
Let’s consider a scenario where a process utilizes 150% CPU on a dual-core system:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12345 user 20 0 4100 1244 932 R 150.0 0.0 0:10.45 example_process
This means the process is using 100% of one core and 50% of the second core. Accordingly, high CPU usage isn’t inherently problematic, but it can impact system performance if critical processes are starved of CPU resources. Hence, monitoring tools like top help us identify such scenarios and take corrective actions, such as killing unresponsive processes or optimizing resource-intensive applications.
5. Additional top Command Usage and Options
The top command provides a plethora of options and interactive commands to customize its display and functionality, such as sorting, filtering, and renicing.
Let’s understand the most commonly used sorting methods using top:
- shift + P: sort processes by CPU usage
- shift + M: sort processes by memory usage
- shift + T: sort processes by running time
Moreover, if we press on shift + H, this will toggle the display of threads. Essentially, this is useful for seeing the CPU usage of individual threads within a process:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1234 user 20 0 4100 1244 932 R 100.0 0.0 0:10.45 example_process
1235 user 20 0 4100 1244 932 R 50.0 0.0 0:05.23 example_thread
1236 user 20 0 4100 1244 932 R 50.0 0.0 0:05.22 example_thread
Now, top shows the individual threads within the example_process.
Finally, we can customize which columns are displayed by top by pressing f and selecting the columns we want to include:
Current Fields: K for SortKey U for User/Mem k for VIRT y for GID
Toggle Fields via: A for display D for default 1 for load1 F for free
PID to add: [Add field here]
Then, we select the desired columns by toggling them on or off.
6. Conclusion
In this tutorial, we learned why the CPU usage reported by top can exceed 100%. And why it’s important for effective system monitoring and performance tuning.
By recognizing that top reports usage per core and knowing how to adjust visualizations, we can gain deeper insights into our system’s performance. Moreover, practical examples with bash scripts help illustrate these concepts, making it easier for us to monitor and manage our Linux system effectively.
Finally, observing CPU usage over 100% is normal behavior in multi-core systems. It simply reflects that processes are utilizing multiple cores. This isn’t indicative of a problem but rather shows the efficiency of modern multi-core processors.