1. Overview
Linux uses an overcommit policy to allocate memory. Briefly, we can ask for more memory than is physically available. This way stems from the observation that processes don’t immediately consume all allocated memory, if they do that at all. So, whether we’re users, administrators, or developers, we should be aware of the implications of this approach. It’s especially important, as the kernel can resort to killing applications in the face of memory shortage.
In this tutorial, we’ll learn how to set the suitable overcommit mode. Then, we’ll check how our choice changes the system’s behavior.
2. The overcommit_memory Parameter
We have three overcommit modes at our disposal. Consequently, we can select one of them with the overcommit_memory parameter:
- 0 – overcommit, but refuse any unreasonable memory request on the heuristics base. This is the default mode
- 1 – always overcommit
- 2 – never overcommit
Further, let’s check the parameter’s value stored in the file:
$ cat /proc/sys/vm/overcommit_memory
0
Next, let’s change its value to, e.g., 1:
$ echo 1 | sudo tee /proc/sys/vm/overcommit_memory
As this setting affects only the current session, we need to add the vm.overcommit_memory=x line in the sysctl.conf file for the permanent result:
$ sudo joe /etc/sysctl.conf
Then let’s reload without reboot:
$ sudo sysctl -p
3. Only Reasonable Requests – overcommit_memory = 0
Now let’s test how the kernel works in the default mode. So, we’re going to start one stress-ng thread to consume 32 GB of memory for 120 s:
$ stress-ng -vm 1 --vm-bytes 32g -t 120
stress-ng: debug: [5668] stress-ng 0.13.12
stress-ng: debug: [5668] system: Linux ubuntu 5.15.0-52-generic #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022 x86_64
stress-ng: debug: [5668] RAM total: 15.5G, RAM free: 9.6G, swap free: 14.9G
# ...
stress-ng: error: [5670] stress-ng-vm: gave up trying to mmap, no available memory
# ...
So, we’ve got a refusal. Thus, let’s reduce the target consumption to 16 GB:
$ stress-ng -vm 1 --vm-bytes 16g -t 120
Although the demand was accepted and everything looked good for a few moments, we noticed the sudden disappearance of windows in our GUI. It happened to the task’s terminal or the web browser’s window as well. So, we experienced the OOM killer activity, triggered by the kernel facing the exhausting memory. Let’s check the fate of the web browser querying journalctl:
Oct 31 17:33:37 ubuntu systemd-oomd[644]:
Killed /user.slice/user-1000.slice/[email protected]/app.slice/snap.firefox.firefox.1809b6a1-ddf8-4a03-aaef-2da35d189c89.scope
due to memory pressure for /user.slice/user-1000.slice/[email protected] being 80.67% > 50.00% for > 20s with reclaim activity
4. Ask for as Much as You Want – overcommit_memory = 1
Now let’s set overcommit_memory to 1. Subsequently, let’s dispatch the 64 GB stressor:
$ stress-ng -vm 1 --vm-bytes 64g -t 120
As expected, the kernel accepted the 64 GB request – twice the RAM and swap size taken together. However, very quickly, the OOM killer eliminated the stress-ng task together with all opened windows. Surely, we should save our work before such an experiment!
Let’s notice that this way is suitable when a lot of zero-filled memory is expected, as the kernel maps related memory pages to one physical zero page.
5. How Much, Come On! – overcommit_memory = 2
Finally, let’s turn on the no-overcommit mode with overcommit_memory = 2. Then, let’s repeat our test with the 16 GB stressor:
$ stress-ng -vm 1 --vm-bytes 16g -t 120
# ...
stress-ng: error: [10792] stress-ng-vm: gave up trying to mmap, no available memory
# ...
So, our request was denied, as opposed to the overcommit_memory = 0 case. However, the application needs to handle such a situation internally.
6. Metrics and Parameters
Let’s notice that the kernel controls the memory usage by means of two metrics, CommitLimit and Committed_AS:
$ cat /proc/meminfo
# ...
CommitLimit: 23759956 kB
Committed_AS: 7770388 kB
# ...
CommitLimit provides the size of available physical memory, including RAM and swap space. Next, in the no-overcommit mode, Committed_AS shows memory allocated and really used by all processes. Then, we can’t ask for more memory than CommitLimit. On the other hand, we should regard Committed_AS as the worst-case memory usage estimate when overcommit is enabled.
Now, let’s check two parameters that determine CommitLimit. So, overcommit_ratio is a percent of the available RAM while overcommit_kbytes expresses the same in kilobytes. Moreover, only one of them is taken into account, with overcommit_kbytes having precedence. Finally, we can set them in files of corresponding names in the /proc/sys/vm folder or in sysctl.conf.
7. Conclusion
In this tutorial, we learned about the memory overcommit concept. So, we used the overcommit_memory parameter to select the overcommit mode. Subsequently, we studied how the system worked under memory pressure in each mode. Finally, we examined parameters that determined the amount of available virtual memory.