What is an OOM kill process

The OOM kill is a process used by the Linux kernel when the system is critically low on RAM memory. This happens because the Linux Kernel has over allocated memory to its processes. Normally, this situation does not cause a problem. However, if enough processes begin to use all of their requested memory blocks then there will not be enough physical memory to support them all.

The solution that the Linux Kernel employs is to invoke the OOM Killer to review all running processes and kill one or more of them in order to free up system memory and keep the system running.

How does it choose which processes to kill?

The OOM Killer works by reviewing all running processes and assigning them a badness score. Whichever process has the highest score is killed.

So Why Is Apache / MySQL Always Killed?

An application such as apache, mysql, ftp server or a mail server will most probably score high on the list. However, as this situation usually occurs on a busy web server Apache or MySQL will be the largest in-memory, non-system processes and consequently get killed. It must be remembered that although the webserver or db server are very important to you when the kernel calls the OOM Killer the situation is critical.

If memory is not freed by killing a process the server will crash very shortly afterwards. Continuing normal operations at this juncture is impossible.

How To Find Out If The OOM Killer Was Running

The easiest way to find if the OOM Killer was the reason that a website went offline is to check the system log. Whenever the OOM Killer is invoked it will write a lot of information to the system log including which process was killed and why. If the following command is run:

dmesg | egrep -i “killed process”

The output will look like the following:

host kernel: Out of Memory: Killed process 2592 (mysql).

In this instance the process that was killed was mysql which has the PID (Process Identification Number) of 2592.

How To Avoid The OOM Killer

The OOM Killer will only get invoked when the system is critically low on memory. Hence, the solution to avoiding it is to either reduce the memory requirements of the server or increase the available memory.

Reduce Memory Usage

The first step that should be taken to reduce memory usage to stop any processes running that are not needed. For example if the server is not shared and FTP is only occasionally used then this process can be initiated prior to uploading and terminated afterwards. Furthermore, you can review the active processes and stop them if they’re not being used.

Increase available memory

Choosing alternative applications or tweaking configuration will only produce limited results. Increasing the available RAM to an amount sufficient to support the needed processes is always the best solution.