Apache memory use

From Helpful
Jump to: navigation, search
Related to web development, hosting, and such: (See also the webdev category)
jQuery: Introduction, some basics, examples · plugin notes · unsorted

Server stuff:

Dynamic server stuff:

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

You may want to know how much memory apache can take - and restrict the maximum.

Per-process use lies mostly in:

  • loading each configured apache module (order of a few MB)
  • stack space given by the OS (a few MB)
  • mapping/allocate further memory itself
most relevant when embedding language runtimes (could allocate GBs at a whim)
most modules are designed to keep within limits

...so under low load, and assuming you don't have fancy scripty runtimes, each apache child

is easily 10MB
may be ~30MB if loading a lot of muck
can often be slimmed to ~5MB if you have a single-purpose server

Total peak memory use is bound by average(/max) per-process memory use, times the maximum allowed processes - which under most MPMs is controlled by ServerLimit.

When tweaking any these settings, you may want to read up on the behaviour of the MPM you're using. Specific MPMs may have other settings that effectively also have a limit. For example, in prefork MaxClients is effectively equivalent to ServerLimit.

Multi-process MPMs deal with load by creating more processes.

Multi-threaded (and hybrid) MPMs are more memory-efficient than non-threaded MPMs (such as prefork) because you get a bulk of handlers/workers/whateeryouwannacall'ems per process, so need fewer processes to serve just as many requests. They're also potentially more trouble in that everything loaded into apache itself needs to be thread-safe. (which e.g. PHP isn't. Note: For modules that restrict themselves to prefork (such as PHP), large-scale setups may well want to consider different setups. In the case of PHP, consider using FCGI-style PHP, or just CGI-style PHP)

With any MPMs that have multiple processes, you want to set a maximum amount of processes appropriate for your installation.

Setting it too high will just lead to swapping. If set much too high, it can also lead apache to run the machine out of memory, to the point where processes will be killed by oom_kill, or when that is disabled (or doesn't apply), lead the server to crash.

Letting connections queue is better than swapping - but queueing only helps for few-second peaks. A better solution is to allow more handlers concurrently, even if that means each will run slower. Better yet would be more efficient handlers, and if/once that's impossible, horizontal scaling).

To set ServerLimit, you figure out the amount of memory you want allocated to apache (often "all memory minus basic system use and services like databases", or "memory that seems to be free when apache is not running"), often based on observing
for a bit.

...and then divide that by the amount of resident memory (RES in top) that each apache processes uses (...in the eventual production environment configuration).

For example: consider a server with 2GB RAM, 1GB of which you gave to a database daemon, and you figure system stuff takes ~200MB. You'll want to give at most ~800MB to apache.

You notice apache children have 35MB-40MB resident once they've served a bunch of requests. You would limit the amount of processes to at most 20.

If you set ServerLimit higher than a dozen or two (or MaxClients with prefork, as it's effectively equivalent to ServerLimit with that MPM), you should be able to explain why that is not a bad idea for your set up (a lot of RAM and/or very lean apache children are decent reasons).

(verify) this section: Amount of mappable memory (VIRT in linux top) can also matter - if you allow more memory to be mapped than your system has, it's either oom_kill or system crashes.

Say you also notice that the processes are mapping considerably more -- say, VIRT is almost immediately 80MB and grows to over 200MB in its lifetime. Two things:

  • this likely means memory leaks, so you'll want to set/lower MaxRequestsPerChild to keep that in check).
  • The 20 processes you've just configured will take 400MB to 4GB of non-resident memory - either swapped or mapped but not used (in which case considering it swapped is a safe-side mental model). You would probably want to create and enable a swap file to catch that flak -- even though it'll probably rarely see any access.


Other notes:

  • MaxClients has varying meanings
    • In prefork it's the amount of processes (making it much the same as ServerLimit)
    • in single-process threaded MPMs it's the amount of threads.
    • in hybrid MPMs it's the total amount of threads -- defaulting to ServerLimit*ThreadsPerChild
  • StartThreads and StartServers are usually pretty irrelevant (except in the first few minutes), as with most MPMs some min/max scheme takes over once that many threads/processes have been started.