Apache memory use

From Helpful
Jump to navigation Jump to search
Related to web development, lower level hosting, and such: (See also the webdev category)

Lower levels


Server stuff:


Higher levels


This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)


How much it needs / what limit to set

You may want to know how much memory apache process might take, to be able to allocate enough RAM, and/or to avoid over-allocation and the potential trashing that might cause.


Per-process use lies mostly in:

  • process's stack space
a few MB allocated, often less in use
  • loading each configured apache module
since these are shared libraries, they don't add much for each additional process
but maybe count on a few MB per process
  • heap allocation from modules - usually intentionally minimal, or at least well controlled/defined
  • libraries loaded by embedded language runtimes
again, these are usually shared, but may add be a few MB per process
  • heap allocation from embedded language runtimes - could be gbytes
it may be architecturally sane to do this in a specific daemon, instead of in apache children


Note that process summaries showing RSS are over-estimated, because of that shared memory going on.


...so under low load, and assuming you don't have fancy scripty runtimes,

each apache child
is easily 10MB
might be slimmed to maybe ~5MB if you have a very single-purpose server
may be ~30MB if loading a lot of muck



Advice #1: avoid trashing

If you think that more apache processes means more speed, then maybe think thrice about what resource it is that is bottlenecking requests.

There is usually one resource that you will hit first, and define your speed limit. (Advice #2: figure out what that is. This is a bit more advanced, though)

More processes only helps if the thing that would bottleneck you (CPU, IO, whatnot) is not yet at its limit (and unless you have a very efficient architecture and code, chances are you can hit that limit with just a handful of processes).

Once it is at its limit, then more processes doesn't improve speed, it just takes more memory.

It then doesn't increase speed, but it may cause trouble: A high enough number will lead to swapping, and eventually trashing. You have just introduced a new bottleneck, that trashing, that you will always hit under enough load.


With any MPMs that have multiple processes, you want to figure out

  • how much RAM you can effectively dedicate - RAM that will not be used by anything else, even under load
  • how much each process uses, typical and peak
  • and divide those two. That's your maximum amount of processes appropriate for your machine/VM.


On MPMs

When tweaking these settings, you may want to read up on the behaviour of the specific MPM you're using, because some have further settings that effectively also have a limit.

For example, in prefork, MaxClients is effectively equivalent to ServerLimit.


Multi-process MPMs deal with load by creating more processes.


Multi-threaded MPMs and hybrid MPMs are more memory-efficient than non-threaded MPMs (such as prefork) because you get a bulk of handlers/workers/whateeryouwannacall'ems per process, so need fewer processes to serve just as many requests. They're also potentially more trouble, in that everything loaded into apache itself must be thread-safe. (which e.g. PHP isn't. Note: For modules that restrict themselves to prefork (such as PHP), large-scale setups may well want to consider different setups. In the case of PHP, consider using FCGI-style PHP, or just CGI-style PHP)



To set ServerLimit, you figure out the amount of memory you want allocated to apache (say, "all memory minus basic system use and services like databases", or "memory that seems to be free when apache is not running"), often based on observing top for a bit.

...and then divide that by the amount of resident memory (RES in top) that each apache processes uses in production.


For example:

A server with 2GB RAM, 1GB of which you gave to a database, and you figure system stuff takes ~200MB? Then you might want to give at most ~800MB to apache.

You may notice apache children have 35MB-40MB resident once they've served a bunch of requests.

You would then limit the amount of processes to at most 20.


If you set ServerLimit higher than a dozen or two, you should be able to explain why that is not a bad idea for your set up (a lot of RAM and/or very lean apache children are decent reasons).


(verify) this section: Amount of mappable memory (VIRT in linux top) can also matter - if you allow more memory to be mapped than your system has, it's either oom_kill or system crashes.

Say you also notice that the processes are mapping considerably more -- say, VIRT is almost immediately 80MB and grows to over 200MB in its lifetime. Two things:

  • this likely means memory leaks, so you'll want to set/lower MaxRequestsPerChild to keep that in check).
  • The 20 processes you've just configured will take 400MB to 4GB of non-resident memory - either swapped or mapped but not used (in which case considering it swapped is a safe-side mental model). You would probably want to create and enable a swap file to catch that flak -- even though it'll probably rarely see any access.

(verify)


Other notes:

  • MaxClients has varying meanings
    • In prefork it's the amount of processes (making it much the same as ServerLimit)
    • in single-process threaded MPMs it's the amount of threads.
    • in hybrid MPMs it's the total amount of threads -- defaulting to ServerLimit*ThreadsPerChild
  • StartThreads and StartServers are usually pretty irrelevant (except in the first few minutes), as with most MPMs some min/max scheme takes over once that many threads/processes have been started.