httpd memory usage

Question

Having some problems with httpd (Apache/2.2.29) memory usage.

Over time, memory usage in the httpd processes creep up until it's eventually at 100%.

Last time I restarted httpd was about 24 hours ago. Output from free -m is:

[ec2-user@www ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          1655       1415        239          0        202        424
-/+ buffers/cache:        788        866
Swap:         1023          4       1019

To prove that it's definitely httpd, I restarted httpd and ran free -m again:

[ec2-user@www ~]$ sudo service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
[ec2-user@www ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          1655        760        894          0        202        360
-/+ buffers/cache:        197       1457
Swap:         1023          4       1019

So, restarting Apache takes free memory from 239 Mb to 894 Mb - which seems like a big leap.

I've been going through the list of currently enabled Apache modules (there's quite a lot) and disabled/removed mod_wsgi and mod_perl (neither of which are required for this server, which is running a PHP-based web application - Magento, specifically).

Based on https://servercheck.in/blog/3-small-tweaks-make-apache-fly, I've run ps aux | grep 'httpd' | awk '{print $6/1024 " MB";}' and get the following output:

[root@www ~]# ps aux | grep 'httpd' | awk '{print $6/1024 " MB";}' 15.1328 MB 118.09 MB 127.449 MB 129.059 MB 117.734 MB 113.824 MB 125.062 MB 123.922 MB 119.855 MB 108.066 MB 136.23 MB 114.031 MB 113.27 MB 110.695 MB 102.113 MB 113.234 MB 186.816 MB 118.602 MB 0.835938 MB

Running the other suggested diagnosis tool for MaxClients which is ps aux | grep 'httpd' | awk '{print $6/1024;}' | awk '{avg += ($1 - avg) / NR;} END {print avg " MB";}' returns the following:

[root@www ~]# ps aux | grep 'httpd' | awk '{print $6/1024;}' | awk '{avg += ($1 - avg) / NR;} END {print avg " MB";}'
110.212 MB

This server (Amazon AWS m1.small instance) has 1.7 Gb of RAM. So, therefore:

Any further pointers/suggestions on how best to tweak the httpd settings or how to diagnose what exactly might be causing this?

Answer 1

Here's what I've done to 'solve' it:

1. Set MaxClients 7 (based on (1740.8Mb Memory on server - 900Mb for MySQL + other stuff) / 111Mb average usage per httpd process = 7.5747747747747747747747747747748)

Therefore:

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients         7
MaxRequestsPerChild  4000
</IfModule>

2. Disable all Apache modules except for authz_host_module, log_config_module, expires_module, deflate_module, setenvif_module, mime_module, autoindex_module, negotiation_module, dir_module, alias_module, rewrite_module, php5_module

3. Remove the mod_ssl package since the client isn't using https:// whatsoever.

I'll report back once this new configuration has been running a while to see if this solves it.

Some inspiration here was borrowed from: http://www.activoinc.com/blog/2009/08/31/performance-optimized-httpd-conf-for-magento-ecommerce/ and http://www.activoinc.com/downloads/httpd.conf-magento

Answer 2

I'm afraid option MaxRequestsPerChild helped in your case, as it's enables process recycling after defined number of requests, so memory leak is there, but not visible anymore.

Additionally: MaxClients = ServerLimit * ThreadsPerChild

In your case if you need only 7 concurrent users (MaxClients=7) it's totally enough with 2 process (just in case if one will fail to minimize downtime), so config can be:

<IfModule prefork.c>
StartServers       2
MinSpareServers    2
MaxSpareServers   20
ServerLimit        2
MaxClients         8
ThreadsPerChild    4
MaxRequestsPerChild  4000
</IfModule>

I use MaxClients 8, just to make more equal request distribution between 2 processes.

Answer 3

Memory computation

The other answers already tackle how to set the values. And for it you need to compute the memory usage of different elements of your system.

I want to tackle memory computation which can help many.

• ps show processes only and is a good tool to use if you want to use RSS as a value
  • RSS (Resident Set Size) over report memory, due to adding the whole shared memory (shared libraries) and repeatedly through all processes.
• htop or top by default show not only all processes but all threads of a given entity

  To show process only press H to toggle between showing threads and not showing them

Smem

For a better tool for computing memory usage, You can use smem

apt install smem

it will give pss value which it's better to rely on:

Provided values

• RSS (Resident Set Size)
  • ram memory allocated to process Including shared memory (repetitive) shared libraries allocated once, used by many processes RSS over report memory usage
• PSS (Proportional Set Size)
  • divide the shared memory by percentage through all processes
  • 4 processes using same shared lib, they wil be computed with 25% each from the shared memory
  • in contrast to RSS, which for each process it compute 100% of that share memory
• USS (Unique Set Size)
• USS and PSS are terms and concepts that were proposed by Matt Mackall, the author of smem.

Some options

• k : memory in mb
• t: show total memory in the bottom 🔥

Filtering

• process
• mapping (libraries)
• user

Getting apache2, mysql, php-fpm, nginx memory usage

Memory for apache2 process with total computation as well

smem -tk -P apache2

Same but only selecting pss column

smem -tk -c pss -P apache2

Showing pss total only

smem -tk -c pss -P apache2 | tail -n 1

Change apache2 with any of the other processes.

Example

You can see each process with it's child processes consumption.

PSS vs RSS

If we want to tune to the max of the server capabilities. PSS may be a better value to use.

Otherwise, we can use RSS as it over-reports memory. And that's can be safer. And then one can tune better under stress testing. Through experimentation.