Having some problems with httpd (Apache/2.2.29) memory usage.

Over time, memory usage in the httpd processes creep up until it's eventually at 100%.

Last time I restarted httpd was about 24 hours ago. Output from free -m is:

[ec2-user@www ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          1655       1415        239          0        202        424
-/+ buffers/cache:        788        866
Swap:         1023          4       1019

To prove that it's definitely httpd, I restarted httpd and ran free -m again:

[ec2-user@www ~]$ sudo service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
[ec2-user@www ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          1655        760        894          0        202        360
-/+ buffers/cache:        197       1457
Swap:         1023          4       1019

So, restarting Apache takes free memory from 239 Mb to 894 Mb - which seems like a big leap.

I've been going through the list of currently enabled Apache modules (there's quite a lot) and disabled/removed mod_wsgi and mod_perl (neither of which are required for this server, which is running a PHP-based web application - Magento, specifically).

Based on https://servercheck.in/blog/3-small-tweaks-make-apache-fly, I've run ps aux | grep 'httpd' | awk '{print $6/1024 " MB";}' and get the following output:

[root@www ~]# ps aux | grep 'httpd' | awk '{print $6/1024 " MB";}' 15.1328 MB 118.09 MB 127.449 MB 129.059 MB 117.734 MB 113.824 MB 125.062 MB 123.922 MB 119.855 MB 108.066 MB 136.23 MB 114.031 MB 113.27 MB 110.695 MB 102.113 MB 113.234 MB 186.816 MB 118.602 MB 0.835938 MB

Running the other suggested diagnosis tool for MaxClients which is ps aux | grep 'httpd' | awk '{print $6/1024;}' | awk '{avg += ($1 - avg) / NR;} END {print avg " MB";}' returns the following:

[root@www ~]# ps aux | grep 'httpd' | awk '{print $6/1024;}' | awk '{avg += ($1 - avg) / NR;} END {print avg " MB";}'
110.212 MB

This server (Amazon AWS m1.small instance) has 1.7 Gb of RAM. So, therefore:

Any further pointers/suggestions on how best to tweak the httpd settings or how to diagnose what exactly might be causing this?

  • Note that to check memory usage by processes you need to look at the -/+ buffers/cache line; however in this case the change is comparable. Depending on how apache has been tuned, it may start with just a couple of processes ready for handling requests; after a period of many concurrent requests there will probably more processes forked to handle the load. The number of idle processes can also be tuned. So if apache grows to use too much memory, you need to do some tuning. – wurtel Feb 27 '15 at 10:28
  • I'm OK at Apache server configuration but haven't done a huge amount in terms of performance tuning for Apache in the past. Any suggestions on which directives I should be looking at or guides? Is there a configuration setting I can enable to tell Apache to kill off forked processes after a certain amount of time? (It seems that Apache doesn't do this by default for some reason.) – James Spittal Feb 27 '15 at 23:51
  • This is the exact reason I've moved away from Apache... try Nginx or even Lighttpd. – user67289 Feb 28 '15 at 7:46
  • Interesting. Heard good things about nginx so maybe it's time we give it a try, but honestly, I've been using Apache for years and never had any major problems. Magento really chews memory though. – James Spittal Feb 28 '15 at 10:23
up vote 11 down vote accepted

Here's what I've done to 'solve' it:

  1. Set MaxClients 7 (based on (1740.8Mb Memory on server - 900Mb for MySQL + other stuff) / 111Mb average usage per httpd process = 7.5747747747747747747747747747748)

Therefore:

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients         7
MaxRequestsPerChild  4000
</IfModule>
  1. Disable all Apache modules except for authz_host_module, log_config_module, expires_module, deflate_module, setenvif_module, mime_module, autoindex_module, negotiation_module, dir_module, alias_module, rewrite_module, php5_module

  2. Remove the mod_ssl package since the client isn't using https:// whatsoever.

I'll report back once this new configuration has been running a while to see if this solves it.

Some inspiration here was borrowed from: http://www.activoinc.com/blog/2009/08/31/performance-optimized-httpd-conf-for-magento-ecommerce/ and http://www.activoinc.com/downloads/httpd.conf-magento

  • Above configuration has definitely helped massively. – James Spittal Feb 28 '15 at 10:24
  • 1
    Couple days later, 'memory leak' problems in httpd seem to be gone completely. – James Spittal Mar 4 '15 at 6:07

I'm afraid option MaxRequestsPerChild helped in your case, as it's enables process recycling after defined number of requests, so memory leak is there, but not visible anymore.

Additionally: MaxClients = ServerLimit * ThreadsPerChild

In your case if you need only 7 concurrent users (MaxClients=7) it's totally enough with 2 process (just in case if one will fail to minimize downtime), so config can be:

<IfModule prefork.c>
StartServers       2
MinSpareServers    2
MaxSpareServers   20
ServerLimit        2
MaxClients         8
ThreadsPerChild    4
MaxRequestsPerChild  4000
</IfModule>

I use MaxClients 8, just to make more equal request distribution between 2 processes.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.