How do I optimize Apache?

[Ed. Note: This discussion is for the current default Apache in Meza which is the prefork version of the MPM. At a minimum, we will be switching to the event MPM for greater performance. We are also considering a switch to NginX ]

In order to run apache2buddy in a Meza installation, you must first modify /etc/apache2/apache2.conf to give it the PidFile directive. The pid file by default is /var/run/apache2.pid so the configuration is
PidFile "/var/run/apache2.pid"

Shortcut

curl -sL https://raw.githubusercontent.com/richardforth/apache2buddy/master/apache2buddy.pl | perl

There is a spreadsheet that you can use to make your calculations. Please copy it to your Google Sheets account, or copy the sheet to a new tab with the name of your server.

Calculation

First you need to know how much memory your Apache processes are using.

You can do this with the top command. Press u to show only a specific user, then enter the name of the apache user (ie. ‘apache’ or ‘www-data’ depending on your OS). This will cause top to only show processes for that user. Normally top refreshes every 1.5 seconds, which is probably too quick for you to see and select the data you want. Press d to change the delay to something like 10. We want to select the column of numbers under ‘RES’ (resident memory) so we can add them all up. Press Ctrl + Alt while dragging the mouse over the column of numbers. You should have something like:

22096
52584
64912
63944
36296
56388
66412
52496
44888

Add that up in a spreadsheet. You’ll get 460016 for the example above. The default output is in KiB, so if you enter

“460016 KiB = MB” in Google, it will translate that for you to 471MB. That’s how much our Apache processes are using. Do this a few times to get an average (especially at different times, like when usage is high, to get a more accurate picture of the types of requests you get during ‘normal business hours’).

Now do the same thing for MySQL (user mysql).

Now u, Enter to show all processes. x to highlight the column which is currently being used to sort the data. Press M to sort by %MEM and see what other high memory applications you have running. For example, ElasticSearch which runs on java is taking 346444 KiB (355MB) of memory and over 2GB of virtual memory on this small test system. You can also get this info with a command like echo [PID] [MEM] [PATH] && ps aux | awk '{print $2, $4, $11}' | sort -k2rn | head -n 20

The KiB Mem total line at the top of the output of top shows that there is only 1927MB of total RAM on this system.

PHP also has a memory limit. grep memory_limit /etc/php.ini

memory_limit = 128M

So, if a user is uploading a large file, or some cron process is running, PHP could be using 128MB all by itself.

Taking the total RAM, minus MySQL and ElasticSearch and NodeJS (parsoid), and PHP leaves us with about 600MB of RAM for all other processes including Apache. Knowing that Apache will eat up about 61MB per child, we probably want to limit Apache to no more than 5 or 6 children on this machine. Note that MySQL will allocate it’s memory at startup, in part based on the size of the InnoDB buffer pool. But, the Operating System will kill MySQL if Apache starts using too much memory, thus it is imperative that we control Apache in order to have a stable system.

Configuration

We want to add a section like this to our Apache configuration:

# prefork MPM
<IfModule prefork.c>
# number of server processes to start (default 5)
StartServers       6 

# minimum number of server processes in reserve for new requests
MinSpareServers    3

 # maximum number of idle server processes before they are reaped 
MaxSpareServers   6

# maximum value of clients for the lifetime of the server before it is reaped to prevent mem leaks
ServerLimit      256 

# MaxRequestWorkers was called MaxClients before version 2.3.13. The old name is still supported.
# translates into the maximum number of child processes that will be launched to serve requests on prefork
# the limit on the number of simultaneous requests that will be served; others queued.
MaxRequestWorkers       10 

# Server 2.3.9 and later. The old name MaxRequestsPerChild is still supported.
# number of connections that an individual child server will handle during its life
MaxConnectionsPerChild  1000 
</IfModule>

With these settings in place, restart Apache. (with a stop and start or restart, not just a reload)

Testing, Tuning

You can then test your server’s response with a load testing tool like ab and use uptime and/or top to see the server load.

Start with low numbers to see if your server can handle it.
ab -n 100 -c 10 https://freephile.qualitybox.us/wiki/Main_Page && uptime
Look for load (uptime) to be less than 10 and for there to be no failed requests. Of course you want to have the requests handled in a timely manner as well; which you can gauge by the requests per second and other stats.

Then increase the numbers to something you think will crash or burden your server.
ab -n 500 -c 100 https://freephile.qualitybox.us/wiki/Main_Page && uptime
Note how bad it is (failed requests, time to execute, and uptime load) and adjust concurrency (simultaneous site users) downward until you find the sweetspot where there are no failures, and load is <10
ab -n 500 -c 15 https://freephile.qualitybox.us/wiki/Main_Page && uptime

Once you’ve determined what your server can handle, you can tweak configuration bit by bit until you have it optimized.

Background / More info