High Performance WordPress – Scaling, Tuning, Optimizing & More

So this past weekend I gave a (nearly) 2 hour talk on High Performance WordPress at Wordconf KL 2011 – it’s a commonly asked question and a lot of people run WordPress but are clueless when it comes to tuning or optimizing their setup.

The talk I gave was pretty technical and in point form, so I’m going to put the notes I made here – the notes the presentation was based on and basically what I talked about. The slides for the talk are here as an overview:

Perhaps you can run through the slides once first then come back here for the expanded version with links/scripts and explanations.

Basics

Start out with WordPress and LAMP architecture
Layers and functional parts of WordPress (core, themes & plugins)
How different things effect performance (themes which make direct DB calls etc)
OS Level stuff (briefly) memory lock, cpu lock, hdd thrashing etc

Start with the OS

What do you do on an OS level?
Turn off logging (only log errors)
tcp.ip kernel settings if necessary (can be touched upon/mentioned) Reference here – Linux Tuning Parameters & Optimizing Linux network TCP/IP kernel parameters
Kill anything that isn’t really necessary but is using memory/cpu cycles
Install a simple firewall to stop zombies/bots costing you a lot of cycles (I suggest CSF/LFD, free and easy to use)
Keep an eye on the hardware for dedicated box users (syslog & messages) – a degrading HDD or dodgy RAM can cause massive unsolvable performance issues (test with bonnie++)
vmem.sh script

Then the LAMP Stack

–Apache

We’ve taken care of the L in LAMP, next is the A — Apache.
Tuning Apache, scripts/methods to calculate Apache main settings (StartServers, MaxClients etc)
Apache Tuning can get really hairy and really intense, if you want to do it properly — do it offline and benchmark with ab (apachebench) or httperf and see how much the hardware can really handle without locking/spiking load.
Script to calculate MaxClients
There are infinite articles out there on tuning Apache, so get out and read them. You will find a lot of conflicting views, but you will also learn a lot. Reduce loaded Apache Modules to a minimum.
Every server needs different settings so I can’t give you a definitive list of settings (KeepAlive etc)
Most default settings by hosts are actually dangerous and can easily kill the server if you get a traffic spike (too many Clients/Threads) — RlimitMem directive and script to calculate it (script for CPanel users on a VPS is here).

–MySQL

There’s a whole other bible on MySQL tuning, don’t worry too much about it just try and make sure your settings won’t kill your server if you get a traffic spike — again too many servers give you dangerous defaults.
The MUST have tools for this job are mysqltuner.pl and tuningprimer.sh — these two tools are basically what you need to get your my.cnf settings in a decent place.
Another must have tool if your server is struggling is mytop, this is basically top for MySQL and will show you a live list of the SQL statements causing.
Also make sure you turn on the slow query log — that can give you some good clues on what is bogging you down (probably a badly coded WordPress plugin)
Make sure you keep your tables defragmented (you can do this with a cron-job at least once a day)
You can also benchmark MySQL using mysqlslap and Super Smack.

–PHP

There’s not a whole lot of performance tuning you can do with PHP, just get your memory settings right for the server and make sure you install some kind of opcode cache — my preference is currently Xcache — but I do use Eaccelerator as well on some sites.
Opcode caching is pretty easy to install so you can test whichever works best for you. I like Xcache because it integrates well with Munin (more on that later).
Cover a few performance related php.ini settings.

Ah finally — WordPress

WordPress plugins are often written by people with no traffic or no consideration for performance, they are often horribly coded and they work — but that’s about it.
Places to look at when WordPress is running like a pig:
-bloated wp_options tables (can crash) – Get Clean Options plugin but use WITH CARE.
-disable all plugins
-switch to default twenty-ten theme
You’d be surprised how much difference plugins and themes can make, if you are really doing a tuning/optimization on a site you need to go through both with a fine-toothed comb.
Themes can make a LOT of unnecessary calls to the database, a lot of which can be hard-coded into the wp_config.php file (such as WP_HOME, WP_SITEURL etc) these stop unnecessary database calls – Reference here: Optimize WordPress Performance with the wp-config.php File
Plugins are the number one biggest problem when it comes to poor performance on WordPress, disable everything you absolutely do not need.

Ok you’ve tuned it up — the server is still struggling — what now?

CDN
Minify
Caching!

Now it’s getting serious, time to look at some kind of caching — the weapon of choice for WordPress ninjas is W3 Total Cache — it can take care of all of the above and can even utilise whatever opcode cache you installed earlier (if it’s Xcache or APC or an old version of eAcclerator)
If fully utilized (including CDN etc) it really can take you to a YSlow! score of A and a 10x increase in load-speed.
If you don’t want to go as far as paying for a CDN, at least separate serving static content onto another more lightweight server using mod_proxy (lightweight = lighttpd, nginx etc). But really CDN wins at life and it’s not expensive (Amazon S3 + Cloudfront is a great and affordable combo).
Make sure you utilise the Browser cache as much as possible (set long expires for things that don’t change often like .css .js and media files).

Ok what more can we do?

Ok if you get to this point, you have a very serious site and should have the money to expand because really you’re going to need more than one machine.
The most basic setup is 2 machines — 1 Apache and 1 MySQL.
I would also suggest at this point, trying to keep Apache out of the picture as much as possible because it’s a memory hog. You could switch to nginx or lighttpd but honestly I don’t see they make that much difference on a really heavily trafficked site unless you use the caching — even then it’s not that great.
This is where something like Varnish comes in, Varnish serves everything from memory — it’s extremely fast and extremely efficient. It doesn’t need oodles of memory either, from my experience the ‘hot’ data set of mosts sites is around 100mb as long as you are using a CDN.
You can run Varnish — Apache and MySQL on a single machine, and it’ll do pretty well. I sugggest at least putting the MySQL on another machine though to be safe.
Run Apache on a different port (4080 or 8080 or whatever floats your boat) and run Varnish on port 80 and point it to https://localhost:8080
Varnish tuning can also get extremely complex, but a basic default setup will server most people just fine.

We’re beyond that too my friend

Well honestly speaking, you’re now reaching a point which very few people reach. You must be looking at some really serious traffic (over 100,00 uniques per day) to want a more serious setup.
Of course this also depends on the hardware specs of the machines, you can easily do that traffic if you have a Quad Xeon CPU machine with 64GB RAM and a Gigabit Internet pipe.
But for most people, that’s not realistic — more small, cheaper machines with load balancing works better (the Google model).
One step up from the above would be a 3 machine setup, 1 Varnish Cache, 1 Apache Web Server and 1 MySQL Database. The database would need the best specced machine, but unless the cache was being totally purged regularly it shouldnt get thrashed too bad.
Then another one up would be a 5 machine set-up, 1 Varnish Cache — 2 Apache Web Servers and 2 MySQL Databases in a Master-Slave config — this can be considered a Fail-over or High-availability setup (HA). Apart from the Varnish machine (which is a single point of failure) — so the ideal situation would be to have 6 machines (1 spare slave Varnish at the front in case the main one dies). This gives you total failover on each of the 3 tiers. The spare Varnish can also function as a backup/media server or whatever you need it for.

What about beyond that, I’m the next Facebook/Twitter/Wordpress.com!

Well then it gets complicated, and it gets complicated really fast! You need to start scaling to n-tier architecture — which basically means it’s infinite.
You need to start looking at a clustered MySQL setup, a distributed file system for the web servers and media like MogileFS or similar.
You need to start looking at memcached to wrap your SQL queries more effectively.
You can also plug HyperDB into WordPress which gives you the ability to read from multiple databases (it replaces the standard wp-db.php class).
HyperDB also supports partitioning of data (putting different tables into different database servers to increase performance).

General Stuff

From a front-end perspective tools like YSlow! and PageSpeed do help A LOT and can really help you narrow down what is slowing your site down. You will see massive gains when using a CDN.
Monitor everything as well as you can without causing a log storm on the server, most important are MySQL slow query logs, Apache error logs and get something like Munin on the servers with as many plugins as possible. Then if something is going titsups you can see when, where, why and check it out immediately.
If you really want lightning fast performance especially on DB servers use SSD not traditional mechanical HDDs.
You can also look at HipHop, some benchmarks for WordPress here and here.

THE END

A lot of the above can be used for any CMS (I’ve done most things on ExpressionEngine too) as long as it’s LAMP based, so use it as you will.

That’s all really, if you have any questions feel free to drop a comment below or drop me an e-mail using the contact form.

You can see all my talks given here.

Comments

comments