Web Performance Optimization – WordPress Experiment Results
Have you ever wondered how to improve the performance of a website? Our Web Performance Lab team decided to undertake a project to make our WordPress site faster and handle more concurrent users. Why? So that we could share our findings with all our web geek friends that enjoy learning as much as we do.
This project sounded pretty easy at first, but it didn’t take long to realize that we have a few things to learn when it comes to properly approaching an optimization effort such as this. We made some mistakes and didn’t document the experiments as well as we should in the beginning, so we had to start over. Then we hit our stride and had a fun time with the experiment process.
Basically the process was:
- Run benchmark tests and compile the data
- Make an optimization
- Run new tests and compare performance results with the benchmark
- Turn off the optimization (reset to original state)
- Make a different optimization
- Test again
- Repeat for all tuning options
When we consider “web performance”, we believe there is a common misunderstanding among developers because most people don’t differentiate between performance testing and load testing. Both are necessary to measure how well a site performs.
As we see it, there are two key parts of web performance that we wanted to test:
- Page Speed
Page speed can be measured with one user. That is a form of performance testing. A key metric for speed is the time to load a page – Page Load Time.
The scalability of a site cannot be measure with a single user, so a load testing tool must be used to simulate many users working on a site at the same time. Key metrics for scalability include: Average Response Time, Throughput, Requests per Second, and Error Rate.
First, we ran tests against the site to establish benchmark data. This data shows us the performance before the tuning, so we have useful comparison information with which to gauge the impact of configuration changes or turning on/off various components of the infrastructure.
The Page Load Time was tested with an online tool at www.webpagetest.org which allows you to analyze speed from multiple locations, connection settings (e.g. DSL, cable) and different browsers. It is a helpful tool because it provides graphs that show every request for a page. It also shows interesting information such as Time to First Byte, Time to Start Rendering, and Time for Initial Connection.
The scalability was tested with LoadStorm, a cloud-based load testing tool which produces real web traffic that run test scripts for virtual users, therefore the site gets hammered with realistic simulated activity. The tool measures each response time, determines if there are any errors, and calculates other metrics which are displayed in colorful graphs as the test runs.
We chose to test 3 of the most important pages on our site. These represent most of our visits, and they are very similar in content of the other pages. While we consistently tested throughout the experiment with Chrome from Dulles, Virginia, we ran benchmark tests from six global locations to originate page speed analysis using Firefox, IE9, and Chrome.
We ran approximately 300 performance tests, and here are some of our benchmark findings:
Optimization: Content Delivery Network
Our experiment data regarding a CDN showed the biggest improvement for scalability. We achieved 476% more users before the site started to timeout requests! We chose to use the point of first timeout as the indicator of reaching an upper limit of scalability. Here is the summary from our load testing results:
Consider these before and after graphs from the load testing tool:
The Page Load Time did NOT achieve similar results from turning on a CDN. We only saw about a 5% performance improvement when analyzing the page:
Content Delivery Networks are enormously beneficial to increase site scalability, but they don’t help nearly as much to speed up an individual page.
More details on web performance optimizations with a CDN
Here are the results:
It is imperative to note here that “perception is reality” when it comes to the moving scripts to the footer. Why? Because we are improving the user experience, and by decreasing time for a page to render, the user sees something on the page sooner. That makes a tremendous difference in how a person perceives the speed of our site. By delaying the running of scripts, the browser can process all the visual elements first. HTML gets parsed, images are retrieved, and stylesheets get applied before the overhead of script handling begins. That is excellent for the user because her perspective of performance is driven visually. In this case, she sees the page nearly twice as fast!
This part of the experimentation was a little awkward, but we successfully proved the performance impact anyway. The problem was that before we ran the benchmark tests, we had already enabled server-side caching. During our conversion from Drupal to WordPress, we found an effective WordPress plugin called W3 Total Cache that provides control over several types of server-side caching.
Caching works. It’s a well-known fact in all aspects of computing and system design. We just didn’t know how much it was impacting our WordPress site because we hadn’t measured it. So we decided to turn it off and run tests to see how much performance dropped.
There were two types of caching features we used:
- Page Caching
- Cache Preload
Here is what we found in the caching optimization experiments:
Therefore, Page Caching gave us a 46% performance gain. Memory is a beautiful thing. Use it wisely – cache as much as possible.
However, the results for Cache Preload were disappointing:
That was unexpected. Cache Preload had a 14% negative affect on performance. Our team looked at the results several times and scratched our heads. We thought this would be a no-brainer, but that’s why we play the game and run the real experiments to prove it for certain.
More details of the server caching performance experiments
The most effective optimization for Page Load Time is definitely page caching. And the change that has the biggest impact for scalability is the implementation of a CDN. Here are the numbers:
- Install CDN: 476% improvement in scalability as measured by point of failure (first timeout)
- Enable Page Cache: 45.7% reduction in average load time
- Move JS to Footer: 23.7% reduction in average load time
- Disable Cache Preload: 13.7% reduction in average load time
- Combine JS: 13.2% reduction in average load time
- Install CDN: 4.7% reduction in average load time
We hope you have found this data useful and interesting. If you have any questions or do not agree with the results, we welcome the opportunity to discuss it with you. We believe we followed appropriate scientific method and removed statistical anomalies.