SilverStripe on HHVM: Part Three
SilverStripe is a (relatively) lean framework, but that doesn't mean it cannot benefit from HHVM's performance gains over Zend PHP. I also ran some tests on a hobbled down SilverStripe to see what kind of performance gains other, not so lean, frameworks might hope to achieve.
Benchmark Setup and Proceedures
I ran these benchmarks back in late October 2012 (I should get better and releasing my results) on a CentOS 6.4 Virtual machine (2 virtual CPUs and 1.5GB RAM) running Nginx 1.4.x and MySQL 5.5.x (Host OS MacOSX 10.8.2, Parallels 8). For the sake of benchmarking, I wanted a 'reasonably' complicated page, and settled on the main blog landing (aka a local version of jaredkipe.com/blog) as the pure processing (and IO operations to MySQL) would be the highest.
For concurrency less than 10 I used 1000 requests, over 10 I used 10000 requests, all on the host computer. (e.g. `ab -n 1000 -c 2 http://jaredkipe.local/blog`) I ran the first of each test twice to let both of them warm up whatever caches they have.
HHVM 2.1* (compiled from source) + JIT enabled. (NGINX + HTTP proxy)
PHP-FPM 5.5.4 (from remi-php55 repo ) + OPCache module. (NGINX + FastCGI)
* technically 2.0 but compiled from the --HEAD shortly before the release of 2.1. Should be representative of 2.1's final performance.
- 34% faster (avg.) mean response time (50% of requests)
- 30% faster (avg.) for worst case (99% of requests)
- 1.52 times as many requests per second! (1.43 to 1.62)
The best part of these benchmarks is that this isn't some kind of, "Well in high concurrency X starts to pull ahead of Y" result. The benefits of HHVM are are a flat 50% or so increase in server throughput across the board for all concurrency/loads.
It should be pointed out that this also isn't a theoretical, pure PHP number crunching, workload. This is PHP doing what PHP does best, making WebPages by loading dynamic files from the hard drive, and dynamic data from a database. Nothing was cached, every request was served from scratch. (Obviously if a PHP source file changed, performance would hurt both fairly equally as HHVM would have to JIT compile the file again, and PHP-FPM would have to scrap that file's opcache results)
10x The PHP Work
I wanted to test how much of this overhead came from making the connection to Nginx and returning to the browser (ab in this case), and also some of the filesystem IO (e.g. both HHVM and PHP-FPM need to make sure the PHP source hasn't changed since the last hit). To make SilverStripe much more greedy, so I made a for loop 10 times over the last two lines of /framework/main.php (the lines that route the URI and build the page).
Note that this still has all of the IO to MySQL
You can think of this as a test on a less lean, more complicated / bloated framework like Magento.
10x Work Analysis
- 43% faster (avg.) mean response time (50% of requests)
- 46% faster (avg.) for worst case (99% of requests)
- 1.74 times as many requests per second! (1.67 to 1.79)
Thats certainly better!
Basically, what this means is that the more complicated the individual page is to render (both in queries and computation like parsing strings) the more of an advantage HHVM is going to have over Zend PHP.