PHP Accelerators : APC vs Zend vs XCache with Zend Framework
December 23, 2007
Tonight I wanted to see how the Zend Framework would run against an application that I have put in quite a bit of components into. I figured with the amount of objects that have been instanciated that there was likely going to be a larger performance hit overall. These results may surprise you with what really made a difference as far as PHP Accelerators.
Zend Framework : Release 1.0.3
- Zend_Cache
- Zend_Config
- Zend_Config_Xml
- Zend_Db
- Zend_Db_Table
- Zend_Registry
- Zend_Loader
- Zend_Controller_Front
- Zend_Auth
- Zend_Acl
- Zend_Acl_Role
- Zend_Acl_Resource
- Zend_Session
- Zend_Session_Namespace
Testing Environment
- PHP 5.2.5
- Apache 2.2.6
Test Results
Pure PHP
Concurrency Level: 10 Time taken for tests: 60.655071 seconds Complete requests: 298 Failed requests: 0 Write errors: 0 Total transferred: 643149 bytes HTML transferred: 516971 bytes Requests per second: 4.91 [#/sec] (mean) Time per request: 2035.405 [ms] (mean) Time per request: 203.541 [ms] (mean, across all concurrent requests) Transfer rate: 10.35 [Kbytes/sec] received
APC
Concurrency Level: 10 Time taken for tests: 60.106697 seconds Complete requests: 914 Failed requests: 0 Write errors: 0 Total transferred: 1962675 bytes HTML transferred: 1582035 bytes Requests per second: 15.21 [#/sec] (mean) Time per request: 657.623 [ms] (mean) Time per request: 65.762 [ms] (mean, across all concurrent requests) Transfer rate: 31.88 [Kbytes/sec] received
Zend Optimizer
Concurrency Level: 10 Time taken for tests: 60.302711 seconds Complete requests: 262 Failed requests: 0 Write errors: 0 Total transferred: 564135 bytes HTML transferred: 454727 bytes Requests per second: 4.34 [#/sec] (mean) Time per request: 2301.630 [ms] (mean) Time per request: 230.163 [ms] (mean, across all concurrent requests) Transfer rate: 9.12 [Kbytes/sec] received
XCache
Concurrency Level: 10 Time taken for tests: 60.261114 seconds Complete requests: 292 Failed requests: 0 Write errors: 0 Total transferred: 628485 bytes HTML transferred: 506597 bytes Requests per second: 4.85 [#/sec] (mean) Time per request: 2063.737 [ms] (mean) Time per request: 206.374 [ms] (mean, across all concurrent requests) Transfer rate: 10.17 [Kbytes/sec] received
Conclusions
What I find quite interesting about this, is that APC was the only one to provide a much higher amount of requests while all the others slowed the requests down. I believe this may be related to utilizing the Zend_Loader functionality which also slows some items down for anything attempting to optimize and cache. I believe after I change that I am going to run these tests again and see if there is any large speed changes. I will post more about this more than likely in a week or so.
16 Comments »
RSS feed for comments on this post. TrackBack URI
Mike,
Please post your configurations for each extension, as those can make a difference. Also, please let us know more about your testing methodology … such as, did you restart PHP between each run of your tests, what sort of hardware were you testing on, how were your sessions configured, if Zend_DB was actually retrieving anything from a database, etc. WIthout that level of context, these results don’t have much meaning.
Comment by Clay Loveless — December 23, 2007 @ 9:08 am
I find them results very surprising, especially considering Zend Optimizer is in fact slower than running PHP purely by itself. It would be interesting to see exactly what you were in fact testing and run it through kcachegrind.
Comment by Andrew Johnstone — December 23, 2007 @ 10:24 am
Hi,
Zend Platform is our solution which among other performance management features includes a byte-code cache. This is the product you should have used in order to do this check.
Zend Optimizer is our byte-code loader for the Zend Guard (encoding) product. It does not include a byte code cache.
You can download ZP from our site or you can contact me if you need any help. In all products there are also certain dials you want to tune for benchmarking. ZP out of the box does much more than byte-code caching so if that’s your only test you’ll want to tweak the install accordingly
Happy Holidays.
Andi.
Comment by Andi Gutmans — December 23, 2007 @ 11:27 am
Interesting observation. Have you read this post ?
http://www.spotsec.com/blogs/archive/sigh-php-including-latency.html
Comment by Kaloyan K. Tsvetkov — December 23, 2007 @ 12:00 pm
It should be noted that in addition to what Andi mentioned, Zend Optimizer without an opcode cache WILL actually slow down your code.
It’s optimizing the opcodes, then running them, and then throwing the optimized opcodes away. Without a cache, it’s doing a LOT of work to optimize the compile stage (much more work than a straight compile), and not reaping the results (the 2nd time a well-optimized script runs, it should faster than the raw non-optimized compile).
Feel free to correct me if I’m wrong, Andi.
S
Comment by Sean Coates — December 23, 2007 @ 12:25 pm
I see this confusion in the field a lot. “Are you using an accelerator?” “Yes, we are running Zend Optimizer”. It’s not an accelerator but the name leads to this confusion. I would love to see this test done with the accelerator from Zend Platform.
Comment by Ivo Jansch — December 23, 2007 @ 6:02 pm
@Ivo
I realize that the Optimizer is not exactly an accelerator but it is good to see what it actually is doing in the sense of performance. I am currently working on having the Zend Platform with the same tests. Actually I contacted Andi from Zend to see if there was anything different that I need to do for the benchmark as it is so feature rich. Essentially I turned off some of the main options and it was still rather slow in comparison but I will wait a little longer till I get some more info from Andi on if I need to tweak some more settings that I did not notice right off the bat.
Comment by Mike Willbanks — December 23, 2007 @ 9:13 pm
“Optimizer is not exactly an accelerator” - in fact, it’s *not* an accelerator, period, and does not belong in this comparison.
Zend Optimizer has two functions:
1. Enabling the execution of Zend Guard encoded files by decoding them in runtime
2. Preforming some in-line code optimizations (will have a very minor effect on performance when compared to byte-code cache or “accelerators”).
Actually #1 there means Optimizer does several things on each file PHP loads that actually slow the execution down. In cases where the code optimizations do not compensate for that, you’re bound to have a slowdown.
However if you are using Zend Platform’s Accelerator or other byte-code cache system, Optimizer can potentially speed things up a little further, because the code optimizations are cached and there’s no need to do them in run time.
If you’re comparing Zend Platform to other byte-code cache systems, just keep in mind that Zend Platform is much more than just a byte-code cache system. If you want to compare apples-to-apples, make sure you turn the other features off and tune Zend Platform to full performance setup before you run your benchmark. I’d be happy to help with that if you want.
Comment by Shahar Evron — December 24, 2007 @ 4:19 am
@Shahar -
I already have redone the benchmarks with ZP, XCache and APC with some different tunings that should make them fairly even keel (or atleast runs them with as close to the same settings as possible). Further it IS in the comparison because XCache can actually utilize the Zend Optimizer being installed.
Comment by Mike Willbanks — December 24, 2007 @ 7:48 am
Zend Optimizer is a bloody stupid name for that product…
Regards,
Rob…
Comment by Rob... — December 24, 2007 @ 3:11 pm
[...] PHP Accelerators : APC vs Zend vs XCache with Zend Framework - a comparison of three tools - APC, the Zend Framework components and the XCache tool [...]
Pingback by developercast.com » Mike Willbanks’ Blog: Using PHP Accelerators — December 26, 2007 @ 9:51 am
[...] PHP Accelerators : APC vs Zend vs XCache with Zend Framework - Mike Willbanks : getting into the min… Tonight I wanted to see how the Zend Framework would run against an application that I have put in quite a bit of components into. I figured with the amount of objects that have been instanciated that there was likely going to be a larger performance hit overall. These results may surprise you with what really made a difference as far as PHP Accelerators. [...]
Pingback by breaker of stuff, destoryer of things, sometimes ninja | php accelerator testing — April 23, 2008 @ 5:57 am
hey any new test , also it would be good to see APC and XCache
run one more time as they should have nearly same performance
Comment by Car Lover — June 11, 2008 @ 12:11 am
[...] Have you tried APC as well? I noticed that APC has more complete requests than other PHP optimizers. PHP Accelerators : APC vs Zend vs XCache with Zend Framework - Mike Willbanks : getting into the min… [...]
Pingback by Which PHP Optimizer do you use? - vBulletin SEO Forums — July 25, 2008 @ 11:31 am
[...] Es difícil decir cual de estos tres aceleradores de php es mejor, depende mucho del hardware, la configuración del software instalado y la aplicación(es) que se esté usando. Por ejemplo alguien comparó Drupal con estos 3 aceleradores y eAccelerator le funciono mejor, alguien más con Zend Framework dice que APC es mejor. [...]
Pingback by Aceleradores de PHP — December 2, 2008 @ 6:19 am
Despite all this “compare apples with apples” crap, this showed me what I needed to know. Without spending money or spending much time configuring or learning, which product can I install to improve performance. Answer is APC. I’m happy with that. Thanks for running this benchmark
Comment by Norio — December 8, 2008 @ 10:26 am