PHP Accelerators : APC vs Zend vs XCache with Zend Framework

by Mike Willbanks on December 23rd, 2007

Tonight I wanted to see how the Zend Framework would run against an application that I have put in quite a bit of components into. I figured with the amount of objects that have been instanciated that there was likely going to be a larger performance hit overall. These results may surprise you with what really made a difference as far as PHP Accelerators.

Zend Framework : Release 1.0.3

  • Zend_Cache
  • Zend_Config
  • Zend_Config_Xml
  • Zend_Db
  • Zend_Db_Table
  • Zend_Registry
  • Zend_Loader
  • Zend_Controller_Front
  • Zend_Auth
  • Zend_Acl
  • Zend_Acl_Role
  • Zend_Acl_Resource
  • Zend_Session
  • Zend_Session_Namespace

Testing Environment

  • PHP 5.2.5
  • Apache 2.2.6

Test Results

Pure PHP

Concurrency Level:      10
Time taken for tests:   60.655071 seconds
Complete requests:      298
Failed requests:        0
Write errors:           0
Total transferred:      643149 bytes
HTML transferred:       516971 bytes
Requests per second:    4.91 [#/sec] (mean)
Time per request:       2035.405 [ms] (mean)
Time per request:       203.541 [ms] (mean, across all concurrent requests)
Transfer rate:          10.35 [Kbytes/sec] received

APC

Concurrency Level:      10
Time taken for tests:   60.106697 seconds
Complete requests:      914
Failed requests:        0
Write errors:           0
Total transferred:      1962675 bytes
HTML transferred:       1582035 bytes
Requests per second:    15.21 [#/sec] (mean)
Time per request:       657.623 [ms] (mean)
Time per request:       65.762 [ms] (mean, across all concurrent requests)
Transfer rate:          31.88 [Kbytes/sec] received

Zend Optimizer

Concurrency Level:      10
Time taken for tests:   60.302711 seconds
Complete requests:      262
Failed requests:        0
Write errors:           0
Total transferred:      564135 bytes
HTML transferred:       454727 bytes
Requests per second:    4.34 [#/sec] (mean)
Time per request:       2301.630 [ms] (mean)
Time per request:       230.163 [ms] (mean, across all concurrent requests)
Transfer rate:          9.12 [Kbytes/sec] received

XCache

Concurrency Level:      10
Time taken for tests:   60.261114 seconds
Complete requests:      292
Failed requests:        0
Write errors:           0
Total transferred:      628485 bytes
HTML transferred:       506597 bytes
Requests per second:    4.85 [#/sec] (mean)
Time per request:       2063.737 [ms] (mean)
Time per request:       206.374 [ms] (mean, across all concurrent requests)
Transfer rate:          10.17 [Kbytes/sec] received

Conclusions

What I find quite interesting about this, is that APC was the only one to provide a much higher amount of requests while all the others slowed the requests down. I believe this may be related to utilizing the Zend_Loader functionality which also slows some items down for anything attempting to optimize and cache. I believe after I change that I am going to run these tests again and see if there is any large speed changes. I will post more about this more than likely in a week or so.

From PHP

22 Comments
  1. Mike,

    Please post your configurations for each extension, as those can make a difference. Also, please let us know more about your testing methodology … such as, did you restart PHP between each run of your tests, what sort of hardware were you testing on, how were your sessions configured, if Zend_DB was actually retrieving anything from a database, etc. WIthout that level of context, these results don’t have much meaning.

  2. I find them results very surprising, especially considering Zend Optimizer is in fact slower than running PHP purely by itself. It would be interesting to see exactly what you were in fact testing and run it through kcachegrind.

  3. Andi Gutmans permalink

    Hi,

    Zend Platform is our solution which among other performance management features includes a byte-code cache. This is the product you should have used in order to do this check.
    Zend Optimizer is our byte-code loader for the Zend Guard (encoding) product. It does not include a byte code cache.
    You can download ZP from our site or you can contact me if you need any help. In all products there are also certain dials you want to tune for benchmarking. ZP out of the box does much more than byte-code caching so if that’s your only test you’ll want to tweak the install accordingly

    Happy Holidays.
    Andi.

  4. Kaloyan K. Tsvetkov permalink

    Interesting observation. Have you read this post ?

    http://www.spotsec.com/blogs/archive/sigh-php-including-latency.html

  5. It should be noted that in addition to what Andi mentioned, Zend Optimizer without an opcode cache WILL actually slow down your code.

    It’s optimizing the opcodes, then running them, and then throwing the optimized opcodes away. Without a cache, it’s doing a LOT of work to optimize the compile stage (much more work than a straight compile), and not reaping the results (the 2nd time a well-optimized script runs, it should faster than the raw non-optimized compile).

    Feel free to correct me if I’m wrong, Andi.

    S

  6. I see this confusion in the field a lot. “Are you using an accelerator?” “Yes, we are running Zend Optimizer”. It’s not an accelerator but the name leads to this confusion. I would love to see this test done with the accelerator from Zend Platform.

  7. @Ivo
    I realize that the Optimizer is not exactly an accelerator but it is good to see what it actually is doing in the sense of performance. I am currently working on having the Zend Platform with the same tests. Actually I contacted Andi from Zend to see if there was anything different that I need to do for the benchmark as it is so feature rich. Essentially I turned off some of the main options and it was still rather slow in comparison but I will wait a little longer till I get some more info from Andi on if I need to tweak some more settings that I did not notice right off the bat.

  8. “Optimizer is not exactly an accelerator” – in fact, it’s *not* an accelerator, period, and does not belong in this comparison.

    Zend Optimizer has two functions:
    1. Enabling the execution of Zend Guard encoded files by decoding them in runtime
    2. Preforming some in-line code optimizations (will have a very minor effect on performance when compared to byte-code cache or “accelerators”).

    Actually #1 there means Optimizer does several things on each file PHP loads that actually slow the execution down. In cases where the code optimizations do not compensate for that, you’re bound to have a slowdown.

    However if you are using Zend Platform’s Accelerator or other byte-code cache system, Optimizer can potentially speed things up a little further, because the code optimizations are cached and there’s no need to do them in run time.

    If you’re comparing Zend Platform to other byte-code cache systems, just keep in mind that Zend Platform is much more than just a byte-code cache system. If you want to compare apples-to-apples, make sure you turn the other features off and tune Zend Platform to full performance setup before you run your benchmark. I’d be happy to help with that if you want.

  9. @Shahar –
    I already have redone the benchmarks with ZP, XCache and APC with some different tunings that should make them fairly even keel (or atleast runs them with as close to the same settings as possible). Further it IS in the comparison because XCache can actually utilize the Zend Optimizer being installed.

  10. Zend Optimizer is a bloody stupid name for that product…

    Regards,

    Rob…

  11. hey any new test , also it would be good to see APC and XCache
    run one more time as they should have nearly same performance

  12. Despite all this “compare apples with apples” crap, this showed me what I needed to know. Without spending money or spending much time configuring or learning, which product can I install to improve performance. Answer is APC. I’m happy with that. Thanks for running this benchmark :)

  13. alex permalink

    Year Norio got it just right!

    I want to have a faster site without
    * reading tons of doc
    * modifiying php code

    basically getting results. period.

    i got my answer thanks to this benchmark => i go for APC first
    i’ll struggle with something else in case I see no genuine improvement with this :)

  14. Very helpful information. I didn’t even know Zend Optimizer was w/o Bytecode Cache :) ty

  15. Nightfly permalink

    So, has anyone benched ZF against Zend Server Optimizer+?

Trackbacks & Pingbacks

  1. developercast.com » Mike Willbanks’ Blog: Using PHP Accelerators
  2. breaker of stuff, destoryer of things, sometimes ninja | php accelerator testing
  3. Which PHP Optimizer do you use? - vBulletin SEO Forums
  4. Aceleradores de PHP
  5. justbe3.com » Blog Archiv » Wordpress beschleunigen mit WP- und Bytecode-Cache
  6. Blog de PHP - FinderIT » Blog Archive » Aceleradores de PHP
  7. Another way to cache

Leave a Reply

Note: XHTML is allowed. Your email address will never be published.

Subscribe to this comment feed via RSS