A little background on why I have endeavored on this path. While running Apache at several jobs and institutions it simply became clear that under heavier traffic loads having all of the ridged custom components of Apache were starting to slow things down. To the point of large instances running sometimes up to 300MB in size. Having this size just to get a simple image or PHP page was just not necessary as well as proved to be a performance bottleneck.

While the first point of departure is to utilize a faster web server to proxy the existing one, it still does not stand to the point of keeping everything to a single task. This is where the FastCGI instances of PHP started to come in very useful as well as having something to monitor the processes aka PHP-FPM.

Updates

• 2010/07/13: I have updated the Nginx configuration to put in place what Martin Fjordvald had mentioned in his comment. The fastcgi_params are now far quicker to put in! Thanks again!.

Overview

This posting assumes that you are not operating under a shared hosting account and that you have some dedicated hardware, a virtual machine or something in the cloud. Currently I have just switched from a dedicated server at The Planet to their new cloud offering which reduced my immediate costs substantially (149 to 99 per month for essentially the same thing but even better – my disk now operates off of a SAN. To you cloud junkies, you know disk speed / IO wait time is a pain in the a**. This simply mitigates these types of issues). Which has allowed me to take some time to move over to Nginx and PHP-FPM without having to have reconfigured my existing machine to what I wanted.

Further, you must be comfortable in running everything under a single user account, this is not something for a shared hosting supplier (while you can accomplish this, it is not why I am writing this post since the security of which, is just terrible). Simply put, if you have a shared hosting account switch to a VM (cloud hosting is a VM) or a dedicated server. This may not make some of you happy to hear but there are generally a large amount of issues with shared hosting – mostly related to security. But that is another rant…. Let’s get started.

PHP-FPM

PHP-FPM is a FastCGI process manager for PHP, say goodbye to the spawn-fcgi script from Lighttpd. The PHP-FPM has additional features that can make it appealing for any crowd. To name a few:

• Emergency restart in case of accidental opcode cache destruction
• Enhancements to FastCGI, such as fastcgi_finish_request() – a special function to finish request & flush all data while continuing to do something time-consuming
• Ability to start workers with different uid/gid/chroot/environment and different php.ini (replaces safe_mode) – Covered in the PHP-FPM workers documentation.
• Adaptive process spawning

PHP-FPM has been available for quite some time, however, it is new to the PHP 5.3 branch (to date, it has not been in a packaged release from PHP). There is a bit of configuration that you will need to do with PHP in order to get up and running with PHP-FPM. If you are afraid of compiling or not using your package managers builds, then you may want to stop reading now.

PHP 5.3 + PHP-FPM Installation

Installing PHP 5.3 and including PHP-FPM is simple. I tend to statically compile many of my extensions into PHP to make maintenance more simple with multiple machines. However, this is not a necessity. To get going follow these instructions (if you already have PHP and want to maintain some of your current configuration do a php -i from the command line and grab the configure statement – excluding apache related configuration (aka apxs). I’ve removed most of my configuration from this tutorial. You may find that you need additional librarys installed (just install them through your package manager).

#!/bin/bash
wget http://us2.php.net/get/php-5.3.2.tar.gz/from/www.php.net/mirror
tar -zxf php-5.3.2.tar.gz
cd php-5.3.2
svn export http://svn.php.net/repository/php/php-src/branches/PHP_5_3/sapi/fpm sapi/fpm
rm -rf autom4te.cache
rm -f configure
./buildconf --force
./configure \
--with-layout=GNU \
--with-libdir=lib64 \
--enable-fpm \
--with-gd \
--enable-mbstring \
--enable-pcntl \
--enable-soap \
--enable-sockets \
--enable-sqlite-utf8 \
--enable-zip \
--with-zlib \
--with-curl \
--with-pcre \
--with-jpeg-dir \
--with-png-dir \
--with-zlib-dir \
--with-gettext \
--with-mcrypt \
--with-mysql \
--with-mysqli \
--with-pdo-mysql \
--with-pdo-sqlite \
--with-tidy \
--with-pear \
--with-xml \
--disable-debug
make && make install

Add Your Init Script

Download the script from: http://svn.php.net/repository/php/php-src/branches/PHP_5_3/sapi/fpm/init.d.php-fpm.in

Now that you have the script, modify it to contain the correct paths. I am not going to do this here since if you’re running your own server or virtual machine I feel you should know to do such things .

Update Your Configuration

The default may run fine for you, however, it is always best to take a peak at what users are running, your port numbers and ensuring that all of your items are setup the way you want them. Look for php-fpm.conf under /etc/php-fpm.conf or /usr/local/etc/php-fpm.conf. Then update this file to your liking. Once coompleted, start it up: /etc/init.d/php-fpm start – if you have issues, ensure that your variables are correct and that the configuration lines up with the init script (aka pid file for instance).

Nginx

Installation

Most of your distributions support nginx. You can install this through your package manager, however, I prefer to have the most recent versions (and running a redhat based distro that generally requires installing from source or creating a package – there is no .spec file so I took the easy way out) therefore, I decided to install from source which is a simple task but yet I customized it a bit to exclude a few things I did not need.

Steps that you will need to follow (to some degree):

1. Download Nginx (we’ll use stable for now)
2. Extract the Package
3. Configure, Make and Make Install

Just as a note, I generally save my configuration in a shell script to allow for easy upgrading. You may want to do the same as it will allow you to quickly upgrade rather than attempting to figure out what you’re configure script was.

wget http://nginx.org/download/nginx-0.7.xx.tar.gz #not the real link, find current stable release on 7.x series)
tar -zxf nginx-0.7.xx.tar.gz
cd nginx-0.7.xx
./configure --with-http_ssl_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module
make && make install

So that was easy, or did you take the cheater way out and just install the package (which most are at 0.6)? Either way, this should still work for you.

Many tutorials (especially the one that I looked at beginning to convert to nginx simply rely on a single index.php – not that it is an issue but be wary about these tutorials since many of them exhibit issues related to security (one of which, is the first result in Google on a search for: nginx wordpress rewrite). I’ll say a little more once we get into the Site portion of the configuration.

The Main Configuration

The main configuration file is generally in /etc/nginx/nginx.conf or /usr/local/nginx/conf/nginx.conf. Here we want to setup a few of our variables to improve the server itself. There is a great guide from Slicehost on nginx defaults, you may want to get started there. Now my configuration is based quite a bit off of the Slicehost configuration as well as a few additional changes when we get more into the PHP section. For reference here is my configuration.

user  www-data;
worker_processes  4;
pid        logs/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    server_names_hash_bucket_size 128;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
 
    access_log  /var/log/nginx/access.log  main;
    error_log   /var/log/nginx/error.log   debug;
 
    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     off;
 
    keepalive_timeout  2;
 
    gzip  on;
    gzip_comp_level 2;
    gzip_proxied any;
    gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
 
    include /usr/local/nginx/sites-enabled/*;
}

One thing to note, is that due to some of my domain name sizes, I needed to increase the variable “server_names_hash_bucket_size” to 128 instead of the current default 32.

Setting up FastCGI Variables

These will be needed in order for your installation to work, as well as, setting up your environmental variables that get sent to PHP. You will notice most of these parameters.

; /etc/nginx/fastcgi_params
fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;
fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param  PATH_INFO $fastcgi_script_name;
 
fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;
 
fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;
 
fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;

Now in that same file, you may want to also include some additional fastcgi configuration. These will likely need to be tuned for your environment and may take a little trial and error until you find the best fit.

fastcgi_connect_timeout 60;
fastcgi_send_timeout 180;
fastcgi_read_timeout 180;
fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_intercept_errors on;

Setting up your First Site

Let’s configure a simple default site that just processes PHP scripts. No real hard trouble here! Create a new file in the sites-enabled directory called default.conf. We are going to have it parse our /var/www/default directory (if you do not have one, create one and add an index.php file of some sort in there). Here is a quick and simple configuration to get started:

server {
        listen 80 default;
 
        root   /var/www/default/public;
        index index.php;
 
        location ~ \.php$ {
                include /usr/local/nginx/conf/fastcgi_params;
                fastcgi_index index.php;
                if (-f $request_filename) {
                    fastcgi_pass 127.0.0.1:9000;
                }
        }
}

Note the location ~ \.php$, this tells us that for every PHP file to push it into fastcgi, now if you parse additional extensions through PHP, for the sake of everything, please add extensions to the location! Otherwise your precious PHP files can be downloaded. Yes, it is a serious thing!

What Not to Do

This configuration is evil and I actually found it on some tutorial but here it is, in it’s fully glory.

        # if the request starts with our frontcontroller, pass it on to fastcgi
        location ~ ^/index.php
        {
                fastcgi_pass 127.0.0.1:9000;
                fastcgi_param SCRIPT_FILENAME /var/www/default/pub$fastcgi_script_name;
                fastcgi_param PATH_INFO $fastcgi_script_name;
                include /usr/local/nginx/conf/fastcgi_params;
        }

See how it only pushes things to index.php? Yes, this means that no other PHP files will be parsed. WTF mate? Yes, so please do not utilize a configuration such as this unless you know the repercussions.

Non-Default Sites aka Virtual Hosts

Virtual hosts are extremely easy in Nginx, all you need to define is the server name (the same goes with subdomains). You can basically copy out the default.conf into a specific site and add in a “server_name” variable to the “sever” section followed by removing “default” from the port. You can add in aliases by simply putting a space in between the servers. Here is an example (I also cache local files for a period of 30 days since they will likely not change):

server {
        listen 80;
        server_name digitalstruct.com www.digitalstruct.com;
 
        location / {
                root   /var/www/digitalstruct.com/public;
                index index.php;
 
        }
 
        # serve static files directly
        location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|html|xml|txt)$ {
            access_log        off;
            expires           30d;
        }
 
        location ~ \.php$ {
                fastcgi_pass 127.0.0.1:9000;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME /var/www/digitalstruct.com/public$fastcgi_script_name;
                fastcgi_param PATH_INFO $fastcgi_script_name;
                include /usr/local/nginx/conf/fastcgi_params;
        }
}

That’s just about all you need for now with Nginx, if you run into issues the nginx wiki has quite a large amount of links to get you started in the right direction with rewriting those old nasty apache rewrite rules.

Conclusion

While this posting was not as detailed as I was hoping it to be, I hope that you can see some of the items that I was attempting to address, specifically the PHP portion in ensuring that you are covering yourself when it comes to ensuring that people are not downloading your configuration. Also if you are running multiple virtual hosts, it is a good idea to ensure that they are running under separate users.

Using Atlassian Bamboo for continuous integration will take you a bit to setup, however, I have found it to be an invaluable tool when utilizing the Atlassian stack (JIRA, Confluence, Crucible, Bamboo and Crowd).

Overview

This posting assumes the following:

• You have Atlassian Bamboo setup
• Ant is available on the system
• You already have PHPUnit setup for your project
• You have selected a coding standard

This posting will go over the following:

• Getting Started
• PHP Depend
• PHP Code Browser
• PHP Code Sniffer
• PHP Copy/Paste Detector
• PHP Mess Detector
• PHPUnit

Getting Started

Welcome to the voyage of continuous integration with Bamboo, now that you are ready, lets get started.

Installing the PHP Tools

We will assume using the PEAR installer for installing all of the tools.

PHP Depend

pear channel-discover pear.pdepend.org
pear install pdepend/PHP_Depend-beta

PHP Code Browser

pear channel-discover pear.phpunit.de
pear install phpunit/PHP_CodeBrowser-alpha

PHP Code Sniffer

pear install PHP_CodeSniffer

PHP Copy/Paste Detector

pear install phpunit/phpcpd

PHP Mess Detector

pear channel-discover pear.phpmd.org
pear install --alldeps phpmd/PHP_PMD-beta

PHPUnit

pear install phpunit/PHPUnit

Build Setup in Bamboo

The Plugins

There are a few plugins that we will utilize to capture build metrics. You will want to install these to follow along:

• Bamboo Checkstyle Plugin
• PMD Plugin

Plan Details

Assuming that you are new to Bamboo and have not created a build as of yet, you will need to login to Bamboo and then click on “Create Plan” in the navigation. Here you will find a few options, most of these are self explanatory. Just remember that you can have several different builds so you may want to specify if it is a trunk build or a specific component build. Once you are completed, click on next.

Source Repository

Put in the details to your source code repository. If the source code repository is not available, there is likely a plug-in for it available. Download it, restart bamboo, and continue to this step. We do not force a clean build every time, this is due to some additional configuration that we have set for an environment in a configuration file and to reduce the amount of time to do a build. Lastly, we do polling to detect if we need to do a build. This helps us because we do not want a 5 minute build during every change but rather within 10 minutes of any change. This helps reduce building too frequently or causing a potential for a race condition (assuming there was a bug in Bamboo). Go to the next step…

Builder

Let’s use ant for the build process. Do not worry about this for now, we will get more into the ant build script later on. State the build file as “build.xml” which will live in the main root of the source code (you could place this elsewhere but for simplicity sakes lets put it here and assume you’re not storing the htdocs at the root – please tell me you’re not). Under target place “clean build” since we will do some manual cleanup of existing build information.

Under the following questions you will want to fill out information:

• Where should Bamboo look for the test result files?
  • Check: “The build will produce test results.”
  • Specify custom results directories: “**/build/logs/phpunit.xml”
• Where should Bamboo look for the PMD code analysis output?
  • Check: “PMD output will be produced”
  • PMD XML Directory: “**/build/logs/pmd*.xml”
• Would you like to view Clover Code Coverage for this plan?
  • Check: “Use Clover to collect Code Coverage for this build.”
  • Integration Options: Select “Clover is already integrated into this build…”
  • Clover XML Location: “**/build/logs/clover.xml”
• Where should Bamboo look for the CheckStyle style analysis output?
  • Check: “CheckStyle output will be produced”
  • CheckStyle Output Directory: “**/build/logs/checkstyle.xml”
  • We left the rest of the options blank as we have a few areas to catch up on – don’t we all?

Now go on, the hard work is just about done for setting up the build…

Requirements

In this screen you should see that the JDK is available and Ant is available. Go on…

Artifacts

We are producing several artifacts that we produce during every build, you may want to as well, therefore, lets get them all setup. Don’t worry about what all of these are at the moment, you will have a better idea later.

Notifications

Determine the types of notifications that you are looking for, I simply have myself notified on every build and a different level to ensure that the notification goes out to anyone that has been attributed to a failed build. This is solely up to you and you can change it all later if you would like to.

Post Actions

We currently are not utilizing these, but I am sure that someone will find some more use for them, I am thinking of things like deploying to our testing environment, however, that’s more than a leap away at this point.

Permissions

Setup the permissions that you would like to have on the build plan, this is really up to you.

Finish up and save your build plan. Now you will have a failing build as we need to get it up and running.

Setting up Ant

Since we specified the Ant builder, we need to setup our build.xml file in the directory root of the source control that we are utilizing. In here, we want to run our tasks in parallel to expedite our processing and then run our unit testing. The following is our build.xml file, without a few excludes and such:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76

<project name="My Project Name" default="build">
    <target name="clean">
        <delete dir="${basedir}/build" />
    </target>
 
    <target name="prepare">
        <mkdir dir="${basedir}/build/logs" />
        <mkdir dir="${basedir}/build/codebrowser" />
        <mkdir dir="${basedir}/build/coverage" />
        <mkdir dir="${basedir}/build/cpd" />
        <mkdir dir="${basedir}/build/dependencies" />
    </target>
 
    <target name="phpcs">
        <exec dir="${basedir}" executable="phpcs" failonerror="false" output="/dev/null">
            <arg line="--extensions=php --ignore=build/*,tests/* --standard=Zend --tab-width=4 --report=checkstyle --report-file=${basedir}/build/logs/checkstyle.xml ." />
        </exec>
    </target>
 
    <target name="phpmd">
        <exec dir="${basedir}" executable="phpmd" failonerror="false">
            <arg line=". xml codesize --reportfile ${basedir}/build/logs/pmd.xml" />
        </exec>
    </target>
 
    <target name="phpcpd">
        <exec dir="${basedir}" executable="phpcpd" failonerror="false">
            <arg line="--log-pmd=${basedir}/build/logs/pmd-cpd.xml
                       --exclude=build
                       --exclude=tests
                       ." />
        </exec>
        <exec dir="${basedir}/build/logs" executable="xsltproc" failonerror="false" output="${basedir}/build/cpd/index.html">
            <arg line="/path/to/cpd.xsl ${basedir}/build/logs/pmd-cpd.xml" />
        </exec>
    </target>
 
    <target name="pdepend">
        <exec dir="${basedir}" executable="pdepend" failonerror="false">
            <arg line="--jdepend-xml=${basedir}/build/logs/jdepend.xml
                       --jdepend-chart=${basedir}/build/dependencies/jdepend.svg
                       --overview-pyramid=${basedir}/build/dependencies/pyramid.svg
                       --optimization=best
                       --exclude=build,tests
                       ." />
        </exec>
        <exec dir="${basedir}/build/dependencies" executable="cp" failonerror="false">
            <arg line="${basedir}/path/to/dependencies.html index.html" />
        </exec>
    </target>
 
    <target name="phpcb">
        <exec dir="${basedir}" executable="phpcb" failonerror="false">
            <arg line="--log ${basedir}/build/logs --source ${basedir} --output ${basedir}/build/codebrowser" />
        </exec>
    </target>
 
    <target name="phpunit">
        <exec dir="${basedir}/tests" executable="phpunit" failonerror="true">
            <arg line="--log-junit ${basedir}/build/logs/phpunit.xml
                       --coverage-clover ${basedir}/build/logs/clover.xml
                       --coverage-html ${basedir}/build/coverage" />
        </exec>
    </target>
 
    <target name="parallel">
        <parallel>
            <antcall target="phpcs" />
            <antcall target="phpmd" />
            <antcall target="phpcpd" />
            <antcall target="pdepend" />
        </parallel>
    </target>
 
    <target name="build" depends="clean,prepare,parallel,phpunit,phpcb" />
</project>

Additional Files Referenced

I referenced a few files above that you will not have, mainly these files help to display some of the components that we have pushed out into our Artifacts. I’ve linked one file and given what is in another for reference.

• cpd.xsl
• dependencies.html – a simple file that embeds the svg images.

exit(0);

This should help you get up and running with Atlassian Bamboo for PHP. We’ve been using this for the last 3+ months and it has been extremely valuable. If you have any feedback or anything to add let me know through the comments.

As stated the slides are below, there is a little cut off and I am not sure if MinneBar did record the video in that session so you will have to see when they release the information. Below are the slides of this presentation.