Brief weekend outage due to Apache and H-Sphere

Exaflop.org became unavailable on Sunday night until early Monday morning. Sunday is usually the best time of the week to have your website unavailable, but this past Sunday was extra good timing what with the Superbowl. The website was unavailable because I had to change it's IP address and the DNS update takes time to propogate. Read on for the reasons why.

It started when I noticed, having written some php for the site, that when I navigated to an address like http://www.exaflop.org/docs in my browser, the address would be changed to http://exaflop.org/docs/  i.e. it was losing the "www." part of the domain name somehow. This turns out to be a problem in two ways:

1. Any cookies the site issues will be duplicated because cookies are identified by both their name and the domain they come from. I'd recently written some javascript for the site sidebar that uses a cookie to determine if the Docs section is open or not and it stores a cookie to retain that state between pages. Having the domain change makes the behaviour inconsistent.
2. Images are cached by the browser based on their full URL, so if the domain changes like the above example, you end up with two copies of the images in the cache and a less responsive user experience.

The reason this happens is down to the way Apache rewrites URLs that refer to a directory rather than a file. /docs on this server is a directory. The server is set up so that if you request a directory it will look in that directory for index.php and serve that if found, if not it looks for index.html before giving up. This is configured by playing the following in the .htaccess file in the webroot:

DirectoryIndex index.php index.html

When this directive is followed though, Apache doesn't just go ahead and serve the index.php file. It adds a slash to the end of the URL to signify that it's serving a directory if it isn't already there. More importantly though it changes the domain name to whatever is in the ServerName directive in the VirtualHost entry for the site in the httpd.conf file. Mine was set to "exaflop.org" which explains why the "www." part got lost. The reason you could access the site as www.exaflop.org as well was because there is also a ServerAlias directive.

How to fix? Well... if it was my server I'd just have changed the ServerName field to www.exaflop.org and got rid of the ServerAlias. This is where things get complicated though, because I'm using shared hosting at the moment and the hosts are managed using the 'popular' H-Sphere control panel. That doesn't let you change the ServerName directive to Apache. After a couple of support emails I was told the solution is to: remove the server alias; add a sub-domain of "www" which results in www.exaflop.org and exaflop.org being treated as completely seperate sites with their own webroots and everything; then I transfer my files across to the new webroot and set up a mod_rewrite rule on the exaflop.org webroot to redirect all requests over to the equivalent location with www.exaflop.org as the domain.

The problem with this plan is that H-Sphere really does treat the different domains as seperate sites and that extends to their IP addresses. I'd taken the option earlier to have exaflop.org on a dedicated IP address. That meant that www.exaflop.org would have a different IP address from before whatever I did. So I made www.exaflop.org a dedicated IP and reverted exaflop.org back to a shared IP address. My hosting company doesn't charge extra for IP addresses, but there's no sense in wasting them. Unfortunately this now meant that the site was effectively down until the DNS changes propogated across the net.

On Monday morning the DNS changes had come through for me and www.exaflop.org was showing the H-Sphere holding page, so I logged into the host and moved the files across. Then I entered this in the .htaccess file for exaflop.org :-

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www.
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

Now it all works as expected. It was a lot of work to get around a small design issue with Apache (I won't call it a bug, but why does it need to change the domain like this?) which is made worse by H-Sphere being 'difficult'. I should mention that the support from my webhosting company was great as usual. They're not the cheapest around, but the support is good so you might want to give them a look: QWK.net