Server – Disaster recovery strategy

The UK server is being synchronized daily with the websites of the USA server, apart from a few exceptions, most website are now fully backed up. Some websites are already being served by the UK server and a synchronisation with the USA server will follow shortly. The big migration is planned for week 33 (Sept 11-15 2008). after that date, a full syncronisation will be taking place to either websites, (some will stay in the US and a mirror will be in action in the UK).

Plans are going really well.

Web Servers Status

Both UK and US web servers now have a status page providing useful information about the their status, the US server has it’s info of the UK server and vice versa. This allows our clients to find out possible down time and causes of the interruption, scheduled or not.

Server Migration – Update

Some additional software incompatibilities are putting the migration on hold for now, some websites will be migrated during the next few days, the good news is that server synchronization tests have been successful, regular mirroring is now going to take place.

Server Migration – Update

Following the early June Server Farm fire and subsequent down time here is a follow up on the progress so far and short terms plans.

The new server in the UK is nearly ready for production and to become the primary server, this will have a impact on server response as there are fewer nodes to go through to reach UK/EU customer base.

However this will be very minimal gains and US, rest of the world visitors will not see a difference. I will also operate a fail over policy for dns, emails and websites:

All websites will be synchronised on an daily basis between the 2 servers, in case of lasting downtime from the UK server, we will be able to reverse to the US server within hours (the time it will take to manually change the DNS records)

For email service, the UK server will handle emails. However, as email is a bit more resilient than websites in a way that if emails cannot be delivered they will be queued until they can, a fail over is less critical.

I am currently experimenting with the possibility of having the two servers able to receive the same email message, if one is not working, the other should. This would require having two email accounts for the same address and access these two accounts using the IP address of the mail server instead of the domain name, it is fairly technical and needs to be tested for feasibility first.

As for DNS, I will also have more than one name server (ie phone book) as DNS protocol require at least one of these to minimise interruption, I will have the record stored in 6 name servers located in different locations. The websites will be unreachable only if the Internet goes entirely down.

Server Migration – Update

Following the early June Server Farm fire and subsequent down time here is a follow up on the progress so far and short terms plans.

The new server in the UK is nearly ready for production and to become the primary server, this will have a impact on server response as there are fewer nodes to go through to reach UK/EU customer base.

However this will be very minimal gains and US, rest of the world visitors will not see a difference. I will also operate a fail over policy for dns, emails and websites:

All websites will be synchronised on an daily basis between the 2 servers, in case of lasting downtime from the UK server, we will be able to reverse to the US server within hours (the time it will take to manually change the DNS records)

For email service, the UK server will handle emails. However, as email is a bit more resilient than websites in a way that if emails cannot be delivered they will be queued until they can, a fail over is less critical.

I am currently experimenting with the possibility of having the two servers able to receive the same email message, if one is not working, the other should. This would require having two email accounts for the same address and access these two accounts using the IP address of the mail server instead of the domain name, it is fairly technical and needs to be tested for feasibility first.

As for DNS, I will also have more than one name server (ie phone book) as DNS protocol require at least one of these to minimise interruption, I will have the record stored in 6 name servers located in different locations. The websites will be unreachable only if the Internet goes entirely down.

Disaster Recovery Plan

A complete memo of Sharpnet UK disaster recovery strategy will be laid out here. This will prove a valuable source of information to other Dedicated server administrators, especially new players.

Comments welcome!