Unexpected downtime (5 minutes)

[mark]

I was paged about the site going down. Looking into it, it was caused by our global master database going offline. I initiated a swap on the global slave to promote it to master and the site was back up a few minutes later.

Slicehost sent the following email:

Dear Slicehoster,

At about 20:00 UTC on 01 May 2009, the server hosting your slice became unresponsive to our monitoring system.

Given the repeated issues this server has experienced, we are performing an emergency server swap to replace the hardware. Once the maintenance is complete, all slices will be restarted.

We apologize for the short notice and any problems this may cause you. Unfortunately this is the best course of action to prevent future problems. If you have any questions, please let us know.

We appreciate your understanding and patience as we deal with this issue. Thank you for your support!

Kind Regards,

Slicehost Support

It's kind of unfortunate that we end up with losing a database with no notice, but it's part of the price paid in a hosting environment like we have.

Comments

[syderia]

I just wanted to celebrate not only your quick reply, but the fact that you updated the Twitter page almost as soon as the site went down, and telling us why it went down.

Thanks !

[smilingslightly]

Hear, hear!

[ladylivin]

I agree.

[blithespirit]

Yep, this. <3 the dreamwidth team. :)