In the Trenches: Recovering From a Crash

Last week, we suffered through a fairly long system outage at the hands of our web host, Bluehost, and it was not fun. It certainly reminded us how vulnerable the world is to a technology failure, though it also showed that we can live without these tools for short periods of time if needed.

Last Wednesday night, we had about a half dozen clients either en route or preparing to travel. Suddenly, emails just stopped coming. Both the Cranky Flier and Cranky Concierge websites went down (they are hosted on different servers), and the only thing running through my head is probably not appropriate to be printed here.

The biggest and most pressing concern was the email system. It wasn’t a huge problem if our website was down. Sure, we might miss out on new clients who needed help, but this was outside normal business hours so it wasn’t a huge issue, as long as it didn’t stretch on for too long. The more pressing problem was with our existing clients. If they had trouble on their trips, they would have sent their concierges a note, and that note wouldn’t have arrived. Uh oh.

Fortunately, I’ve corresponded with all of our concierges via their personal email addresses before (usually before they joined up), so I quickly put together an email blast and let them know what was happening. For those with the most pressing concerns, I was on the phone and using text messages as well to make sure everything was covered.

I had the concierges send an email to their clients from their personal addresses, asking travelers to email them temporarily at that address if there were any problems. That seemed to do the trick, and our clients quickly responded. All was good. Had we lost the ability to send any email, that would have been of greater concern, but there are enough ways to get on the internet that there should always be a workaround. (If not, then it’s a catastrophic failure and the world has bigger problems than cancelled flights.)

In total, the sites were down for about an hour. This was not a short outage. Email came back to life around 10:30 p.m. local time here, and everything should have gone back to normal. But it didn’t: My confidence in the web host was gone.

I had already been unhappy with my web host, but this was the final straw. It pushed me to make the switch. I’ll talk about that in my next column.

About Brett Snyder

Brett is the Founder and President of Cranky Concierge air travel assistance. He also writes the consumer air travel blog, The Cranky Flier.
This entry was posted in Employees and tagged , . Bookmark the permalink.
2 comments
Xandrios
Xandrios

Hi Brett,I remember a while ago (maybe 6 months?) that you were considering switching hosts. Looking back that was probably not a bad idea ;-)When you business relies so heavily on a stable webhosting platform, bluehost simply cannot do. They are of the "for $5 you get everything you ever wish for, although we dont guarantee that you get anything" kind. That is not enough when your business relies on their services.Time to switch to a quality host i'd say. Dont even look at the ones that offer unlimited anything - by definition offering unlimited is not possible. Look at a host with good reviews, find a package that exactly matches your needs in terms of space and bandwidth, and just buy the next package up. It will cost you a bit more, you wont get 'unlimited' anything (which you dont use anyway)...and you'll be a whole lot happier.

Chris Halcon
Chris Halcon

Brett, sorry to hear about your experience. Outages, even little ones, can do a lot of damage. But you did everything right, letting people know what had happened as soon as you found out and working through it until you were up and running again. These types of outages are more common than many SMBs think and when one passes their way, many aren’t prepared. I work at Symantec and our 2011 Disaster Preparedness Survey found that half of SMBs worldwide do not have a disaster recovery plan in place, and what’s worse, the median cost of downtime is $12,500 per day. And just like you with your web host, 54 percent of SMB customers ditch vendors due to unreliable computing systems, so there’s clearly a lot more to lose than just time and data. My colleague wrote a recent blog post on how SMBs can build a disaster/crisis plan. Would love to hear your thoughts: http://bit.ly/s7Y5E9.Chris Halcon Symantec

Trackbacks

  1. [...] In the Trenches: Recovering From a Crash – Intuit Small Business Blog Our web host crashed last week, and that meant we had to revert to a back up plane to keep our clients happy. [...]