The disaster-recovery lessons we learned after Katrina
The destruction from the hurricane itself, and the subsequent flooding that put most of New Orleans underwater knocked many businesses out of commission—and more than a few completely out of existence. Thankfully, we have learned a lot of hard lessons in the wake of Hurricane Katrina that businesses can use to be better-prepared for the next major disaster.
An article from USA Today in 2007—two years after Hurricane Katrina—estimates that 7,900 businesses in New Orleans and southeast Louisiana went out of existence as a result of Katrina. Some of those businesses failed as a result of lost revenue resulting from nearly half a million people displaced from the region, but many of those businesses failed as a direct result of the destruction and impact the storm had on their ability to continue operating.
For some of the smallest businesses there really is no solution—no way to guard against a catastrophe like Katrina or prepare to handle the next major disaster. Companies that are dependent on physical location or rely exclusively on revenue from local customers will always by heavily impacted by an event like Katrina. However, many businesses did learn the hard way that there are things that can and should be done to increase resiliency and facilitate business continuity during a major catastrophe.
For those businesses that aren’t dependent on a specific location or purely on local customers, for example, the biggest lesson from Hurricane Katrina is the same as the lesson businesses in New York learned from the terrorist attack on 9/11. It can be summed up in the phrase “Don’t leave all of your eggs in one basket,” with the added caveat, “If you do leave your eggs in one basket, at least put that basket in the cloud where it’s safe from natural disasters.”
[ A LOOK BACK: The Aftermath of Hurricane Katrina ]
“When Hurricane Katrina’s devastation struck New Orleans, one of our offices in the city was under water and couldn’t reopen for several months,” explained Kate Campion, president of CPR (Cooperative Processing Resources).
Ric Jones, CIO of LifeShare Blood Centers, faced a similar tragedy. “When Hurricane Katrina’s devastation struck New Orleans, several of our regional centers were closed, demand for blood product substantially increased and donors were not able to give blood due to our site closures, interrupting the flow of business.”
LifeShare Blood Centers also figured out that redundancy is essential for business continuity. LifeShare works with Sungard Availability Services to provide disaster recovery and cloud-based resiliency. “If the Shreveport center is down one or two days, it can’t collect the data it needs to provide blood on demand. Due to this, we have the mirror copy of our Shreveport-based data systems replicated in Sungard AS’ northeast data center,” said Ric Jones.
The bonus lesson is to have a plan. It’s almost a certainty that your business will face a major disaster event at some point and you need to have a clear plan in place that defines how to respond and maintain as much normalcy in business operations as is possible.
Ric Jones warned that lack of planning by his organization could have potentially cost lives. “We did not have a disaster recovery plan in place when Katrina struck our office in Shreveport Louisiana and several other regional centers 10 years ago. The storm closed several locations, and as a result, kept blood donors from being able to give blood at a time of substantial demand increase. The levels of critical blood types dropped to dangerously low levels.”
Entergy—one of the largest energy providers in the area—also suffered the wrath of Katrina. Kay Jones, an Entergy spokesperson, explained, “Our data center was in the storm's way, and we were aware of that. After the event, we made a decision to move the data center. Although we didn't have flooding in the immediate area of our data center, there was no power, our lines were down and we had to run on our generator and keep topping it off.”
In response to the struggles it faced during Katrina, Entergy decided in the winter of 2006 to establish two mirror-image data centers in geographically separated regions for better resiliency. Jones said, “We retrofitted an old library in Little Rock with sturdy brick walls, moving hardware and critical applications from New Orleans piece by piece to the backup facility. We had this completed by 2008,” adding, “Our brand-new $30 million Jackson, Mississippi, data center was completed in 2010. We load-balance several systems, including email, between those two facilities.”
It’s easy to understand how a business could crumble and fail following Katrina. A 2013 study by the Ponemon Institute found that an unplanned data center outage costs an average of $7,900 per minute. That’s nearly $500,000 per hour. Many of the affected businesses were much smaller “Mom & Pop” operations that probably lost significantly less than that per minute or hour, but still enough to force the business to shut its doors. There aren’t many businesses with pockets deep enough to keep things afloat when facing those kinds of losses.
Thankfully, catastrophes the magnitude of Hurricane Katrina are rare. It’s a decade later and parts of New Orleans are still struggling to recover from the havoc wreaked by Katrina. All it takes, though, is one fire, flood, earthquake or other disaster that knocks your business offline to demonstrate the importance of ensuring your business can withstand a catastrophic event.
Make sure you have a business continuity and disaster recovery plan in place and put your mission critical systems and data in the cloud or in redundant data centers to ensure you will have access to them and be able to continue operations even if one site is completely destroyed.