Parts 1 and 2 of this series detailed the disastrous events experienced and the subsequent return to normalcy. Now we get to the meat of it: What have we learned? Is it possible to deal with such adverse conditions differently in the future? If we start by looking forward, preparing for the event, and not for the aftermath, the answer is ‘yes’.
First, traditional Disaster Recovery/Business Continuity measures cannot be discounted. They must work flawlessly. Datacenters, primary or backup, must be functional and accessible. Server and database reloads must occur smoothly and by the numbers. Nothing new here…
However, a combination of just a few basic elements can make any business much more resilient.
Access to cloud-based services gives us options we did not have just a few short years ago. Many of our production systems can be migrated to the cloud, which opens the possibility of access from alternate locations in the event that the workplace becomes damaged or inaccessible. Further, it should be noted that the architecture of these cloud service providers are designed to withstand the type of disastrous conditions that darkened most of New Jersey without interruptions to normal service.
EXAMPLE: Customer Service Lines Unreachable:Internal phone systems, lacking electricity, represented a dead end for incoming customer inquiries. Customers, if unable to contact you, may move on to one of your competitors that IS reachable.
Consider the advantages of a hosted IP-based phone system. All its features remain intact and accessible from anywhere, so the dependence on access to fully operational corporate offices goes away. Such a system is accessible by alternate means, such as a PC based client or smartphone app.
Having production systems up and running is of little value if there are no users able to access them. In an emergency situation, your people must be able to access key systems and communicate with customers and management.
EXAMPLE: In the aftermath of Sandy, restricted or hazardous travel conditions left companies lacking, even though they had power, because agents were not able to report to work to take customer calls.
The solution? Fortunately, technology has been pushing us in the right direction for years… Mobility. Tablets or laptops with cellular internet cards can provide a reliable fallback in most cases where standard internet connectivity is not available. Agents can be productive from home or a hotel room.
Even the most efficient laptops and tablets would be hard pressed to provide more than 8 hours of use without recharging.
EXAMPLE: A departmental manager is able to serve as the focal point for logistics and communications while working from home. However, a cascade transformer failure causes a long-term loss of power. High winds have downed trees; ripping telephone wires from poles. Cell phones and laptops slowly discharge, leaving the manager’s department without leadership.
What we CAN do is buy ourselves a little time. In the event that commercial power is unavailable, automotive jump-start packs can be a lifesaver. Many have built-in lighter plugs, and can be coupled to power supplies designed for car use. These are really DC to DC converters that take the 12 volts supplied and convert it to the voltages needed by laptops, which is generally between 12 and 18 volts. By this method, one may continue to be productive for quite a while; hopefully until a source of commercial power becomes available.
Several sets of these power packs and DC to DC converters can be stored at corporate offices and distributed to key employees in the days leading up to a predictable, impending disaster. Alternately, certain employees could be granted a ‘preparedness allowance’ to purchase and keep such items ready for use.
- Contingency Planning
A critical piece of the whole system that is often overlooked is people. Where do they work? Can they get there? How do they get there? A Disaster Protocol must be established and shared with employees so that everyone knows what to do in the event that the primary workplace becomes inaccessible. Skeleton crews from each critical department must be prepared to communicate and work remotely during any outage.
EXAMPLE: Prior to the Sandy emergency, employees were told that the offices would be closed the following day for safety reasons. Department managers were to notify their groups by telephone when an alternate work plan had been established. Loss of power and phone lines prevented this from occurring. Failure to provide for this possibility resulted in the entire department remaining unproductive for the duration of the outage.
How do you avoid this? It is not enough to say ‘stay home, we’ll let you know what to do’. A series of scheduled departmental call-in sessions should be established in order to react to changing conditions. Communication must be maintained. Without effective communication channels, business continuity is not possible.
A good plan involves everyone, and above all, ensures communication is maintained to the extent possible during a disaster. A drill or practice session should coincide with your usual DR testing.
Many health and safety needs and issues may arise during any sizable event. The first directive of any Disaster Protocol must always be to attend to these health and safety needs first and only proceed to other directives when these needs have been addressed.
We will be posting a sample Disaster Protocol document in the coming days.
One must keep in mind that the most critical need in an emergency is reliable communication, by whatever means are available. When phones and company email are down, are there other methods to get your messages through? Do you make use of social media to keep your customers and employees informed?
No process or plan can mitigate every conceivable risk during an emergency situation. The best plans provide for the safety of all personnel and at least limited functionality under extremely adverse conditions.
Obviously a business is not capable of addressing all of the needs of its employees during an emergency. It IS, however, capable of providing assistance by advising employees on the best ways to address some of them themselves. Reminder emails urging employees to fill their gas tanks, stock up on supplies and be prepared for evacuation would be a good start. For further reading on this topic, refer to the FEMA web site, specifically https://www.ready.gov/make-a-plan, for very thorough and possibly lifesaving advice.