when disaster strikes

The CTOs/CIOs from Xavier University, Premier Health Partners,and Capital Access Network on:

When Disaster Strikes:Essential Technology Solutions

for Keeping the Lines ofCommunication Open

David W. DoddVice President, Information Resources, and CIO, Xavier University

Mikki ClancyVice President and CIO, Premier Health Partners

Franck FatrasChief Technology Officer, Capital Access Network Inc.

IT emergencies are rarely as dramatic — or predictable — as thosecaused by Hurricane Katrina. Often, all it takes is a lightning striketo knock out your communications networks. Many businesses now

rely on 24x7 connectivity — which could spell D-I-S-A-S-T-E-R, unlessyou’ve got backup systems, recovery plans, and trained staff in place.In this ExecBlueprint, CTO/CIO authors share some of the strategiesthat have proven successful in their business continuity and emergencyplanning. Acknowledging that the intrinsic uncertainty of how a crisiswill precisely affect your company’s operations can pose a major chal-lenge to planning efforts; these authors emphasize the need to establishoff-site backup mechanisms as well as redundant pathing for commu-nications systems. In addition, you must provide your staff with the req-uisite training and conduct “mock” disaster drills to test the readinessof both equipment and staff. But, most important of all, they agree, is to have a planning process that you can change as needed, but canexecute with certainty and speed should that time ever come. ■

in partnership with Aspatore Books

™ExecBlueprints

www.execblueprints.com

Action Points

I. In Disaster Planning, What Areas Will IT Need to Address?It only takes a fire or thunderstorm to at least temporarilyknock out your company’s main communicationsinfrastructure: are you prepared? To start, you need tounderstand your company’s mission-criticalcommunications needs and how you will support them — even when you can’t foresee which part of your system will be affected.

II. The Bottom LineDeveloping redundant systems and emergency planscan be expensive — but so are disruptions to yourcommunications and operations. To prepare adequately(but not excessively) for such problems, you need todetermine how much downtime your company canafford. Then, test: will your emergency plans meet thattimeframe?

III. Must-Have Strategies for Testing Your DisasterPlansPlanning is good, but how can you really know how your team, systems, and infrastructure will perform in a crisis unless you test, test, test? While you won’t beable to predict the characteristics of an actualemergency, you can try to make the disaster scenario as realistic as possible by using real disruptive activities (such as office moves).

IV. The Golden Rules for Managing Your Staff inTimes of DisasterYour staff’s labor and skills may never be more important than during an emergency. However, suchevents can be very stressful, and people can easilymake mistakes if they don’t understand ahead of timewhat could happen, and how they can specifically help.Then, as the crisis unfolds, continue to offer them help, support, food, and adequate rest.

V. Essential Take-AwaysBecause the core characteristics of emergencypreparedness must be flexibility and adaptability, its real value is in the “planning” and not in the “plan.”Solutions for keeping communications channels openinclude backup (including Web-based) systems,redundant pathing, and staff who know theorganization’s restoration priorities.

Contents

About the Authors . . . . . . . . . . . . . . . . . . . . p.2

David W. Dodd . . . . . . . . . . . . . . . . . . . . . . . p.3

Mikki Clancy . . . . . . . . . . . . . . . . . . . . . . . . . p.6

Franck Fatras . . . . . . . . . . . . . . . . . . . . . . . . p.9

Ideas to Build Upon & Action Points . . . p.11

Copyright 2009 Books24x7®. All rights reserved. Reproduction in whole or part is prohibited without the prior written permission of the publisher. This ExecBlueprints™ document was published as part of a subscription based service. ExecBlueprints,a Referenceware® collection from Books24x7, provides concise, easy to absorb, practical information to help organizations address pressing strategic issues. For more information about ExecBlueprints, please visit www.execblueprints.com.

© Books24x7, 2009 About the Authors ExecBlueprints 2

About the Authors

David Dodd is vice president forinformation resources and CIO atXavier University. In this position,

he has comprehensive leadership respon-sibility for information technology, theuniversity library, Web development,instructional technology, the registrar, andstrategic information resources.

Mr. Dodd has over 25 years of ITexperience in business, research anddevelopment, and higher education. His technical expertise includes highavailability systems and networks,

collaborative technologies, and conver-gence architectures. In recent years, Mr.Dodd’s areas of professional concentra-tion have included IT strategic planning,organizational transformation, leadershipissues in higher education, and the strate-gic use of information technology. AtXavier, his primary focus is on the devel-opment of a fully integrated informationresources organization capable of supporting 21st-century learners.

He has been an active writer, presenter,and speaker at industry conferences

including EDUCAUSE. Prior to joiningXavier, Mr. Dodd held IT leadershippositions in the University of NorthCarolina and University of South Carolina systems and in various corporate assignments.


☛ Read David’s insights on Page 3

Mikki Clancy is currently the vicepresident and chief informationofficer for Premier Health

Partners (a joint operating companycomprised of Miami Valley Hospital,Middletown Regional Hospital, andGood Samaritan Hospital) in Dayton,Ohio. Ms. Clancy has 20 years of infor-mation systems and security experiencein a variety of industries. At Premier, shehas also served as the senior systems spe-cialist for the Internal Audit Department,Y2K program director, and consultingdirector of computer applications.

Previously, Ms. Clancy worked as abusiness systems analyst for the Centerfor the Disabled, a clinic, school, andmanufacturing employer for the disabledpopulation in Albany, NY. She alsoworked for five years in the United StatesMarine Corps as a programming officerfor network and mainframe systems atCamp LeJeune, NC.

Ms. Clancy is the current chairman ofthe board for Technology First, and is alsoserving on TF’s Greater Dayton ITAlliance board of trustees. She is the former president of the Southeast Ohio

chapter of ISACA and currently a mem-ber of ISACA, HIMSS, CHIME, andACHE. She has been a national and localspeaker on health care information secu-rity, audit, and information technologytopics since 1997. She has previouslyserved as a member of the MIS TrainingInstitute’s HealthSec Seminars AdvisoryBoard.


☛ Read Mikki’s insights on Page 6

Franck Fatras joined Capital AccessNetwork Inc., in 2000 and broughtmore than 13 years experience in

building network infrastructure and appli-cations for national and global use. He isresponsible for directing information

technology, information systems, information security, and facilities.

Mr. Fatras’ previous engagementshave included many diverse industries,including major brands such as Western Union International andWoodward-Clyde.

In 2007 Mr. Fatras and his teamreceived the American Business Award(Stevie) for best MIS and IT organization.


☛ Read Franck’s insights on Page 9

Susceptible SystemsDisasters and challenges to businesscontinuity can range from the rel-ative mundane occurrences such assevere weather, to more critical dis-aster scenarios as were seen after theterrible events of 9-11 andHurricane Katrina. Determiningour most susceptible systems in thecase of a disaster depends to a greatdegree upon anticipating the kindsof disasters that might strike.

If there is one lesson we tookfrom our colleagues at other insti-tutions after the disasters theyfaced, it was that planning is essen-tial. The time to plan for recoveryis not after a disaster has struck.Over the past three years, businesscontinuity and disaster planninghave become standing priorities for our division, and we haveresponded accordingly with plansthat are now tested regularly. Theseplans are based on the awareness ofmission-critical systems, applicationsoftware and data, and recovery pri-orities. The entire campus has beeninvolved in the compilation andtesting of our disaster recovery plan,including the identification of crit-ical systems and specified recoverywindows.

Our first priority is to have a veryreliable, fault-tolerant infrastructure.To that end, we have a productioncomputing facility as well as a pro-duction backup facility on campus.We also spend a considerable amountof time on disaster recovery, business

continuity, and crisis management.Fundamentally, this exercise is aboutiterative planning, testing, and modification/improvement.

Preparing for DisasterAfter the terrible events ofHurricane Katrina and the shoot-ings at Virginia Tech, disaster andcrisis preparedness are getting muchgreater attention in higher educa-tion. Clearly I cannot speak for alluniversities, but the institutionsthat I am familiar with are payinga great deal of attention to thisissue.

Our best practices for maintain-ing continuity during a disaster firstinvolve planning for redundancyand fault tolerance. An IT depart-ment must plan for diversity in thetype of technologies that might bedepended upon during a crisis, andthen develop a plan and test, test,and test again. Testing tells us whatworks and what does not work, andwe learn a great deal when we testand things do not go well; we thenmake changes and improvementsimmediately. Testing is simplyinvaluable in preparing for disaster,and we make the testing environ-ment as real as we possibly can; ithelps us to be prepared and gives usa level of confidence that we will beable to respond effectively in theevent of a disaster. In the process ofour disaster and business continu-ity planning my expectation for ourstaff is simple: it is better to find

deficiencies and omissions in theplanning and testing stage thanwhen we are in the middle of a cri-sis. I do not expect perfection; I doexpect continuous improvement.

We assess the appropriateness ofour disaster planning by looking atbest practices and identifying the besttechnologies available and howthey are being used effectively. Wealso look at what experiences otheruniversities and businesses have hadwith the technologies in question. It should be noted that, in a crisis scenario, communicationis probably the most crucial system.I think some of the lessons from disasters such as those mentioned

© Books24x7, 2009 David W. Dodd ExecBlueprints 3

David W. DoddVice President, Information Resources,

and CIOXavier University

“We manage disaster recovery plan-ning with the same rigor that we man-age other products and initiatives.”

• 25 years of IT experience

• Previously in IT leadership positions: University of North Carolina,University of South Carolina

• Master’s degree, State University ofNew York at Binghamton

• Ph.D. (in process), Higher EducationAdministration, Ohio University

Mr. Dodd can be e-mailed [email protected]


Having a diverse, highly redundantcommunication capability accessed by multiplemeans is critically important.

David W. Dodd

Vice President, Information Resources, and CIOXavier University

have been clear: you cannot relyon voice communication in termsof standard landline phones; youcannot rely confidently on cellphones; you cannot even rely onfault-tolerant networking capabilities.

In our disaster preparedness, wewant to make sure that we are max-imizing the reliability of all of thosesystems, so that in a time of disas-ter or crisis there is likely to be somesubset or component of those systems available to support ourcommunication needs.

Training Key EmployeesOur human resource is our great-est asset, and this is never more truethan in the event of a disaster. Wespend a lot of time preparing ourkey employees to handle a disaster,and we utilize a number of topstrategies. The number-one strategyis to have highly reliable and redun-dant systems and networks that are“owned” by the individuals whosupport them. That is, owing totheir professionalism and dedica-tion, these individuals view thesesystems and networks as symbols oftheir commitment to the institutionand to their customers. As a result,

the quality of those systems and net-works is as high as possible — acritical proposition in the event ofa crisis or disaster.

The second strategy is to give ourcompetent, dedicated, and well-prepared staff experience gainedthrough regularly-conducted train-ing and preparation. We ensure thatour staff members are the mostqualified and committed individu-als available, and we support themas well as we possibly can. As partof that process, we identify criticalskills and knowledge, and I try toensure that resources are available to support our staff members inmaintaining those skills. It is aresponsibility I have to them, so that they can in turn meet theirresponsibility to the institution.

The third strategy is to rigorouslytest responses to scenarios thatcould occur, and by that I mean wedo a lot of simulations and “what-ifs.” We simulate crises and disas-ters, test our responses, and learn agreat deal from the results. Anexample of this involves our databackup system and protocol that fellinto question last year during a dis-aster recovery test. As a result of

what we learned from that process(specifically the difficulties we facedwith restoring data during the test-ing), we completely reviewed andchanged our backup and restoresystem to make it faster and morereliable.

A fourth strategy is to establisha communication plan involvingmultiple means of communication,so that in the event of a disaster ourfolks will know what to do andhow to respond. This is actuallypart of a larger commitment ofensuring that our staff has the besttechnology possible to help them dotheir jobs well. In the event of a dis-aster, having the best technologywith the greatest array of commu-nications capabilities can make anenormous difference. Coping effec-tively with a crisis will require high-quality technologies that are highlyflexible and adaptable, featuringmultiple means of communicationand access. But these technologiesmust also be supported by individ-uals who are well prepared andhighly committed. In my experience,this combination is the best way Iknow to be prepared for thoseevents that we hope never happen.


David W. DoddVice President, Information Resources, and CIO, Xavier University (continued)

We are paying particular attention to cell phone technology and earlynotification systems; in other words, we are looking at texting and voicemail capabilities to provide early warning and early response. We want toensure that the Web is going to be up and running at the earliest possibletime, even if we have to move the hosting to a new location. We also want to make sure that our e-mail server is reliable and able to be restored rapidly. Experience from Katrina demonstrated that the Web, e-mail, and other forms of digital communication are heavily relied uponvery early after a disaster.

David W. Dodd

Vice President, Information Resources, and CIOXavier University

Supporting University StaffFortunately, we have not had to dealwith a serious disaster in the pastfive years. While we have dealt witha number of problems and chal-lenges, we have really not beenfaced with a true disaster. In theevent that we did have to managea real crisis, however, the first thingwe would do is assess the situationas part of a larger institutionalresponse.

Because Xavier has planned wellfor the possibility that a disastercould occur that would requireremote system and network recov-ery, the protocol is largely estab-lished and ready to be carried out.Restoring basic communicationswith university personnel throughall means available is the singlehighest priority. The next priority isto recover systems and communi-cation capability according to theprotocol previously established sothat mission-critical functions of the university can proceed with theshortest interruption possible. Agreat deal of our system architecturehas been engineered to function viathe Web, and to be accessiblesecurely from various locationsover the Internet. The premise isthat as soon as system capabilitiesare restored and communicationestablished, individuals can accessrequired functions.

As stated previously, the natureof the response is to a great extentdependent upon the nature of theincident. Should a disaster bedeclared that requires the restora-tion of systems from our remoterecovery facility, staff would be noti-fied and preparations made for stafftravel, facility availability, andretrieval of backup media.

Challenges in MaintainingCommunicationsThe single greatest challenge tomaintaining a reliable and consis-tent communications system is thatyou simply cannot foresee whichpart of the networks might beaffected by a given disaster. Forexample, will landlines be cut?Will cell towers still function? Willelectrical power be available? Again,diversity in communication chan-nels is critical to avoid putting “allof your eggs in one basket.”Therefore, having a diverse, highlyredundant communication capa-bility accessed by multiple means iscritically important. It is never a per-fect solution, but we try to plan foras much diversity, backup, and

redundancy as possible. Withouteffective communication capability,no real recovery is possible.

One of the key concepts I stressto my audiences and classes whendiscussing strategic planning is thecritical importance of flexibility andadaptability. As Dwight Eisenhowernoted, the real value is in the plan-ning, and not in the plan. Havinga firm grasp of all relevant infor-mation, preserving the opportunityto make changes as required, andacting with commitment and speed,are key factors to success in a highlydynamic situation with unforesee-able complications. And so this ishow we approach disaster recoveryand business continuity planning. ■


David W. DoddVice President, Information Resources, and CIO, Xavier University (continued)

4 Essential Strategies for Preparing IT Staff for Disaster

1. Allow your top members to take ownership of particular systems and networks.

2. Provide regularly conducted training and preparation; identify critical skills and knowledge that they will need.

3. Rigorously test responses to scenarios that could occur; simulate crises and disasters.

4. Establish a communication plan involving multiple means of communication.

The Most Precious SystemsWe have disruptions several timesper month, but they are not all dis-asters. Recently, lightning struck ahospital’s antennae, and that dis-rupted paging. One of our Internetservice providers had a local firethat affected local Internet serviceto some of our remote sites, as wellas a power disruption that affectede-mail.

We do not always have three suchmajor problems every month, butwe do have disruptions on a regu-lar basis. We designed our disasterprogram so that it instructs throughthe range of situations: from a prob-lem to a crisis to a disaster. A dis-aster for us is when we havemultiple systems down; a crisisinvolves individual disruptions.

Our e-mail and paging systemsare the most critical communicationlines; shortly behind them come thecell phone and traditional landlinephones. Because we use VoIP insome of our facilities, any major dis-ruption to the data center will cre-ate a disruption to those systems.We try to put our phones throughdifferent paths so if we lose one wedo not lose them all. We workredundancy into the design of ourcommunication systems. We have tomake sure that phones, pagers, ande-mail have some level of divergentpath, so if one goes down they donot all go down.

Disaster-Plan SpecificsWe have a pretty extensive disasterplan for the hospital, and responseteams for each hospital have ongo-ing responsibilities. The electricalstrike occurred during the course ofa normal thunderstorm, whichaffected outgoing pages and our cellphones. We considered this a crisis.

Because we had to provide alter-native communication mechanisms,people became more reliant on thepagers and two-way radio systems.

For those areas that were dis-rupted by the wireless telephone disruption, we deployed two-wayradios as we waited for the localvendors to repair the antennae.(They had to wait until the stormswere over to go up on the roof.) Thelightning also affected two pieces ofequipment that another vendorhad to come in and repair. This sit-uation required us to bring techni-cians on-site to interface with the vendors, and to ensure that thecommunications part of our com-mand center was up and running sothat we could communicate withour end users. We used e-mail tokeep people informed of the repairstatus, and we managed to haveeverything operational within several hours.

Redundancy Is KeyWe take redundancy very seriously,so we have built it between andamong systems. Our wireless envi-ronment in our critical areas hasredundant pathing. Less criticalareas have a single-threaded envi-ronment, but it is usually backed upwith a hard-line phone for com-munications. We try to make surethat we are four layers deep in ourredundancy to ensure communica-tion via cell phone, landline phone,wireless phone, two-way radio, orsome other means of communica-tion. Emergency disaster phones areanother means of communication.Payphones were once a resource,but they are now defunct.

In the next year we are planningto make some changes in the use of redundancy for our wireless

© Books24x7, 2009 Mikki Clancy ExecBlueprints 6

Mikki ClancyVice President and CIOPremier Health Partners

“My real job is to make sure that people have what they need to get the job done while they are in the middle of a disaster.”

• 20 years of information systems andsecurity experience

• Listed in 2004’s Who’s WhoExecutives & Professionals

• B.S., Engineering Science-TechnologyManagement, Vanderbilt University

• M.B.A., Technology Management,University of Phoenix

Ms. Clancy can be e-mailed [email protected]


During disasters I helpmaintain morale andproductivity withinour department byfeeding them (pizza,sweets, and fruit) andhaving the leadershipteam available toconstantly offer helpand support.

Mikki Clancy

Vice President and CIOPremier Health Partners

environment. We will also spendsome time on the paging environ-ment because we have been noticing more disruption in thatenvironment than we would like.

TrainingThe leadership undergoes trainingtwice per year and the staff usuallygoes through once per year. Becausehands-on training is always themost effective, we use table-top andpractice drills. After-action studiesare always carried out, so that wecan create improvements going forward.

We do, however, realize that anystressful situation can trigger ahuman-nature response; people getscared and tired, and sound judg-ment is not always used. We areextremely neutral during these studies and do not institute any kind

of repercussion for bad judgment(unless there is a trend of bad judg-ment calls on record). People needto be honest about their weaknesses,and we are very careful to create anenvironment that is not punitive.Unreasonable, repeated mistakesneed to be addressed, but the overall atmosphere should be positive.

The CIO’s RoleAn on-site disaster recovery coor-dinator works very closely with theIT department, and we participatein specific hospital disaster plans aswell as identify different tool setsand alternative communicationmechanisms that might be availableduring a disaster. In preparing theplans, I ensure that someone frommy leadership team is available atall times, with the resources needed

to support people and fix problemsswiftly.

During disasters I help maintainmorale and productivity withinour department by feeding them(pizza, sweets, and fruit) and hav-ing the leadership team available toconstantly offer help and support.If it looks like the crisis will lastmore than two or three hours, weput people on rotating shifts veryquickly. We make sure that every-one receives a chance to get restedand refreshed. Luckily, we can procure beds right in the hospital for people to use if the situation warrants.

Measuring Cost andEffectivenessBecause disaster planning is such acomplex topic, we have a great deal of improvements to make in


Mikki ClancyVice President and CIO, Premier Health Partners (continued)

Unless we have spare inventory on site, we usually need outside vendors todo repair work. However, because I have technicians for our traditionalPBX telecommunications area, we usually use our own people as first- andsecond-level technical support. The technicians inside of IT take the troubleshooting as far as they can, and then they bring in the resources to helpwith any third-level support needs.

Mikki Clancy

Vice President and CIOPremier Health Partners

Four Layers Ensure Communications ContinuityA Communications Essential: Redundancy

Cell phones Landline phones Wireless phones Two-way radios Emergency disaster phones

studying return on investment. Welook at the cost of downtime, over-time, repair, and how much redun-dant work we had to put in place.Consequently, it is quite challengingto get a clear picture.

When thinking about hiring out-side vendors, we calculate the per-cent of availability that the hospitalneeds. If they want to have a sys-tem that is 97 percent available, it will cost a certain amount; 99

percent availability will cost more.Our IT steering committee deter-mines what that availability num-ber needs to be for each technologyset in our clinical, financial, andadministrative environments. Thisnumber is then measured on ourbalance scorecard and technologyinvestments are made accordingly.

We measure improvement byhow rapidly we return to normal.To determine the effectiveness of

our action planning, we use aseries of disaster-recovery metrics.On the balance scorecard there is adisaster recovery rating, and it is ourgoal to score as high as possible. Wealso have a security rating and thosetwo together give us our total vul-nerability — or progress — towardthat goal. High availability numbersmean that we are dealing with disruptions in a swift and precisemanner. ■


Mikki ClancyVice President and CIO, Premier Health Partners (continued)

Common Sense TestingDuring a disaster, many communi-cation systems would be quite vul-nerable. Because they run across thesame bandwidth, telephone, e-mail,faxes, and Internet are all inter-connected. While even the loss ofone would be devastating, youwould probably lose all four owingto this interconnectivity. Thus, wetest and update an emergency con-tingency plan once a year, and refine it throughout the year. Our test results are always verygood, and we are content knowingthat, while we are not 100 percent flawless, our emergency process iseffective.

Our testing methods are infusedwith common sense and we try tobe as realistic as possible. Weassume that, during a crisis, hotelsdowntown will still have high-speed Internet at our disposal, andthat our cell tower will still pick upthe signals from our cell phones.(We have not, however, tested thecompany’s reaction to a meteorstrike because, if all services werewiped out, we would probably begone as well!) We practice how wewould react to the loss of the mainoperation center and other realisticrisks. Of course, as much as we

think we are prepared, there areprobably issues that we cannot fore-see. Yet, there are resources that onecan always look to for support.

Single Point of FailureDuring the testing process there isone objective: identify the singlepoint of failure. That is our goalduring the yearly test of the businesscontingency and recovery plans.Every year we go through a mockupof what a disaster would look like.Our primary focus is identifying thesingle point of failure, while alsocommunicating to the staff that adisaster has happened, that a planis in place, and that everybodyknows their roles within that plan.

We did have a telephone outagefor four hours, and we simply re-routed the calls to our New York

location. However, we have neverhad a long-lasting outage. We prac-ticed a few years ago for a wholeweekend. We were moving from

© Books24x7, 2009 Franck Fatras ExecBlueprints 9

Franck FatrasChief Technology Officer

Capital Access Network Inc.

“It is all about the single point of failure. If you can eliminate the singlepoint of failure, you will recover efficiently.”

• With company since 2000

• Over 20 years of experience in system and network architecture

• 2007 recipient, American BusinessAward for best MIS and IT organization

Mr. Fatras can be e-mailed [email protected]


We test and update anemergency contingencyplan once a year, andrefine it throughout theyear.

Franck Fatras

Chief Technology OfficerCapital Access Network Inc.

After identifying the single point of failure, the company must create astrategy to surmount the problem. The level of pressure at this stage dependson the type of company experiencing the problem. I had the great pleasureof working for Western Union International, a company that cannot affordto be down for any amount of time. AdvanceMe, however, is a little moretolerant because we don’t take real-time transactions; they don’t have thesame high-intensity requirements.

Franck Fatras

Chief Technology OfficerCapital Access Network Inc.

one facility to another so it was aperfect opportunity to simulate a complete outage over a main oper-ation center. It went very well.Realistically speaking, we wouldprobably not be faced withoutcommunication capacity for severaldays in real life.

Identify Your Company’sAssetsOur disaster recovery plan takesadvantage of the fact that we havemultiple locations within the com-pany. If there were a disaster inAtlanta, the company is well-versedin how to communicate with New

York to stay informed about theemergency and proceed with busi-ness. While we do not have train-ing programs for all employees, theydo know their individual roles during an emergency and whatalternatives are available. ■

© Books24x7, 2009 Franck Fatras ExecBlueprints 10

Franck FatrasChief Technology Officer, Capital Access Network Inc. (continued)

Testing Capital Access Network’s Emergency Contingency PlanWhen:Once per year

What:Reacting to the loss of the main operation center

Follow-up:Perform updates as needed

Action Steps:1. Identify single point of failure2. Communicate the following to staff: a. Disaster has happened. b. Plan is in place. c. Everybody knows their roles within the plan.

I. In Disaster Planning, What AreasWill IT Need to Address?The disaster doesn’t have to be as far-reachingor long term as 9/11 or Hurricane Katrina tocause serious disruption to your communicationsnetworks. Sometimes the causes, in fact, can berelatively common local phenomena, such as firesor thunderstorms. For this reason, IT must beprepared for the likely consequences of suchevents, including disruptions in service anddamaged equipment and infrastructure. Yourplanning processes should therefore include thefollowing considerations:

• What are likely — or possible —crisis/disaster scenarios for your region?

• Which communications channels (e.g.,telephones, wireless services, Internet) aremost critical to your company’soperations?

• How long can your company afford to bewithout communications capability?

• How can you plan when you don’t knowwhich specific areas of yourcommunications infrastructure will beaffected?

• How will you contact technicians andvendors to conduct repairs duringemergency situations — and how will theyactually execute the repairs?

• How will you support company staffmembers who will be affected — andscared — by the crisis/disaster?

II. The Bottom LineWhile it’s true that your company will derive nodirect income from disaster planning, it will sufferless business disruption (and therefore loss ofincome) during times of crisis if it has investedadequately in developing redundant systems andresponse strategies. When considering the size and scope of your business continuity budget,therefore, your first question must be: in what way(s) is your company vulnerable? (i.e., whatcould it lose?) Then: how can you measure theeffectiveness of your disaster plans (before anactual disaster occurs)? Approaches fordetermining the answers to these questions, andtherefore your company’s appropriate level ofemergency plan investment, are:

• How much do the following cost yourcompany?

• Downtime?

• Staff overtime?

• Repairs?

• Redundant systems?

• What percentage of uptime (or availability)do you require for each system? How doincreases in this percentage impactmaintenance and vendor costs?

• During testing, how did your backup-and-restore systems perform?

• What have your action studies revealedabout staff preparedness and ability toexercise sound judgment in the midst of anemergency?

• How rapidly do your disaster plans returnyour business operations to “normal”? Isthat an acceptable timeframe?

III. Must-Have Strategies for TestingYour Disaster PlansTo prepare adequately for disaster, IT must developan emergency response plan and then “test, test,and test again.” Testing tells you what works —and what does not. As it is always better to finddeficiencies and omissions in the testing phase, youshould endeavor to make the testing environmentas real as possible, incorporating, as appropriate,the following approaches:

• Creating simulations of crises/disasters and“what-if” scenarios

• Making educated-guess assumptions aboutdisaster conditions, based on yourcommon-sense knowledge of your businessenvironment

• Asking your staff during the drill toidentify the “single point of failure”

• Taking advantage of natural disruptions,such as office moves or systemconversions, for your disaster tests

IV. The Golden Rules for ManagingYour Staff in Times of DisasterEmergencies can be scary events, and you needto be prepared for the fact that your staffmembers, as human beings, might not alwaysexercise the best judgment during such times. Awise IT manager, therefore, never expectsperfection. If you adopt the following practices,you should, however, be able to expectcontinuous improvement:

• Identify the staff skills and knowledge thatare necessary during an emergency, andensure that resources are available tosupport your members in acquiring andmaintaining these competencies.

• Let your higher-level staff “own” — andbe responsible for — certain systems and networks, as symbols of theircommitment to the organization.

• Establish communication plans so thatpeople will know what to do and how torespond.

• Even though unreasonable, repeatedmistakes should be addressed, do notpunish poor performance on a disasterdrill.

• Ensure that a leadership team member isavailable at all times during disasterconditions with the resources to supportpeople (including with food, travel, andrest) and fix problems.

• If the crisis will last more than two orthree hours, establish rotating shifts so thateveryone has a chance to attain adequaterest.

• In an actual emergency, inform staff that adisaster has happened and that a plan is inplace.

V. Essential Take-AwaysIn order to cope effectively with a crisis, an ITdepartment must have highly flexible andadaptable technologies that have multiple meansof communication and access, and that aresupported by well-prepared and highly committedindividuals. Moreover, there must be a plan fordeploying these technologies that undergoescontinual testing and modification. Specific bestpractices and technology features that areessential for maintaining continuity following anemergency include:

• Backup facilities, preferably in separatelocations from the main computing facility

• Redundant pathing for cell phone,landline, pager, and e-mail service, so thatcalls can be re-routed quickly, if necessary

• Web-based system architecture that can beaccessed securely from various locationsover the Internet

• Backup communications channels (e.g.,pagers, two-way radios, emergencydisasters phones)

• Early notification systems (e.g., texting andvoice mail capabilities)

• Clearly delineated priorities for servicerestoration ■

© Books24x7, 2009 Ideas to Build Upon & Action Points ExecBlueprints 11

Ideas to Build Upon & Action Points

© Books24x7, 2009 Ideas to Build Upon & Action Points ExecBlueprints 12

Ideas to Build Upon & Action Points (continued)

ExecBlueprints is a subscription-based offering from Books24x7, a SkillSoft Company. For more information on subscribing,please visit www.books24x7.com.

10 KEY QUESTIONS AND DISCUSSION POINTS

When disaster strikes, which communications systems (e.g., telephone, e-mail, wireless) at yourcompany are especially vulnerable? Why are these systems the most vulnerable? What types ofdisasters rend them vulnerable?

How many staff members work in areas that are vulnerable to communications disruptions owing toweather, infrastructure, or other problems? Which communications systems do they use in theregular performance of their jobs? What communications functions are essential to maintain duringa disaster?

What role does IT play in supporting company staff and communications equipment duringdisasters? Has this role changed over time? What staffing levels does IT itself need to maintainduring disasters?

What types of disasters has your department already had to address within the past five years?Which communications systems were affected? How did you handle the situation? How could IT’sresponse have been improved? What steps have you taken to improve IT’s response?

What are your best practices for maintaining continuity of communications during a disaster? Whatredundant systems have you deployed? How do you prioritize the building of this redundancy?

How do you plan to sustain department morale and productivity during and immediately following adisaster? How effective have these strategies been? What could be improved?

What are your top five strategies for preparing key employees to respond to a disaster? What roledo training programs play? Emergency drills? Alternative communications methods? Emergency-response handbooks? What was your role in formulating these strategies?

In the next 12 months, do you plan to make any changes in your communications continuitystrategies? If so, what type of changes are you planning? If not, why not?

When attempting to maintain communications continuity during a disaster, what challenges doesyour company face? In what ways are these unique to your organization? In what ways are theysimilar to those encountered in other organizations? Have these challenges changed over time?

How do you measure the ROI for your continuity strategies? What other ways do you benchmarkthe appropriateness of your disaster-planning strategies?

10

9

8

7

6

5

4

3

2

1

?

when disaster strikes

Documents

disaster planning

backup systems

trained staff

mock disaster

disaster scenario

companys operations

capital access network

communications systems