disaster planning & emergency preparedness

Session 1

Disaster Planning and Emergency Preparedness

Cathy Simoneaux, Loyola University New Orleans

Bob Quinn, The Pennsylvania State University

Mary Haldane, Department of Education

2

Disaster Planning & Emergency Preparedness

Cathy Simoneaux

3

When We Left Work on Friday

Katrina Was Forecast to Hit Florida

4

Late Friday, the storm changed course

• “Emergency Plans” began to be implemented Saturday morning

– We came in and secured our basement office

– Students urged to leave campus

– Mandatory evacuation implemented by Sunday morning

5

The Levees Failed Late Monday

6

7

Lakeview-Where My Parents Lived

8

St. Dominic –My Elementary School

9

Mt. Carmel –My High School

10

My Neighborhood –1 mile from Loyola

11

Katrina’s Impact• Personal Impact

(as of 8/14/06)

• 80% of New Orleans flooded, an area equal in size to seven Manhattan Islands

• 1,464 people died; 134 remain missing

• 204,000-plus homes severely damaged 

12

Katrina’s Impact

• 800,000-plus citizens forced to live outside of their homes - the greatest diaspora since the Dust Bowl of the 30's

• 81,688 FEMA trailers occupied

• 1.2 million families received Red Cross assistance

• 33,544 persons rescued by Coast Guard

13

Katrina’s Impact

• 34 years’ worth of trash and debris in New Orleans alone

• 900,000 insureance claims at a cost of $22.6 billion

• (Reference:Women of the Storm

• http://www.womenofthestorm.net/index.php )

14

Hurricane Katrina:Our Story

• Our Emergency Plans Assumed One of the “Normal” Means of Communication Would Always Be Available

– How will you “reconnect” when nothing works?

– Revised basic plan now posted at http://www.loyno.edu/emergency/

15

The Road to Recovery

• Loyola’s IT Department had a comprehensive disaster recovery plan in place

– Data routinely sent to Chicago

– Mainframe operations restored within two weeks of the storm

– Operations transferred to Houston then back to New Orleans

16

Resuming Operations

• Minimal staff pulled to the University of Houston (Academic Affairs) and Alexandria LA (Business Office, Student Affairs, Advancement)

– Shortage of office space “post Katrina”

• NEW PLAN: if this happens again, we are all off to Houston

17

The Internet Let Us Function

• Web-based Policy and Procedures Manual gave everyone access to operational info needed

– Need to add technical processes

• Do Need to Be Sure to Have Some Security Measures to Protect Sensitive Info in the Manual

18

Critical Issue

How Do We Get Students to

Return to New Orleans?

19

Communication With Others Key

• Identify major contact at each major host school

• Over 3,000 students enrolled at over 400 schools during the semester

• President, Provost, Deans, Academic Advisors visited many campuses to answer questions and address concerns

20

• Emergency Web Sites for Students /Parents

– http://loyno.edu/emersite/students

– http://www.loyno.edu/neworleans/

21

“Be A Part of History”

• Opportunities for Community Service– The “Loyola Corps”

http://cba.loyno.edu/loyolacorps/– The “NOAH Project”

http://noah.loyno.edu/Loyola University Community Action Program

http://www.loyno.edu/lucap/

22

Enrollment ComparisonFALL 05 SPR 06 FALL 05 SPR 06 FALL 05 SPR 06 FALL 05 SPR 06

COLLEGE FRESH FRESH SOPHS SOPHS JUNIOR JUNIOR SENIOR SENIOR "Day" Divisions 1041 809 810 702 742 662 698 701 City College 44 22 52 40 118 84 170 138 Law 280 201 44 33 268 235 222 208 TOTAL 1365 1032 906 775 1128 981 1090 1047

2/9/2006

23

Facing the Future

• Projected Drop in Enrollment in the Freshmen Class at all metro New Orleans Schools

– Impacts Revenue for Next Four Years

– Safety of the City is the Major Concern

• Restructuring Plan Being Implemented at Loyola –”Pathways to Our Future”

• http://www.loyno.edu/strategicplan/

Katrina: One Year Later-What Should We Do?

http://www.loyno.edu/

universityministry/katrina.html

25

President’s August 2006 Letter to the Community

• http://www.loyno.edu/financialaid/University%20Update%20%5Bloyolaparents%5D.pdf

26

• We are thankful for all the support that we received in our “hour of need”

– The Department of Education

– The University of Houston

– The Financial Aid Community

27

Contact InformationI appreciate your feedback and comments.I can be reached at:

Cathy Simoneaux

Phone: 504-865-3369

E-mail: cmsimone@loyno.edu

28

Disaster Planning & Emergency Preparedness

Bob Quinn

29

Penn State at a Glance

• 24 campus structure

• 80,124 total enrollment during Fall 2005

– 41,289 at Main Campus (University Park)

– 39,415 at all others

• $650 million in aid disbursed 2005-06

– Most administrative functions performed at UP

30

Penn State Campus Structure

31

A Brief History of DR at Penn State

• The 1980s – computing isolation

• The 1990s – distributed computing

• Post 2000 – DR Initiatives

– Change in leadership (admin computing)

– September 14, 2001 meeting

– 2003 audit cited building proximity

– DR Planning Committee

32

Some Definitions

• Disaster Recovery (DR) – Recovery from unplanned interruptions of normal business processes beyond the immediate ability of the organization’s staff and normal management structure to control. Strongly related to the recovery of the IT infrastructure

33

Some Definitions

• Business Continuity Planning (BCP) – The proactive processes and procedures an organization puts into place to ensure that essential functions can continue during and after a disaster

• DR enables BCP

34

Some Definitions• Incident – An event that leads to a short term

business interruption or loss of data on a small scale i.e.. Water damage to an IT area

• Disaster – Loss of the primary IT area or the loss of the secondary mirrored IT computing facility, but not both facilities

• Catastrophe – Loss of the primary and secondary computing facilities

35

DR Committee Recommendation #1

• Establish a Disaster Recovery Position

• This was probably the single most important strategy

• Currently have multiple positions with responsibility for DR

• Many other participating in DR activities

36

PSU DR Manager

Ken Schroyer

krs5@psu.edu

814.863.8888

37

DR Committee Recommendation #2

• Establish a Catastrophe Contingency Site

• For-Hire sites are expensive

• We chose Altoona Campus

– Somewhat remote

– 40 miles away

– Proximity to high-speed data lines

38

Recommendation #2 (continued)

• Establish a Catastrophe Contingency Site

• Problem: No facilities available

• Immediate/Short-term strategy:

– Mobile Recovery Units• http://www.rentsys.com/

– Rental Equipment• http://mainline.com/

39

Mobile Recovery Unit

40

Mobile Recovery Unit

41

Penn State Campus Structure

42

DR Committee Recommendation #3

• Establish a Local Recovery Site

• Decided to retain local computing facility

• Intended for localized incidents & disasters (more likely than a large event)

• May run some production applications

• http://www.vmware.com/

43

DR Committee Recommendation #4

• Procure Software for Disaster Planning

• Facilitates the development of a DR plan

• Does not build the plan for you

• http://www.strohlsystems.com/

• LDRPS – Living Disaster Recovery Planning System

44

DR Committee Recommendation #5

• Promote DR Awareness

• General staff awareness

• Brochures, web site, DR Day

• http://ais.its.psu.edu/disaster_recovery/index.html

45

DR Committee Recommendation #6

• Establish Emergency Information Repository

• Designed to track students, faculty & staff

• Consolidate general emergency & local information

• Web-based interface

46

Contact InformationI appreciate your feedback and comments.I can be reached at:

Bob QuinnPhone: 814-863-3580Fax: 814-863-0322Email: req1@psu.edu

Disaster Planning & Emergency Preparedness

Mary Haldane

48

Department of Education Continuity of Operations Plan (COOP)

• COOP is the Department’s “Umbrella plan” covering ED’s management of “essential functions.”

• Assures public that Federal Government is operating

• Every Federal Department /Agency has a COOP

• Required by law

49

Department of Education COOP Priorities

• In the event of a crisis, the Department’s priorities are:– The safety and welfare of employees and contractors in,

and visitors to, Department facilities– Ensuring continuation of Leadership and

communication within the Department to include IT resumption of normal business function in all Department offices

– Continuation of the essential business functions of the Department of Education

50

Department of Education Priorities Actions

• Personnel Safety – Evacuation– Shelter in Place (SIP)– Communication

• Call trees• Accounting of staff• Toll free line• Website (Department & Office of Personnel & Management)• Local Media

51

Department of Education Priorities Actions

• Continuity of Leadership– Alternate Sites

• Secretary of Education• Department senior leadership• Essential staff work at home or alternate location

• Continuity of Business Functions– Department

• Hot Site• Federal Student Aid• Business Continuity Plan (BCP) governs the operations during a crisis

and during the restorations of operations

52

Federal Student Aid EnvironmentFederal Student Aid Regional Map

Seattle, WA (13)

San Francisco, CA (61)

Boston, MA (13)

Washington, DC (675)

Atlanta, GA (79)

Denver, CO (17)

Kansas City, MO (24)

Chicago, IL (78)

Philadelphia, PA (21)

New York, NY (28)Seattle, WA (13)

San Francisco, CA (61)

Boston, MA (13)

Washington, DC (675)

Dallas, TX (28)

53

Federal Student Aid EnvironmentFederal Student Aid Major Systems Locations

Meriden, CT

Columbus, GA

Utica, NY

Iowa City, IA

Rockville, MD

Niagara Fall, NY

Bakersfield, CA Atlanta, GA

Louisville, KY

Montgomery, AL

Plano, TX

Greenville, TX

San Antonio, TX

Lawrence, KS

54

Federal Student Aid Business Continuity Plan (BCP)

• Background

– Business Impact Analysis• Priority of business functions

• Essential staff to support priorities

• Essential systems to support priorities

• Records/Files to support priorities

• Alternate site(s)

55

Federal Student Aid Business Continuity Plan (BCP)

• Priorities and recovery times– Capability to process student aid applications and determine

eligibility – three days– Capability to deliver funds to schools to support Title IV aid to

eligible students and parents – three days– Delivering student aid only to fully eligible and participating

schools – three days– Capability to make payments to lenders and Gas in support of

FFELP lender and GA functions – three days– Ensuring that DL, DLC, & DCS (FFELP, FPL, grant

overpayments) loans enter into repayment on time and interest benefits are calculated – four days

56

Federal Student Aid Business Continuity Plan (BCP)

BCP Team Organization and Roles

• Senior Management Team – activates and direct plan in a crisis

• Business Impact Assessment Team – assesses damage and recovery efforts and time frames for recovery

• Business Function Recovery Team – vital system staff lead continuity of operations during crisis and recovery of operations after crisis ends

• BCP Coordinator - Develops and maintains BCP

57

Federal Student Aid Business Continuity Plan (BCP)

Succession Staff and Vital Systems Staff• Chief Operating Officer – five deep• Vital Systems Staff – four deep• Alternate site is home

– Access to Department network– Vital records access

• Shared drive capacity– Communications

• Alternate media: landline, cell, BlackBerry, fax, personal computer

58

Federal Student Aid Business Continuity Plan (BCP)

Awareness and TrainingAll staff

– New staff orientation– All staff - yearly required training – All staff call tree test four times a year

Federal Student Aid BCP Teams and vital systems staff– Tests two times a year– Tabletop exercises two times a year– All staff call tree test four times a year

59

Contact InformationI appreciate your feedback and comments.I can be reached at:

Mary HaldanePhone: 202-377-4324Fax: 202-275-0907Email: mary.haldane@ed.gov