Incident Management SystemBlackrock 3 Training & Consulting

Program Overview

About Us

4 Who We Are– Deep global experience in Incident Management and Critical Infrastructure– Incident Management: Fire, Special Operations & Law Enforcement

• HazMat/WMD (CBRNE), Technical Rescue, Anti-Terrorism, Counter Proliferation– Critical Infrastructure: Industry

• Fiber Networks, Data Centers, Oil & Gas, Power, Capital Markets– Market Leader in IMS for IT

4 What We Do– Maximize Uptime During High Severity IT Incidents

• Assess, Train, Evaluate & Exercise Incident Response Teams– Collaborate with Customers to Build a Culture of Incident Response

• Global Cloud Providers, Fortune 500 Enterprises, Service Providers & Scale Web Properties• Incorporate IMS into ITIL, DevOps, Agile, Lean Practices

– Engage with Teams Across the Customer’s Organization• NOC, Ops, Site Reliability, Cybersecurity, Mission Critical Support, SMEs, Executives

2

Why IMS?

4 Incident Management System (IMS)– National standard for managing all-hazard/all-risk incidents for the last 40 years– Battle-tested methodology (not software) designed specifically for emergency response– Public Safety across U.S. uses IMS to manage over 1 Million incidents/year– Organizational framework & processes for leading people during emergencies– Blackrock 3 Partners (BR3) pioneered use of IMS in IT environments 3

4

Problem Monitoring

Incident Commander

Network Database

DBA - 1 DBA - 2

SAN / Storage

Customer Liaison

Executive Liaison

IncidentResponse

Resolution

AAR & RCA

Notification

Incident Lifecycle

Predictive

Adaptive

P/SEV Level

Normal Ops Normal Ops

Uptime

Peacetime

Downtime = WartimePeacetime

Uptime

731

731

IMS

Is Your Team’s Response?

PredictableRepeatableOptimizedClearEvaluatedScalableSustainable

5

BR3 Philosophy & Methodology

4 Training– All training starts from a common foundational level– Training classes are progressive through the curriculum– Scenario based training is most effective when tailored closely to production environment– Focus on interpersonal skills of leadership under adverse conditions

4 Exercises– Exercises are challenging and progressive

• Inject human behaviors and situations into exercises to evaluate specific conditions – Exercise methodologies

• Select group against random problem• Random group(s) against same problem• Random group(s) against random problems

– Timing of exercises• Announced date and time• Unannounced date and time

4 Training & Exercises are the Best Predictors of Future Incident Performance

Train & Exercise

4 Training Curriculum– Awareness Level

• IMS Awareness for Executives• IMS Awareness for Stakeholders

– Operations Level• IMS Operations for SMEs

– Command Level• IMS/IC (Incident Commander) Class

4 Exercise Curriculum– Tabletop Workshops

• IMS P1/S1 Workshop• IMS Command Workshop

– Mobilization Exercises (MOBEX)• Periodic (monthly, quarterly, annually) and escalating exercises

– Game Day• “Live”, unannounced exercise in production or near production environment

IMS Awareness for Executives

4 Overview of IMS Awareness for Executives– Explain IMS as a best practice for IT Ops– Define the state of Peacetime and Wartime and the shift between states– Describe the role of IC during high severity incidents– Explain how Executives participate in IMS– Recommend how Executives message the importance of IMS– BR3 leads 1-2 hour IMS face to face discussion with Executives

4 Target Audience– Senior Business Executives

4 Recommendations– Identify Executives – Schedule face to face discussions to maximize participation– Formal (on-site visits) or informal (over dinner/off-site) settings

8

IMS Awareness for Stakeholders

4 Overview of IMS Awareness for Stakeholders– Explain IMS as a best practice for IT Ops– Define the state of Peacetime and Wartime and the shift between states– Describe the role of IC during high severity incidents– Explain how Stakeholders participate in IMS– BR3 leads 3 hour IMS face to face class with Stakeholders

4 Target Audience– Stakeholders including Customer Support, Internal & External Communications

4 Recommendations– Identify all Stakeholders– Schedule face to face classes to maximize participation– Location and # of classes determined by # of locations and # of Stakeholders

9

IMS Operations for SMEs

4 Overview of IMS Operations for SME– Explain IMS as a best practice for IT Ops– Define the state of Peacetime and Wartime and the shift between states– Describe the role of IC during high severity incident bridges– Define “span of control” and how it must be understood by IC and SMEs– Explain rules of engagement for SMEs and how to participate in IMS– Instructor led 4 hour IMS training class for SMEs

4 Target Audience– Anyone that could respond to a P1/S1 incident– SME management/engineers by function (e.g. Network, Database, Storage, App)– Vendors, contractors, consultants

4 Recommendations– Identify everyone in Target Audience – Schedule IMS Operations for SMEs to maximize participation– Location and # of classes determined by locations and # of SMEs

10

IMS/IC Class

4 Overview of IMS/IC Class– Explain IMS as a best practice for IT Ops– Define the state of Peacetime and Wartime and the shift between states– Describe the role & function of the Incident Commander (IC) – Describe the differences between Command and problem solving– Conduct a series of escalating team exercises – Develop interpersonal skills for effective incident management – BR3 instructors deliver training/exercises– Class is 2 consecutive days: Day 1 - IMS; Day 2 – IC

4 Target Audience– ICs or individuals identified as potential ICs

4 Recommendations– Optimal class size is 10-12 participants due to nature of in-class exercises– Coordinate IMS/IC Class around shift schedules to include all in Target Audience– Location and # of classes determined by # of locations and # of ICs

11

IMS P1/S1 Workshop

4 Overview of IMS P1/S1 Workshop– Facilitated role-playing for scenarios solving difficult technical problems– Participants assume IC and SME roles for each technical scenario– Each participant will be IC for at least one scenario– Client to provide 6-8 technical problem scenarios and drillmaster– All scenarios led by BR3 and client technical drillmaster

4 Target Audience– ICs, SMEs and Operations/Site Reliability leadership

4 Recommendations– Each scenario requires 1 hour for set-up, role play, evaluation– Coordinate P1/S1 Workshops around shift schedules to include all in Target Audience– Location and # of P1/S1 Workshops determined by # of locations and # of ICs

12

IMS Command Workshop

4 Overview of IMS Command Workshop– Facilitated role-playing for scenarios with soft people skills problems– Participants assume IC and SME roles for each scenario– Each participant will be IC for at least one scenario– Client to provide 6-8 technical problem scenarios and drillmaster– BR3 to provide behavioral and situational injects for scenarios– All scenarios led by BR3 and client technical drillmaster

4 Target Audience– ICs, SMEs and Operations/Site Reliability leadership

4 Recommendations– Each scenario requires 1 hour for set-up, role play, evaluation– Coordinate Command Workshops around shift schedules to include all in Target Audience– Location and # of Command Workshops determined by # of locations and # of ICs

13

MOBEX and Game Day Exercises

4 Mobilization Exercises (MOBEX)– Exercise designed to evaluate the readiness of multiple teams to a P1/S1– Quarterly

• 2-3 hours in length– Annually

• 4-6 hours in length

4 Game Day– “Live”, unannounced exercise in production or near production environment– Purposely injecting major failures into critical systems

• Discover flaws and subtle dependencies• Tests systems, software AND people response

– Long duration, multiple “Operational Periods” exercise• Multiple behavioral and situational injects• Exercise design focuses on specific areas and items spotlighted for evaluation

Standard Engagement

4 BR3 Engagement Steps– BR3 signs Mutual Non-Disclosure Agreement (NDA) with client– BR3 provides IT Ops Questionnaire to client

• Client responds with statistics, existing process, etc.• Client identifies teams and individuals by location and function• Client describes challenges facing IT Ops team• Client describes growth plan for its’ business

– BR3 conducts follow-on interviews to review client responses• On-site or conference call

– BR3 provides recommendations and Proposal to client• Training Plan and Exercise Plan• Project or Program Basis

– BR3 and client execute Services Agreement/Purchase Order– client sets up BR3 in client’s Purchasing System – BR3 and client schedule & implement Training Plan– BR3 provides feedback to client management & recommends next steps

15

Portfolio of Blackrock 3 ServicesDescription

1) Gap Analysis to Best Practices

2) IMS Awareness for Executives and Stakeholders

3) IMS Operations for Subject Matter Expert (SME)

4) IMS/IC Training

5) Incident Conference Call/Communications Review, Evaluation & IC Coaching

6) IMS P1/S1 Workshop Exercises

7) IMS Command Workshop Exercises

8) After Action Review (AAR) Process

9) Staffing Analysis & Recommendations

10) Personnel Selection, Development and Advancement

11) Executive Briefing

12) Unified Command (UC), Emergency Operations Center (EOC), War Room Training & Exercises

13) Game Day scale exercise in production environment

14) Crisis Communications to external stakeholders16

Blackrock 3 IMS Experience

4 Large Scale, Multi-Jurisdictional, All-Hazard, High Consequence Events

4 Emergency Response– 2,800 Major Hazardous Material Incidents in Northeast Corridor

• Numerous “white powder” anthrax calls post 9/11/01 in Capital Beltway area– Large-Scale Wildland Fires (1990 – 2015)– Major Oil Spills in San Francisco Bay (2007 & 2009)– World Trade Center (2001), Urban Search and Rescue

4 Pre-Planned Events– Presidential and Presidential Candidate Visits (2008 - 2012)– BART/Oakland/Oscar Grant Riots (2009 & 2010)

4 Exercises– Urban Shield (2013-2015), largest annual Homeland Security exercise

• IC for Red Command: Fire; Hazardous Materials; Technical Rescue– Weapons of Mass Destruction (WMD) exercises in 40 countries (1999 – 2015)

• SWAT, Bomb Squad, Border Patrol, Ports, Law Enforcement, Fire, EMS, Intelligence 18

Principals

Rob SchneppRob's emergency response career spans 33 years in international public safety / counter terrorism as a Special Operations Fire Chief, Incident Commander, consultant and published author in Chemical, Biological, Radiological, Nuclear and Explosive (CBRNE) Hazardous Materials response for the Alameda County Fire Department, Department of Defense, and numerous public agencies, companies and two Department of Energy national laboratories. Rob is on the curriculum development team and teaches at the National Fire Academy's Special Operations Program. Rob authored Hazardous Materials: Awareness and Operations and serves on the Fire Engineering magazine editorial advisory board, and the Fire Department Instructors Conference (FDIC) executive advisory board. Rob has developed risk assessment, incident management and hazardous materials programs for Fortune 500 companies, national laboratories and universities.

Ron VidalRon’s technology career spans 35 years as a senior executive in critical infrastructure including fiber optic and wireless telecommunications networks; data centers; electric power networks; and oil & gas facilities for Level 3 Communications, MFS Communications, UUNet Technologies and Kiewit. Ron led teams on $19 billion of M&A transactions and $14 billion of public market financings. Ron managed Level 3’s executive response in New York City after the 9/11 World Trade Center terrorist attack and previously served on Mayor Dinkins’s New York City Task Force on Network Reliability. Ron has testified before the U.S. House of Representatives Subcommittee on Technology and the Internet, as well as public utility commissions in California, New York and Massachusetts. Ron is a technical peer reviewer for FEMA’s annual $650 million Assistance to Firefighter Grant program and has served as a volunteer firefighter in four states.

Chris HawleyChris’s emergency response career spans 35 years coordinating international counter terrorism / public safety activities as a Special Operations Incident Commander, consultant and published author in Chemical, Biological, Radiological, Nuclear and Explosive (CBRNE) Hazardous Materials response for the Department of Defense, FBI, Secret Service, Baltimore County Fire Department and numerous agencies and companies. Chris co-developed the FBI’s WMD and Hazardous Materials Operations training program. Chris has authored: Hazardous Materials Incidents; Hazardous Materials Air Monitoring & Detection Devices; Hazardous Materials Response & Operations and co-authored Special Operations: For Terrorism and Hazmat Crimes. Chris is a technical consultant to numerous Fortune 500 companies and government agencies in the area of incident management and hazardous materials. Chris conducts threat assessments and on-site building vulnerability assessments for protection against terrorism threats and attacks worldwide. Chris is the executive producer of the International Association of Fire Chiefs (IAFC) International Hazardous Materials Response Team Conference, the world’s largest conference of its kind held annually in Baltimore.

18

www.blackrock3.com

Chris Hawley –

Chris@blackrock3.comRob Schnepp –

Rob@blackrock3.comRon Vidal –

Ron@blackrock3.com

San Francisco & Baltimore

19