homeland security exercise and evaluation …montanadma.org/sites/default/files/hseep...

PRE-DECISIONAL DRAFT 1

Homeland Security Exercise and Evaluation Program (HSEEP) Pre-Decisional Draft—Not For Distribution January 2012

PRE-DECISIONAL DRAFT

1

2

This page intentionally left blank. 3

PRE-DECISIONAL DRAFT i

Contents 4

INTRODUCTION AND OVERVIEW ....................................................................................... INTRO-1 5 Purpose .......................................................................................................................................... Intro-1 6 Applicability and Scope ................................................................................................................ Intro-1 7 Supersession .................................................................................................................................. Intro-1 8 How to Use This Document .......................................................................................................... Intro-2 9 Revision Process ............................................................................................................................ Intro-2 10

1. HSEEP FUNDAMENTALS ............................................................................................................. 1-1 11 Overview ............................................................................................................................................. 1-1 12 HSEEP Principles ................................................................................................................................ 1-1 13 HSEEP Methodology .......................................................................................................................... 1-2 14

Program Management ................................................................................................................... 1-2 15 Design and Development .............................................................................................................. 1-2 16 Conduct ......................................................................................................................................... 1-3 17 Evaluation ..................................................................................................................................... 1-4 18 Improvement Planning .................................................................................................................. 1-5 19

2. EXERCISE PROGRAM MANAGEMENT ................................................................................... 2-1 20 Overview ............................................................................................................................................. 2-1 21 Multiyear Planning .............................................................................................................................. 2-1 22

Exercise Program Objectives ........................................................................................................ 2-1 23 Multiyear Training and Exercise Plan (TEP) ................................................................................ 2-2 24 Exercise Types .............................................................................................................................. 2-3 25 Corrective Action Process ............................................................................................................. 2-5 26

Exercise Program Management Best Practices ................................................................................... 2-5 27 Stakeholder Engagement .............................................................................................................. 2-5 28

3. EXERCISE DESIGN AND DEVELOPMENT .............................................................................. 3-1 29 Overview ............................................................................................................................................. 3-1 30 Exercise Planning Team and Events ................................................................................................... 3-1 31

Exercise Planning Team Considerations....................................................................................... 3-1 32 Exercise Planning Team Positions ................................................................................................ 3-2 33 Planning Activities ........................................................................................................................ 3-3 34

Exercise Design ................................................................................................................................... 3-8 35 Exercise Objectives ....................................................................................................................... 3-8 36 Scope ............................................................................................................................................. 3-9 37 Scenario ...................................................................................................................................... 3-10 38 Exercise Documentation ............................................................................................................. 3-11 39 Media or Public Affairs Guidance .............................................................................................. 3-16 40

Exercise Development ....................................................................................................................... 3-17 41 Planning for Exercise Logistics .................................................................................................. 3-17 42 Planning for Exercise Control ..................................................................................................... 3-19 43 Planning for Exercise Evaluation ................................................................................................ 3-21 44

4. EXERCISE CONDUCT ................................................................................................................... 4-1 45

PRE-DECISIONAL DRAFT ii

Overview ............................................................................................................................................. 4-1 46 Preparing for Exercise Play ................................................................................................................. 4-1 47

Setup for Seminars, Workshops, Tabletop Exercises (TTXs), and Games................................... 4-1 48 Setup for Drills, Functional Exercises (FEs), and Full-Scale Exercises (FSEs) ........................... 4-1 49 Briefings ........................................................................................................................................ 4-2 50 Conducting the Exercise ............................................................................................................... 4-2 51 Conduct for Seminars, Workshops, Tabletop Exercises (TTXs), and Games .............................. 4-4 52 Conduct for Drills, Functional Exercises (FEs), and Full-Scale Exercises (FSEs)....................... 4-5 53 Contingency Process ..................................................................................................................... 4-6 54

Wrap-Up Activities ............................................................................................................................. 4-6 55

5. EVALUATION .................................................................................................................................. 5-1 56 Overview ............................................................................................................................................. 5-1 57 Plan the Exercise Evaluation ............................................................................................................... 5-1 58

Select Lead Evaluator and Evaluation Team ................................................................................ 5-1 59 Define Evaluation Requirements .................................................................................................. 5-2 60 Recruit, Assign, and Train Evaluators .......................................................................................... 5-3 61 Develop and Finalize Evaluation Documentation ........................................................................ 5-3 62 Conduct a Pre-Exercise Controller and Evaluator (C/E) Briefing ................................................ 5-4 63

Collect Exercise Data .......................................................................................................................... 5-4 64 Observing a Seminar, Workshop, Tabletop Exercise (TTX), or Game ........................................ 5-4 65 Observing a Drill, Functional Exercise (FE), or Full-Scale Exercise (FSE)................................. 5-5 66 Conducting a Player Hot Wash ..................................................................................................... 5-6 67 Collecting Supplemental Data ...................................................................................................... 5-6 68

Analyze Data ....................................................................................................................................... 5-6 69 Analyzing Data from a Tabletop Exercise (TTX) or Game .......................................................... 5-6 70 Analyzing Data from a Drill, Functional Exercise (FE), or Full-Scale Exercise (FSE) ............... 5-7 71 Develop Quick-Look Report ......................................................................................................... 5-7 72 Develop the Draft After Action Report (AAR)/Improvement Plan (IP) ....................................... 5-8 73 Summary Report Content for a Seminar or Workshop ................................................................. 5-8 74

6. IMPROVEMENT PLANNING ....................................................................................................... 6-1 75 Overview ............................................................................................................................................. 6-1 76 Conduct After Action Conference (AAC) ........................................................................................... 6-1 77 Develop Improvement Plan (IP) .......................................................................................................... 6-1 78 Finalize After Action Report (AAR) and the Improvement Plan (IP) ................................................. 6-2 79 Track Corrective Action Implementation ............................................................................................ 6-2 80

Event Points of Contact (POCs) ................................................................................................... 6-2 81 Organization Points of Contact (POCs) ........................................................................................ 6-3 82 Action Officers ............................................................................................................................. 6-3 83 Continual Improvement ................................................................................................................ 6-3 84

GLOSSARY OF TERMS AND ACRONYM LIST ................................ TERMS AND ACRONYMS-1 85

86

PRE-DECISIONAL DRAFT iii

1

2

This page intentionally left blank. 3

PRE-DECISIONAL DRAFT Intro-1

Introduction and Overview 4

Purpose 5

The Homeland Security Exercise and Evaluation Program (HSEEP) provides a set of principles for 6 exercise programs as well as a common methodology for exercise program management, design and 7 development, conduct, evaluation, and improvement planning. Exercises are a key component of national 8 preparedness—they provide whole community stakeholders with the opportunity to assess capabilities, 9 identify strengths, and address areas for improvement in a no-fault learning environment. 10

Through the use of HSEEP, exercise program managers develop exercise program objectives—informed 11 by strategy documents, threat, hazard, and capability assessments, and results from previous exercises. 12 These program-wide objectives guide the design and development for individual exercises, as planners 13 develop exercise-specific objectives and associated core capabilities for validation and assessment during 14 exercise conduct. Exercise evaluation assesses exercise performance against exercise objectives—15 documenting strengths, areas for improvement, core capability levels and gaps, and corrective actions in 16 an After Action Report/Improvement Plan (AAR/IP). Through improvement planning, exercise 17 stakeholders prioritize, track, and analyze corrective actions, ensuring that the corrective actions inform 18 future exercise program objectives. 19

In this way, the use of HSEEP—in line with Presidential Policy Directive 8 (PPD-8), the National 20 Preparedness Goal, and the National Preparedness System—supports efforts to improve national capacity 21 across the whole community to collectively build, sustain, and deliver core capabilities. 22

Applicability and Scope 23

HSEEP serves as exercise and evaluation doctrine that is flexible, scalable, and adaptable to the needs of 24 stakeholders across the whole community, including Federal, State, territorial, local, tribal governments 25 and private-sector, community, and nongovernmental organizations (NGOs). HSEEP doctrine and 26 methodology is applicable for exercises across all national preparedness mission areas—prevention, 27 protection, mitigation, response, and recovery. As exercise program guidance, HSEEP is part of the 28 National Preparedness System—an integrated set of guidance, programs, and processes enabling the 29 Nation to meet the National Preparedness Goal. 30

HSEEP doctrine is based on national best practices and is supported by training, technology systems and 31 tools, and technical assistance. The National Exercise Program (NEP) is consistent with the HSEEP 32 methodology, and exercise practitioners are encouraged to apply and adapt HSEEP doctrine to meet their 33 specific needs. 34

Supersession 35

This 2012 iteration of HSEEP supersedes the 2007 HSEEP Volumes. The current version reflects the 36 feedback, lessons learned, and best practices of the exercise community as well as current policies and 37 plans. 38

PRE-DECISIONAL DRAFT 2

How to Use This Document 39

This document serves as a description of HSEEP doctrine. It includes an overview of “HSEEP 40 fundamentals” that describes core HSEEP principles and overall methodologies. This overview is 41 followed by several chapters providing more detailed guidance to exercise practitioners on how to 42 operationalize the program’s principles and methodology. 43

The doctrine is organized as follows: 44

• Chapter 1: HSEEP Fundamentals describes the core principles and methodology of HSEEP. 45

• Chapter 2: Exercise Program Management provides guidance on developing Multiyear 46 Training and Exercise Plans (TEPs) and conducting Training and Exercise Planning Workshops. 47

• Chapter 3: Exercise Design and Development provides detailed guidance regarding planning 48 meetings and preparing participants for exercise conduct. 49

• Chapter 4: Exercise Conduct provides guidance on setup, exercise play, and demobilization 50 activities. 51

• Chapter 5: Evaluation describes exercise evaluation planning through data collection, analysis, 52 and development of an AAR/IP. 53

• Chapter 6: Improvement Planning provides guidance on addressing corrective actions from 54 exercise IPs and the implementation process of tracking corrective actions to resolution. 55

Revision Process 56

DHS/FEMA will review HSEEP doctrine and methodology on a biennial basis or as otherwise needed to 57 effect necessary modifications and incorporate lessons learned. 58

PRE-DECISIONAL DRAFT 1-1

1. HSEEP Fundamentals 59

Overview 60

HSEEP includes both core principles to frame a common approach to exercises and a detailed, proven 61 methodology for planning and executing all aspects of an exercise. This doctrine is supported by training, 62 technology systems, tools, and technical assistance and is based on national best practices. The 63 methodology presented in this document serves to enhance consistency in exercise conduct and 64 evaluation while promoting safety; ensure exercises are realistic and closely portray real-world events; 65 and remain relevant to overall national preparedness. 66

HSEEP Principles 67

HSEEP describes an integrated set of principles for managing exercises. These principles include: 68

• Common Methodology. The use of HSEEP promotes a common approach to exercises that is 69 applicable to all mission areas—prevention, protection, mitigation, response, and recovery. This 70 common approach enables organizations of divergent sizes, geographies, and capabilities to have 71 a shared understanding of exercise program management, design and development, conduct, 72 evaluation, and improvement planning. It also fosters exercise-related interoperability and 73 collaboration across jurisdictions and organizations. 74

• Informed by Risk. PPD-8 calls for systematic preparation for the threats and hazards that pose 75 the greatest risk to the security of the Nation. Risk-based analyses (e.g., Threat and Hazard 76 Identification and Risk Assessments [THIRA]) can help organizations identify and prioritize core 77 capabilities and objectives to be assessed and validated through exercises. Accordingly, using 78 risk-related information to inform the design, development, and conduct of exercises is an 79 important component of HSEEP. 80

• Progressive Planning Approach. A progressive approach to designing exercise programs is a 81 central element of HSEEP. Progressive exercise planning does not necessarily imply a linear 82 progression of exercise types. Rather, a progressive planning approach improves core capabilities 83 through a series of exercises that is anchored to a common set of objectives and involves an 84 increasing level of complexity over time. 85

• Whole Community Integration. The fostering of whole community participation—including all 86 levels of government; the private sector; nonprofit organizations; individuals (including those 87 who have access and functional needs, disabilities, or limited English proficiency); families; and 88 communities—in exercises is a central tenet of HSEEP. The use of HSEEP encourages exercise 89 planners, where appropriate, to engage the whole community throughout exercise program 90 management, design and development, conduct, evaluation, and improvement planning. 91

• Capability Based, Objective Driven. The National Preparedness Goal identifies a series of core 92 capabilities and associated objectives and targets across prevention, protection, mitigation, 93 response, and recovery mission areas. Through HSEEP, organizations can use exercises as a way 94 to assess core capability levels and gaps. Exercises focus on assessing performance against 95 capability-based strategic and operational objectives. 96


HSEEP Methodology 97

HSEEP uses a common methodology for exercises. This 98 common methodology is applicable to all national preparedness 99 mission areas and ensures a consistent and interoperable 100 approach to exercise program management, design and 101 development, conduct, evaluation, and improvement planning, 102 as depicted in Figure 1. 103

Program Management 104

Exercise program management is the act of overseeing a variety 105 of individual exercises and supporting activities sustained over 106 time. An effective exercise program helps whole community 107 stakeholders maximize efficiency, resources, time, and funding 108 by ensuring that individual exercises are part of a coordinated, 109 integrated approach to building, maintaining, and delivering 110 core capabilities. 111

Key elements of HSEEP’s approach to exercise program 112 management include: 113

• Exercise Program Objectives. These overarching objectives can inform the development of 114 exercise-specific objectives, ensuring that individual exercises build and maintain core 115 capabilities in a coordinated and integrated fashion. 116

• Multiyear Training and Exercise Plan (TEP). A TEP aligns training and exercise objectives 117 and schedules, encouraging stakeholders to coordinate efforts to validate, assess, and enhance 118 core capabilities. 119

• Corrective Action Process. A critical output of any exercise is the AAR/IP, which defines and 120 assigns corrective actions needed to address issues observed during exercise evaluation. The use 121 of HSEEP encourages organizations to track progress on all corrective actions, releasing periodic 122 reports that document progress in resolving corrective actions and highlighting those corrective 123 actions that are incomplete or behind schedule. 124

Design and Development 125

The exercise design and development process builds on exercise program management to select 126 participants for an exercise planning team, schedule planning conferences, identify and develop exercise-127 specific objectives, design the scenario, create documentation, plan exercise conduct, select a focus for 128 evaluation, and coordinate logistics. 129

Key elements of HSEEP’s approach to exercise design and development include: 130

• Exercise Planning Team. The exercise planning team is responsible for the successful execution 131 of all aspects of an individual exercise. The planning team works with exercise stakeholders to 132 determine or refine exercise objectives based on core capabilities. The exercise planning team 133 also creates a realistic scenario to achieve exercise objectives and develops documentation to 134 guide exercise conduct and evaluation. The planning team’s organization and management 135 principles should reflect those of the National Incident Management System (NIMS), with clearly 136 defined roles and responsibilities and a manageable span of control. 137

• Exercise Planning Activities. Effective exercise design and development involve a combination 138 of exercise planning activities, often in the form of planning meetings and conferences. These 139

Figure 1: HSEEP Exercise Cycle


planning activities bring together exercise stakeholders to discuss and agree on key aspects of the 140 exercise’s design and development. Various factors—including exercise scope, type, and 141 complexity—inform the types of planning activities needed, and exercise planners tailor the 142 planning schedule to suit the particular nature of the exercise. Use of video or teleconferences or 143 webinars is encouraged, where practical, as an economical means of coordination. 144

• Simple, Measurable, Achievable, Realistic, and Task-Oriented (SMART) Objectives. Well-145 defined exercise objectives provide a framework for scenario development and inform exercise 146 evaluation criteria. Exercise planners use SMART objectives in designing and developing 147 exercises. Exercise planners limit the number of exercise objectives to allow for a focused 148 evaluation. 149

• Scenario. A scenario provides the storyline that drives an exercise to validate objectives. The 150 scenario selected for an exercise is informed by the actual threats and hazards faced by the 151 exercise stakeholders. The exercise scenario should realistically stress the delivery of core 152 capabilities, providing a mechanism for testing objectives and assessing core capability levels and 153 gaps. 154

• Exercise Documentation. Thorough exercise documentation is a critical component of effective 155 exercise design and development. There is a range of exercise documentation available to 156 planners. Factors such as exercise scope, type, and complexity inform the types of documentation 157 needed. 158

• Exercise Control Planning. Exercise control maintains exercise scope, pace, and integrity 159 during conduct under safe and secure conditions. The control structure in a well-developed 160 exercise ensures that exercise play accommodates objectives in a coordinated fashion at all levels 161 and at all locations for the duration of the exercise. In the design and development phase, exercise 162 planners consider control-related issues such as the staffing, structure, training, and 163 communications of the control staff. 164

• Exercise Evaluation Planning. Exercise evaluation activities begin as soon as exercise 165 objectives are identified. In the design and development phase, exercise planners should account 166 for issues such as the staffing, structure, training, and communications of evaluation staff 167 members. 168

• Logistics Planning. Effective logistics are essential to a successful exercise. They can make the 169 difference between a smooth, seamless exercise and one that is confusing and ineffective. 170 Exercise planners consider the full range of logistics-related issues, such as venue layout, onsite 171 communications, and site security. 172

Conduct 173

After design and development activities are complete, the exercise is ready to take place. Exercise 174 conduct involves activities such as preparing for exercise play, managing exercise play, and conducting 175 immediate exercise wrap-up activities. Key elements of exercise conduct include: 176

• Exercise Play Preparation. Immediately prior to exercise play, exercise planners conduct a 177 series of activities to prepare for exercise play. These activities include finalizing venue-related 178 setup and confirming important venue- or exercise-specific logistical details. In addition, exercise 179 planners hold a series of briefings targeted to different types of exercise participants (e.g., 180 controllers, evaluators, players). These briefings are an opportunity to distribute exercise 181 documentation, provide necessary instructions and administrative information, and answer any 182 outstanding questions prior to exercise play beginning. 183


• Exercise Play. During exercise conduct, participants play various, specific roles. These roles may 184 depend on the type of exercise being conducted. The following are types of exercise participant 185 roles: 186

- Players have an active role in responding to an incident by either discussing or performing 187 their regular roles and responsibilities. 188

- Facilitators and controllers guide exercise play. During a seminar, workshop, or tabletop 189 exercise, the facilitator is responsible for ensuring that participant discussions remain focused 190 on the exercise objectives. In drills, functional exercises, and full-scale exercises, controllers 191 plan and manage exercise play, set up and operate the exercise incident site, give key data to 192 players, and may prompt or initiate certain player actions. It is recommended that all 193 controllers be accountable to one senior controller. 194

- Evaluators collect exercise-related data in order to evaluate player performance during 195 exercise play and assess whether exercise objectives were met and core capabilities were 196 validated. Evaluators are often chosen based on their expertise in the functional areas they 197 evaluate. Evaluators have a passive role in the exercise and only record the actions and 198 decisions of players; they do not interfere with exercise flow. 199

- Actors are volunteers who simulate specific roles, such as disaster casualty victims, in order 200 to add realism to an exercise. 201

- Simulators, generally controllers, perform the roles of individuals, agencies, or organizations 202 that are not actually participating in the exercise in order to drive realistic exercise play. 203

• Exercise Wrap-Up. Immediately following the end of exercise conduct, exercise planners 204 conduct a series of activities to conclude exercise conduct. Key wrap-up activities often include a 205 Player Hot Wash and various debriefing sessions in which exercise players, controllers, and 206 evaluators can discuss exercise performance and identify strengths and areas for improvement 207 from exercise play. The information from these wrap-up activities informs subsequent exercise 208 evaluation and improvement planning activities. 209

Evaluation 210

Evaluation is the cornerstone of exercises and must be considered through all phases of exercise 211 planning—beginning as soon as exercise objectives are identified. Effective evaluation assesses 212 performance against the exercise objectives, documents core capability levels and gaps, and identifies 213 strengths and areas for improvement. Key elements of the HSEEP approach to exercise evaluation 214 include: 215

• Data Analysis. HSEEP includes an objectives-driven approach to evaluating exercise 216 performance. This approach is grounded in collection and analysis of both quantitative and 217 qualitative data. During exercise play, evaluators collect a range of data—data that evaluators can 218 synthesize and analyze to assess how effectively exercise objectives were met. Effective 219 evaluation involves not merely identifying what issues emerged, but also discovering the root 220 causes of those issues. This root cause analysis enables exercise stakeholders to target how best to 221 address areas for improvement and close capability gaps. 222

• AAR/IP. The development of post-exercise AAR/IPs—documents that summarize key exercise 223 information and performance against exercise objectives and core capabilities—is a critical 224 element of evaluation within HSEEP. The IP portion of the AAR/IP includes corrective actions 225 for improvement, along with timelines for their implementation and assignment to responsible 226 parties. 227


Improvement Planning 228

During improvement planning, corrective actions from the AAR/IP—such as additional training, 229 planning, and/or equipment acquisition—are tracked to completion, ensuring that exercises yield tangible 230 preparedness improvements. A key element of the HSEEP approach to improvement planning includes: 231

• IP Tracking and the Corrective Action Program. Ongoing IP tracking and corrective action 232 processes ensure each corrective action is tracked to completion. An effective corrective action 233 program develops IPs that are dynamic documents, which are continually monitored and 234 implemented and are part of the larger system of improving preparedness.235


2. Exercise Program 236

Management 237

Overview 238

Exercise program management is the act of overseeing a variety of individual exercises and supporting 239 activities sustained over time. An effective exercise program helps entities maximize efficiency, 240 resources, time, and funding by ensuring that individual exercises are part of a coordinated, integrated 241 approach to building, maintaining, and delivering core capabilities. 242

Effective exercise program management promotes a multiyear approach to: 243

• Developing overarching program objectives; 244

• Developing a Training and Exercise Plan (TEP) that aligns exercise program objectives with 245 other preparedness system components such as training; 246

• Using a progressive approach to address program objectives through exercises of various 247 complexities; and 248

• Capturing, analyzing, and reporting corrective action data that yield improvements to core 249 capabilities. 250

Through effective exercise program management, individual exercises become supporting components of 251 a larger program with overarching objectives. Exercise practitioners are encouraged to apply and adapt 252 HSEEP doctrine on exercise program management to meet their specific needs. 253

Multiyear Planning 254

Multiyear planning allows whole community stakeholders to identify and develop a set of multiyear 255 exercise objectives informed by existing assessments, strategies, and plans. These long-term priorities and 256 objectives help exercise planners design and develop a program of individual exercises to build, maintain, 257 and validate core capabilities. Multiyear exercise planning encourages whole community stakeholders to 258 develop overarching objectives for their exercise programs and to develop and implement a Multiyear 259 TEP to guide future exercise activities. 260

Exercise Program Objectives 261

An exercise program should be guided by a set of overarching exercise program objectives linked to core 262 capabilities. These overarching objectives can inform the development of exercise-specific objectives, 263 ensuring that individual exercises build and maintain core capabilities in a coordinated and integrated 264 fashion across the whole community of stakeholders. They may also support national training and 265 exercise program guidance as described in the National Preparedness System. 266

In developing program objectives, exercise program managers should review and consider various factors 267 such as: 268

• Jurisdiction-specific threats and hazards (e.g., from a Threat and Hazard Identification and Risk 269 Assessment [THIRA]); 270

• Identified areas for improvement from real-world incidents and exercises; 271


• External requirements such as strategic plans and frameworks, emergency operations plans, 272 homeland security doctrine (e.g., the National Preparedness Goal) or those recommended by 273 industry and governmental reports; 274

• Accreditation standards or regulations. 275

Figure A-1 illustrates some of the specific factors for consideration in developing exercise program 276 objectives. 277

278 Figure A-1: Factors for Consideration in Developing Exercise Program Objectives 279

Multiyear Training and Exercise Plan (TEP) 280

Once core capabilities and overarching exercise program objectives have been identified, exercise 281 program managers can develop a Multiyear TEP. The Multiyear TEP articulates overall exercise program 282 objectives and outlines a multiyear schedule of training and exercise activities designed to meet those 283 objectives. 284

TEPs build and test capabilities by using a progressive approach of linked training and exercise activities. 285 In effective TEPs, exercises are strategically scheduled to validate training and associated capability 286 improvements. 287

Training and Exercise Planning Workshop (TEPW) 288

Stakeholders from across the whole community may sponsor, or participate in, Training and Exercise 289 Plan Workshops (TEPWs) to produce a TEP. At the TEPW, stakeholders work together in a collaborative 290 workshop environment to identify and set exercise program objectives based on core capabilities. Based 291 on these program objectives, TEPW stakeholders develop a multiyear schedule of specific training and 292 exercises. 293

Exercise program stakeholders should encourage senior leaders—such as government executives (e.g., 294 mayors, county managers); private-sector management officials; and nongovernmental organization 295 (NGO) heads—from across the whole community to actively participate in TEPWs. TEPW participants 296 should understand their entity’s capabilities, disaster and emergency preparedness strategy, improvement 297


planning action items, and training and exercise calendars. More importantly, they should have the 298 authority to commit personnel and resources to the activities scheduled in the Multiyear TEP. 299

In developing a TEP, stakeholders should remember that public law, presidential directives, or various 300 regulations may outline requirements and timelines for certain exercises. Additionally, funding from 301 Federal, State, local, or private-sector programs can have associated conditions that must be considered 302 when planning training and exercise activities. 303

TEPWs can be held on a periodic basis (e.g., annual or biennial) depending on the needs of the program. 304

Exercise Types 305

An effective exercise program uses a combination of exercise types to address program objectives. A 306 comprehensive, integrated program of exercises will often involve various exercise types, all tailored to 307 meet the objectives of the individual exercise and the program overall. 308

Seminars 309

Seminars generally orient participants to, or provide an overview of, authorities, strategies, plans, 310 policies, procedures, protocols, resources, concepts, and ideas. As a discussion-based session, seminars 311 can be valuable for entities that are developing or making major changes to existing plans or procedures. 312 Seminars can be similarly helpful when attempting to gain awareness of, or assess, the capabilities of 313 interagency or inter-jurisdictional operations. 314

Seminars do not typically result in a formal, comprehensive After Action Report (AAR)/Improvement 315 Plan (IP); however, a final report, or Summary Report, can be developed to capture the discussions; issues 316 raised; and, when appropriate, corrective actions that will address these issues. 317

Workshops 318

Although similar to seminars, workshops differ in two important aspects: participant interaction is 319 increased, and the focus is placed on achieving or building a product. Effective workshops entail the 320 broadest attendance by relevant stakeholders. 321

Products produced from a workshop can include new standard operating procedures, emergency 322 operations plans, continuity of operations plans, and mutual aid agreements. To be effective, workshops 323 should focus on a specific issue, and the desired outcome, product, or goal must be clearly defined. 324

Tabletop Exercises (TTXs) 325

A tabletop exercise (TTX) is typically held in an informal setting intended to generate discussion of 326 various issues regarding a hypothetical, simulated emergency. TTXs can be used to enhance general 327 awareness, validate plans and procedures, rehearse concepts, and/or assess the types of systems needed to 328 guide the prevention of, protection from, mitigation of, response to, and recovery from a defined incident. 329 Generally, TTXs are aimed at facilitating conceptual understanding, identifying strengths and areas for 330 improvement, and/or achieving changes in attitudes. 331

During a TTX, players are encouraged to discuss issues in depth, collaboratively examining areas of 332 concern and solving problems. The effectiveness of a TTX is derived from the energetic involvement of 333 participants and their assessment of recommended revisions to current policies, procedures, and plans. 334

TTX procedures are divided into basic and advanced categories. In a basic TTX, the scenario is presented 335 and remains constant—it describes an emergency and brings discussion participants up to the simulated 336 present time. Players apply their knowledge and skills to a list of problems presented by the facilitator or 337 moderator; problems are discussed as a group; and resolution is reached and documented for later 338 analysis. 339


In an advanced TTX, play advances as players receive pre-scripted messages that alter the original 340 scenario. The moderator usually introduces problems one at a time in the form of a written message, 341 simulated telephone call, videotape, or other means. Players discuss the issues raised by each problem, 342 referencing established authorities, plans, and procedures for guidance. Player decisions are incorporated 343 as the scenario continues to unfold. 344

During a TTX, all participants should be encouraged to contribute to the discussion and be reminded they 345 are making decisions in a no-fault environment. Effective TTX facilitation is critical to keeping 346 participants focused on exercise objectives. 347

Games 348

A game is a simulation of operations that often involves two or more teams, usually in a competitive 349 environment, using rules, data, and procedures designed to depict an actual or hypothetic situation. Games 350 explore the consequences of player decisions and actions and are therefore excellent tools to use when 351 validating or reinforcing plans and procedures or evaluating resource requirements. 352

During game play, decision-making may either be slow and deliberate or rapid and more stressful, 353 depending on the exercise design and objectives. The open, decision-based format of a game can 354 incorporate “what if” questions that expand exercise benefits. Depending on the game’s design, the 355 consequences of player actions can be either prescripted or decided dynamically. Identifying critical 356 decision-making points is a major factor in the success of games because players make their evaluated 357 moves at these crucial points. 358

Drills 359

A drill is a coordinated, supervised activity usually employed to validate a specific operation or function 360 in a single agency or organization. Drills are commonly used to provide training on new equipment, 361 validate procedures, or practice and maintain current skills. For example, drills may be appropriate for 362 establishing a community-designated disaster receiving center or shelter. Drills can also be used to 363 determine if plans can be executed as designed, to assess whether more training is required, or to 364 reinforce best practices. A drill is useful as a stand-alone tool, but a series of drills can also be used to 365 prepare several agencies and organizations to collaborate in a full-scale exercise (FSE). 366

For every drill, clearly defined plans, procedures, and protocols need to be in place. Personnel need to be 367 familiar with those plans and trained in the processes and procedures to be drilled. 368

Functional Exercises (FEs) 369

Functional exercises (FEs) are designed to validate and evaluate capabilities, multiple functions and/or 370 subfunctions, or interdependent groups of functions. FEs are typically focused on exercising plans, 371 policies, procedures, and staff members involved in management, direction, command, and control 372 functions. In FEs, events are projected through an exercise scenario with event updates that drive activity 373 at the management level. An FE is conducted in a realistic, real-time environment; however, movement of 374 personnel and equipment is usually simulated. 375

Response- and recovery-focused FEs are generally focused on exercising the plans, policies, procedures, 376 and protocols, and staffs of the direction and control branches of the Incident Command System (ICS) 377 and Unified Command, or multiagency coordination centers (e.g., Emergency Operations Centers). 378

A prevention-focused FE generally concentrates on exercising the plans, policies, procedures, 379 agreements, networks, and staffs of law enforcement intelligence centers or agencies with 380 counterterrorism missions. Adversary actions are largely simulated and delivered in the form of shared 381 intelligence; however, some adversary actions may be carried out by simulated adversaries (red teams) in 382 a separate but coordinated category of exercise play. 383


FE controllers typically use a Master Scenario Events List to ensure participant behavior remains within 384 predefined boundaries. Simulators in a Simulation Cell can inject scenario elements to simulate real 385 events. 386

Full-Scale Exercises (FSEs) 387

FSEs are typically the most complex and resource-intensive type of exercise. They involve multiple 388 agencies, organizations, and jurisdictions and validate many facets of preparedness. FSEs often include 389 many players operating under cooperative systems such as the ICS or Unified Command. 390

In an FSE, events are projected through an exercise scenario with event updates that drive activity at the 391 operational level. FSEs are usually conducted in a real-time, stressful environment intended to mirror a 392 real incident. Personnel and resources may be mobilized and deployed to the scene where actions would 393 be conducted as if a real incident had occurred. The FSE simulates reality by presenting complex and 394 realistic problems that require critical thinking, rapid problem solving, and effective responses by trained 395 personnel. 396

The level of support needed to conduct an FSE is greater than that needed for other types of exercises. 397 The exercise site for an FSE is usually large, and site logistics require close monitoring. Safety issues, 398 particularly regarding the use of props and special effects, must be monitored. Throughout the duration of 399 the exercise, many activities occur simultaneously. 400

Corrective Action Process 401

After execution of an exercise, exercise stakeholders develop an AAR/IP, which articulates specific 402 corrective actions to remedy identified areas for improvement. An AAR/IP assigns all corrective actions 403 to a responsible person or organization and includes performance metrics and deadlines for completion. 404

As part of a multiyear approach to exercise program management, an exercise program manager should 405 track progress on all resulting corrective actions identified in exercise program AAR/IPs. The exercise 406 program manager can develop and release periodic reports that document progress toward completing 407 corrective actions and that highlight corrective actions that are incomplete or behind schedule. Long-term 408 tracking of corrective actions enables exercise stakeholders to identify how exercise activities are closing 409 core capability gaps and improving preparedness. 410

Improvement Planning Workshop (IPW) 411

An Improvement Planning Workshop (IPW) provides a forum to identify training and exercise priorities 412 based on a review of corrective action items from exercises and real-world events. IPW activities can 413 inform TEPW activities or can be integrated into the TEPW. 414

IPW activities typically include identifying and reviewing strengths and areas for improvement from 415 previous real-world events and exercises through a review and analysis of AAR/IP data. Building on 416 those identified strengths and areas for improvement, IPWs allow stakeholders to identify and discuss 417 potential training and exercise priorities. 418

Exercise Program Management Best Practices 419

Exercise program managers should consider the following best practices when developing and 420 implementing exercise programs. 421

Stakeholder Engagement 422

Exercise program managers should identify as wide a range of exercise program stakeholders as possible 423 and seek to create a database cataloging stakeholder points of contact. This database can include contact 424 information, areas of expertise, and prior exercise experience. 425


When identifying stakeholders, exercise program managers should consider individuals and organizations 426 from throughout the whole community that would be involved in an actual incident or event, including 427 but not limited to: 428

• Individuals with administrative responsibility relevant to exercise conduct (e.g., Federal, 429 Regional, State, territorial, local, and tribal agency and private-sector procurement officials); 430

• Representatives from first responder disciplines to be included in exercises; 431

• Emergency Support Function representatives; 432

• Representatives from volunteer, nongovernmental, or nonprofit organizations, such as Citizen 433 Corps Councils and the American Red Cross; 434

• Social support organizations and advocacy groups working with children and adults with and 435 without disabilities who have access and functional needs; 436

• Federal, State, territorial, local, tribal, private-sector, and nongovernment officials who affect, or 437 are affected by, exercise activities; and 438

• Senior leaders or those responsible for providing resources to support exercises. 439

Once a comprehensive set of stakeholders has been identified, exercise program managers can integrate 440 them into the exercise program by having them regularly participate in TEPWs. Program managers 441 attempting to build a new exercise program may begin by hosting an exercise working group involving 442 representatives from all stakeholder entities. 443

To engage stakeholders and secure their support for exercise activities, exercise program managers can 444 develop a stakeholder communications plan. This plan should contain clearly defined communications 445 objectives (e.g., to coordinate exercise efforts, to solicit feedback) along with timeframes and methods for 446 regular communication. An organization’s Federal and/or State reporting obligations can also be 447 addressed as part of an effective communications plan. 448

Resource Management 449

An effective exercise program must efficiently utilize the full range of available resources. 450

Exercise Budget Management 451

Effective budget management is essential to the success of an exercise program, and it is important for 452 exercise managers to maintain awareness of their available resources and expected expenditures through a 453 program budget. In developing and maintaining an exercise program budget, program managers should 454 work with the full range of financial stakeholders to identify financial management requirements and 455 define monitoring and reporting requirements as required by the exercise director. 456

Program Staffing 457

Exercise program staffing needs are determined largely by the activities called for in the Multiyear TEP. 458 Program managers should identify the administrative and operational staff needed to oversee the exercise 459 program. Program managers should also identify gaps between staffing availability and staffing needs. 460 Exercise program managers can consider alternative means of procuring staff members, such as adding 461 volunteers, students from universities (e.g., student nurses or emergency management students), or 462 interns. 463

Other Resources 464

Exercise program managers should also consider other resources that can support exercises. Such 465 resources include equipment (e.g., smoke machines); training courses; exercise tools and resources (e.g. 466 document templates); materials from previous exercises; mutual aid agreements, memoranda of 467


understanding and memoranda of agreement; technical assistance; and information technology (e.g., 468 HSEEP Toolkit, modeling and simulation capabilities). A number of organizations and Web sites, such as 469 the National Emergency Management Agency, National Governors Association, and the Lessons Learned 470 Information Sharing System (LLIS.gov) house resources that can facilitate exercise activities. 471

472


3. Exercise Design and 473

Development 474

Overview 475

The design and development process builds on exercise program strategy to identify and develop 476 objectives, develop an exercise planning timeline with milestones, select participants for an exercise 477 planning team, schedule planning conferences, design the scenario, create documentation, coordinate 478 logistics, plan exercise conduct, and select a focus for evaluation. 479

This document is intended to provide exercise practitioners with an overview of key exercise design and 480 development concepts, methodologies, and planning considerations, all grounded in national best 481 practices. Exercise practitioners are encouraged to apply and adapt the program’s doctrine on exercise 482 design and development to meet their specific needs. 483

Exercise Planning Team and Events 484

Exercise Planning Team Considerations 485

The exercise planning team manages, and is ultimately responsible for, exercise design, development, 486 conduct, and evaluation. The team determines exercise objectives based on the capabilities selected to be 487 assessed; creates a realistic scenario to test the exercise objectives; and develops supporting 488 documentation, processes, and systems used in evaluation, control, and simulation. Planning team 489 members also help with developing and distributing pre-exercise materials and conducting exercise 490 planning conferences, briefings, and training sessions. 491

The exercise planning team should be of manageable size yet represent the full range of participating 492 organizations as well as other relevant stakeholders. For multijurisdictional exercises, planning team 493 members should include representatives from each jurisdiction and participating functional area or 494 relevant disciplines. The membership of an exercise planning team should be modified to fit the type or 495 scope of an exercise, which varies depending on exercise type and complexity. An exercise planning team 496 leader manages the planning team. To design and develop exercises most effectively, exercise planning 497 teams should: 498

• Adhere to a clear organizational structure, with a distinct chain of command, roles and 499 responsibilities, and accountability ending in the exercise planning team leader; 500

• Use proven management practices, processes, and tools, such as project plans and timelines, 501 status reports, and other communications; 502

• Clearly identify and understand the desired objectives for the exercise and design and develop the 503 exercise accordingly; 504

• Use subject-matter experts (SMEs) to develop a realistic and challenging scenario; and 505

• Follow a standardized exercise design and development process. 506

Support agencies/organizations working with children and adults with and without disabilities who have 507 access and functional needs must also be included from the beginning of the planning process to ensure 508 that issues facing this population are included and addressed during the exercise. In doing so, exercise 509 planners can better understand their perspectives and promote early understanding of roles and 510


responsibilities and planning assumptions to identify potential problems, inaccuracies, inconsistencies, 511 omissions, and questionable assumptions. 512

Generally, planning team members are not exercise players. However, for cases in which resources are 513 limited, exercise planning team members who act as both planners and players should be especially 514 careful not to divulge exercise information in advance. 515

Exercise Planning Team Positions 516

Regardless of the scale and complexity of an exercise, the exercise planning team can be most effective if 517 it adheres to a coherent organization structure that clearly delineates roles and responsibilities. In 518 developing a structure for the planning team, exercise planners should use Incident Command System 519 principles, as stated in the National Incident Management System. This structure can be scaled up or 520 down to reflect the scope of the exercise and the available resources and personnel of the participating 521 organizations, and, depending on available resources, the same personnel can be used to execute multiple 522 functions. This structure may include the following: 523

• Command Section. The Command Section is responsible for coordinating all exercise planning 524 activities. Within the Command Section is the exercise planning team leader, who assigns 525 exercise activities and responsibilities, provides guidance, establishes timelines, and monitors the 526 development process. 527

• Operations Section. The Operations Section provides most of the technical or functional 528 expertise for both scenario development and evaluation. This includes development of the Master 529 Scenario Event List (MSEL). 530

• Planning Section. The Planning Section is responsible for compiling and developing all exercise 531 documentation. To accomplish this effectively, the Planning Section also collects and reviews 532 policies, plans, and procedures that will be assessed in the exercise. This group is also responsible 533 for planning exercise evaluation. During the exercise, the Planning Section may be responsible 534 for developing simulated actions by agencies not participating in the exercise and for setting up a 535 Simulation Cell (SimCell) as required. 536

• Logistics Section. The Logistics Section provides the supplies, materials, facilities, and services 537 that enable the exercise to function smoothly without outside interference or disruption. This 538 section consists of two subsections: service and support. The service subsection provides 539 transportation, barricading, signage, food and drinks, real-life medical capability, and exercise 540 security. The support subsection provides communications, purchasing, general supplies, 541 management of very important persons (VIPs) and observer processing, and recruitment and 542 management of actors. 543

• Administration/Finance Section. The Administration/Finance Section provides financial 544 management and administrative support throughout exercise development, including exercise 545 registration support and scheduling. Figure B-1 shows a sample exercise planning team 546 incorporating these principles. 547


548 Figure B-1: Sample Exercise Planning Team 549

Planning Activities 550

This section describes the types of planning activities—often in the form planning meetings or 551 conferences—most useful in exercise design and development. The exercise planning team members 552 decide the type and number of planning activities needed to successfully plan a given exercise, based on 553 its scope and relative complexity. Meeting and exercise site locations should take into consideration those 554 individuals who require assistance or accommodations during their attendance. Example timelines for 555 scheduling exercise planning activities can be found online (https://hseep.dhs.gov). 556

Concept and Objectives (C&O) Conference 557

Primary Focus 558

A Concept and Objectives (C&O) Conference is the formal beginning of the planning process. It is held 559 to identify the type, scope, objectives, and purpose of the exercise. For less complex exercises and for 560 organizations with limited resources, the C&O Conference can be conducted in conjunction with the 561 Initial Planning Conference (IPC). 562

Representatives from the sponsoring agency or organization, representatives from potentially 563 participating organizations, the exercise planning team leader, and senior officials typically attend the 564 C&O Conference. The C&O Conference helps planners identify the capabilities and tasks that are going 565 to be assessed, design objectives based on those capabilities and tasks, and identify exercise planning 566 team members. 567

Discussion Points 568

Topics or issues generally covered during a C&O Conference include the following: 569

• Exercise scope and purpose; 570

• Proposed capabilities, tasks, and objectives; 571

• Available exercise resources; 572

• Proposed exercise location, date, and duration; 573

• Exercise planning team; 574

• Anticipated extent of play for exercise participants; 575

• Exercise assumptions and artificialities; 576

• Exercise control and evaluation concepts; 577

https://hseep.dhs.gov/�


• Exercise security organization and structure; 578

• Local issues, concerns, and sensitivities; 579

• Exercise logistics; and 580

• Exercise planning timeline and milestones. 581

Tools 582

The primary tools for the C&O Conference are an agenda and the background and rationale for 583 conducting the exercise. Briefings are useful for presenting the exercise background and rationale as well 584 as exercise methodology for persons unfamiliar with HSEEP. 585

Outcomes 586

The following outcomes are expected from the C&O Conference: 587

• Agreement regarding exercise concept (type, preparedness mission area, capabilities to be 588

assessed) and exercise objectives; 589

• Consensus on the target exercise timeframe and the date of the next planning conference; 590

• Anticipated extent of participation; 591

• Identification of exercise planning team members; and 592

• An exercise planning timeline with milestones. 593

Initial Planning Conference (IPC) 594

Primary Focus 595

The IPC marks the beginning of the exercise development phase, and, regardless of whether a C&O 596 Conference is held, an IPC should be conducted for all exercises. Its purpose is to determine exercise 597 scope by gathering input from the exercise planning team; design requirements and conditions (e.g., 598 assumptions and artificialities); objectives; extent of play; and scenario variables (e.g., time, location, 599 hazard selection). The IPC is also used to develop exercise documentation by obtaining the planning 600 team’s input on exercise location, schedule, duration, and other relevant details. 601

During the IPC, exercise planning team members are assigned responsibility for activities associated with 602 designing and developing exercise documents—such as the Exercise Plan (ExPlan) and the Situation 603 Manual (SitMan)—and logistics. 604


Topics or issues generally covered during an IPC include the following: 606

• Ensuring clearly defined and measurable capabilities, tasks, and objectives; 607

• Developing the exercise scenario; 608

• Incorporating all relevant plans, policies, and procedures into exercise design; 609

• Identifying local issues, concerns, or sensitivities; 610

• Determining the extent of play for each participating entity by establishing what they will 611

demonstrate and be evaluated on at the exercise; 612

• Ensuring that exercise planners understand their roles and responsibilities for the exercise; 613


• Choosing subjects for photographs and/or audio/visual (A/V) recordings to incorporate into 614

exercise documents and multimedia presentations (to enhance realism); 615

• Deciding whether or not to record exercise proceedings (audio or video); 616

• Determining the optimum duration of the exercise; 617

• Selecting or customizing the appropriate Exercise Evaluation Guides (EEGs) to determine 618

whether exercise capabilities and objectives were achieved and to allow participants to provide 619

feedback (if a Midterm Planning Conference [MPC] is held, then the selection and customization 620

of EEGs can be accomplished at that time); 621

• Any discussion points listed in the C&O Conference section above if a C&O Conference was not 622

conducted; 623

• Reaching a consensus regarding the date, time, and location for the next conference; and 624

• Modeling and simulation planning. 625

Tools 626

The primary tools for the IPC are the read-ahead packet, agenda, capabilities and tasks from the EEGs (if 627 an MPC will not be conducted), hazard information (if applicable), a proposed room layout (if 628 applicable), and the exercise planning timeline with milestones. A briefing is useful for presenting an 629 overview of the exercise to the planning team. 630

Outcomes 631

The following outcomes are expected from the IPC: 632

• A refined exercise planning timeline with milestones; 633

• Clearly defined, obtainable, and measurable capabilities, tasks, and objectives; 634

• Identified exercise scenario variables (e.g., threat scenario, scope of hazard, venue, conditions); 635

• A list of participating exercise organizations and anticipated organizational extent of play; 636

• Identification and availability of SMEs and presenters, as necessary, for scenario vetting and/or 637

expert evaluation; 638

• Determination of preferred communication methods among the exercise planning team; 639

• Availability of all source documents (e.g., policies, plans, procedures) needed to draft exercise 640

documents and presentations; 641

• Clearly identified and assigned responsibility for exercise logistical issues; 642

• Draft SitMan or ExPlan; 643

• A list of tasks to be accomplished by the next planning conference with established dates for 644

completion and responsible planning team members identified; 645

• Any outcomes listed in the C&O Conference section above if a C&O Conference was not 646

conducted; and 647


• An agreed-upon date, time, and location for the next conference and the actual exercise. 648

Midterm Planning Conference (MPC) 649

MPCs are typically used in more complex exercises such as functional exercises (FEs) and full-scale 650 exercises (FSEs). MPCs provide additional opportunities to settle logistical and organizational issues that 651 may arise during planning. 652

Primary Focus 653

The MPC is a meeting to discuss exercise organization and staffing concepts, scenario and timeline 654 development, scheduling, logistics, and administrative requirements. It is also held to review draft 655 documentation. If only three planning conferences are scheduled (i.e., IPC, MPC, and Final Planning 656 Conference [FPC]), a portion of the MPC should be devoted to developing the Multiple Scenario Events 657 List (MSEL). See the next section, MSEL Conference, for more information. 658


Possible topics or issues for an MPC include the following: 660

• Comments on draft ExPlan; 661

• Identification of exercise venue artificialities and/or limitations; 662

• Agreement on final logistical items; 663

• Assignment of additional responsibilities; and 664

• Construction of the scenario timeline—usually the MSEL—if an additional MSEL Planning 665

Conference will not be held. 666

Tools 667

MPC tools include, but are not limited to, an agenda; IPC minutes; draft scenario timeline; draft 668 documentation (e.g., ExPlan, Controller/Evaluator [C/E] Handbook); and other selected documentation 669 needed to illustrate exercise concepts and provide planning guidance. 670

Outcomes 671

The following outcomes are expected from the MPC: 672

• A fully reviewed exercise scenario timeline, which is typically the MSEL (if an additional MSEL 673

Conference will not be held); 674

• Fully reviewed ExPlan; 675

• Draft C/E Handbook; 676

• Well-developed scenario injects (imperative if an additional MSEL Planning Conference is not 677

scheduled); 678

• Agreement on the exercise site; and 679

• Finalization of date, time, and location of the MSEL Planning Conference and/or FPC. 680


Master Scenario Events List (MSEL) Conference 681

For more complex exercises, one or two additional planning conferences—or MSEL Conferences—may 682 be held to review the scenario timeline. If not held separately, topics typically covered in a separate 683 MSEL Conference can be incorporated into the MPC and FPC. 684

Primary Focus 685

The MSEL Conference focuses on developing the MSEL. The MSEL is a chronological list that 686 supplements the exercise scenario with event synopses; expected participant responses; capabilities, tasks, 687 and objectives to be addressed; and responsible personnel. It includes specific scenario events (or injects) 688 that prompt players to implement the plans, policies, procedures, and protocols that require testing during 689 the exercise, as identified in the capabilities-based planning process. It also records the methods that will 690 be used to provide the injects (e.g., phone call, radio call, e-mail). 691


In developing a MSEL, the exercise planning team must first consider the tasks, conditions, and standards 693 set forth by each exercise objective. A condition is the environment in which a task is performed—it can 694 be provided by the scenario or through the MSEL. 695

• If scenario conditions do not stimulate performance of the appropriate task, the exercise planning 696 team must develop a MSEL entry to simulate the desired situation. A well-written entry considers 697 the following questions: 698

• Is the event key (i.e., is it directly related to meeting an exercise objective)? 699

• What is the desired task? Who will demonstrate the task? 700

• What will stimulate the behavior (e.g., course of play, phone call, actor, video)? 701

• Who originates the stimulant? Who receives it and how? 702

• What action is the player expected to complete? 703

• Should a contingency entry be developed for injection into the exercise in case the players fail to 704 demonstrate the task? 705

Tools 706

MSEL Conference tools include, but are not limited to, previous planning conference minutes, draft 707 exercise documentation, and an agreed-upon MSEL template. 708

Outcomes 709

Following a MSEL Conference, the level of MSEL completion may vary. At a minimum, key events and 710 the time of their delivery are identified, and responsibility for constructing the remaining events is 711 assigned. 712

Final Planning Conference (FPC) 713

Primary Focus 714

The FPC is the final forum for reviewing exercise processes and procedures. An FPC should be 715 conducted for all exercises to ensure that all elements of the exercise are ready for conduct. Prior to the 716 FPC, the exercise planning team receives final drafts of all exercise materials. No major changes to the 717 exercise’s design, scope, or supporting documentation should take place at or following the FPC. The 718 FPC ensures that all logistical requirements have been met, outstanding issues have been identified and 719 resolved, and exercise products are ready for printing. 720



The following items are addressed during the FPC: 722

• Resolve any open exercise planning issues and identify potential last-minute concerns. 723

• Review all exercise logistical activities (e.g., schedule, registration, attire, special needs). 724

• Conduct a comprehensive, final review of—and approve—all remaining draft exercise documents 725 (e.g., SitMan, MSEL, C/E Handbook) and presentation materials. 726

Tools 727

The primary tools for the FPC include IPC and/or MPC minutes, an agenda, and previously finalized 728 and/or drafted exercise documents. 729

Outcomes 730

The FPC should not generate any significant changes. The following outcomes are expected: 731

• Attendees have a clear understanding of—and give final approval for—exercise processes and 732 procedures. 733

• Exercise documents and materials for production are approved. 734

• Last-minute issues are identified and resolved. 735

• Logistical elements, including equipment, facilities, and schedule, are confirmed. 736

Follow-Up 737

The exercise planning team finalizes all publications, prepares all supporting materials, rehearses 738 presentations and briefings, and prepares to conduct the exercise. Prior to the exercise, documentation and 739 any additional instructions should be disseminated to the appropriate personnel (e.g., presenters, 740 facilitators, controllers, evaluators, simulators). 741

Exercise Design 742

The exercise planning conferences serve as the principal mechanism for executing the major steps of 743 exercise design. These core components of design include setting exercise objectives, establishing the 744 scope of the exercise, creating an exercise scenario, developing exercise documentation, and determining 745 media and public relations guidance. 746

Exercise Objectives 747

The cornerstone of exercise design is the selection of capabilities to be assessed and determination of 748 exercise objectives based on those capabilities. The exercise planning team determines core capabilities 749 and objectives based on the overall exercise program objectives. The objectives should reflect the 750 exercise sponsor’s specific needs, environment, plans, and procedures, while providing a framework for 751 scenario development and a basis for evaluation. 752

Planners should create objectives that are simple, measurable, attainable, realistic, and task-oriented 753 (SMART). Figure B-2 depicts guidelines for developing SMART objectives. 754


SMART Guidelines for Useful Objectives

Simple A good objective is simply and clearly phrased. It is brief and easy to understand and is usually one sentence long.

Measurable The objective should set the level of performance so that the results are observable and one can tell when the objective has been achieved. This does not mean that a quantifiable standard has to be set. It means there can be agreement on whether the players succeeded.

Achievable The objective should not be too difficult to achieve, and achieving it should be within the resources that the organization is able to commit to an exercise.

Realistic The objective should present a realistic expectation for the situation. Even though an objective might be achievable, it might not be realistic for the exercise.

Task Oriented

The objective should focus on a behavior or procedure. With respect to exercise design, each objective should focus on a capability.

Figure B-2: SMART Objectives Guidelines 755

Generally, planners should limit the number of exercise objectives to enable timely exercise conduct, 756 facilitate reasonable scenario design, and support successful evaluation. Capabilities, tasks, and objectives 757 are initially prepared during a C&O Conference or IPC. For seminars, workshops, tabletop exercises 758 (TTXs), and games, objectives typically focus on strategic, policy-oriented issues; whereas for drills, FEs, 759 and FSEs, objectives typically focus on operational issues. 760

Scope 761

Determining exercise scope enables planners to “right-size” an exercise to meet the objectives while 762 staying within the resource and personnel constraints of the exercising organizations. Key elements in 763 defining exercise scope include exercise type, participation level, exercise duration, exercise location, and 764 exercise parameters. Some of these elements are determined, or initially discussed, through program 765 management activities. However, the exercise planning team finalizes the scope based on the individual 766 exercise objectives. Alterations to the scope are reviewed with the exercise objectives in mind; planners 767 must consider whether a change in the scope will improve or impede the ability of players to meet the 768 objectives. 769

Exercise Type 770

A first step in defining exercise scope is determining what exercise type to conduct. The exercise type is 771 selected based on the purpose of the exercise. If the intent is to validate a new policy, plan, or set of 772 procedures, a TTX may be appropriate. If the intent is to validate the responders’ knowledge of a plan, 773 policy, or set of procedures, an FE or FSE may be appropriate. 774

Participation Level 775

Active participation by appropriate entities and key leaders is paramount to meeting the exercise 776 objectives successfully. Participation level refers to which organizations will play in the exercise; what 777 level of personnel (e.g., tactical operators, line supervisors, agency directors) will play in the exercise; and 778 the general number of personnel who will play in the exercise. 779

At times, scheduling conflicts, real-world events, or other competing requirements will limit an 780 organization’s or key players’ ability to participate in an exercise. In this case, exercise designers will 781 need to simulate the decisions and actions of those participants through an exercise SimCell. Extent of 782 Play Agreements (XPAs) are a good way to define this level of participation. 783


Exercise Duration 784

When selecting the exercise duration, the planning team must determine how long it will take to address 785 the exercise objectives effectively. Seminars, workshops, TTXs, games, and drills are generally shorter, 786 ranging from a couple of hours to a full day. FEs and FSEs may take longer. Often, prevention-focused 787 FEs that validate Fusion Centers and other intelligence operators will last up to 30 days with limited 788 duration of play each day. Resource constraints, including the opportunity cost of having employees away 789 from their primary roles, should be factored into determining duration. 790

Exercise Parameters 791

Establishing exercise parameters assists planners in identifying what should be included in an exercise 792 scenario based on the objectives and scope and what should not be exercised. Often there is a desire to 793 add exercise activities that fall outside of the scope of the exercise to meet diverse planning and training 794 requirements. While these activities may be useful to a jurisdiction, they may impact the ability of players 795 to meet exercise objectives or may reduce the benefit of the exercise by diluting its focus. Clearly 796 defining the exercise scope early in the design process will help exercise planners keep the exercise to a 797 manageable and realistic level. 798

Scenario 799

A scenario is an outline or model of the simulated sequence of events for the exercise. It can be written as 800 a narrative or depicted by an event timeline. For seminars, workshops, TTXs, and games, a scenario 801 provides the backdrop that drives participant discussion, and it is contained in a SitMan. For drills, FEs, 802 and FSEs, a scenario provides background information about the incident catalyst(s) of the exercise—the 803 overall scenario is provided in the C/E Handbook, and specific scenario events are contained in the 804 MSEL. 805

Exercise planners should select and develop scenarios that enable an exercise to meet its capabilities and 806 objectives. All scenarios should be realistic, plausible, and challenging; however, designers must ensure 807 the scenario is not so complicated that it overwhelms players. 808

A scenario consists of three basic elements: (1) the general context or comprehensive story; (2) the 809 required conditions that will allow players to demonstrate proficiency and competency in meeting the 810 exercise capabilities and objectives; and (3) the technical details necessary to accurately depict scenario 811 conditions and events. The exercise planning team ensures that the design effort is not characterized by a 812 fixation on scenario development—rather, the scenario facilitates assessment of exercise capabilities and 813 objectives. Because of this, exercise planners should refrain from developing the scenario until after the 814 objectives and scope of the exercise have been clearly defined. Furthermore, scenarios should be 815 constructed to avoid any sensitivities that may arise, such as the use of real names of terrorist groups or 816 sensitive venues. 817

Threat or Hazard 818

The first step in designing a scenario is determining the type of threat or hazard on which the exercise will 819 focus. Each type of emergency has its own strengths and weaknesses when it comes to evaluating 820 different aspects of prevention, protection, mitigation, response, and recovery. The exercise planning 821 team should choose a threat or hazard that best assesses the capabilities and objectives on which the 822 exercise will focus. The identification of this threat or hazard scenario should also be based on the entity’s 823 risk based analysis (e.g. Threat and Hazard Identification and Risk Assessment [THIRAs]). 824

Modeling and Simulation 825

When incorporated into the development of the scenario and overall exercise design, modeling and 826 simulation can bring versatility, cost savings, and fidelity to exercises. A model is a representation of a 827 system at a point in time or space intended to expand an understanding of the real system. Simulation is a 828


method of implementing the performance of a model, or combination of models, over time. Modeling and 829 simulation supports decision-making processes by providing human and/or computer feedback to players 830 during exercise play, thus dynamically representing the impact of their decisions. For example, human-831 based simulation during exercises is often manifested through the SimCell, which represents 832 nonparticipating entities. An example of a computer-based simulation could include wind damage and 833 storm surge forecasting models developed by the National Oceanic and Atmospheric Administration, 834 which enable simulation of the effects a hurricane may have on coastal communities. 835

Modeling and simulation can also be applied in situations where reality cannot be achieved. For example, 836 for safety reasons a bioterrorism exercise cannot be conducted by releasing a deadly virus into the 837 environment. However, it is still important to exercise the capabilities necessary to respond to this type of 838 scenario. The use of modeling and simulation can realistically replicate variables such as disease 839 propagation, radiation, and chemical attacks. 840

Exercise Documentation 841

Comprehensive, organized exercise documentation is critical to ensuring that an accurate account of the 842 exercise can be preserved. This in turn allows organizations to leverage past documentation to support 843 future exercises and, more importantly, ensures that all critical issues, lessons learned, and corrective 844 actions are appropriately captured to support improvement efforts. 845

While most exercise materials are not sensitive or classified, some materials (e.g., scenario details) may 846 necessitate restrictions on distribution. It is important for the exercise sponsor(s) to understand their 847 specific requirements for security marking rules and requirements, access and dissemination, storage, 848 disposal, and incident reporting of sensitive documents. 849

Consideration should also be given to the accessibility of presentations and documents, such as making 850 information available in alternative formats (e.g. large print, compact disc, Braille); closed captioning or 851 another form of text display; or the provision of sign language interpreters. 852

Controller and Evaluator (C/E) Handbook 853

The C/E Handbook specifically describes the roles and responsibilities of exercise controllers and 854 evaluators and the procedures they should follow. Because the C/E Handbook contains information about 855 the scenario and about exercise administration, it is distributed to only those individuals specifically 856 designated as controllers or evaluators. The C/E Handbook supplements the Exercise Plan (ExPlan) and 857 contains more detailed information about the scenario. It points readers to the ExPlan for more general 858 exercise information, such as participant lists, activity schedules, required briefings, and the roles and 859 responsibilities of specific participants. 860

The C/E Handbook usually contains the following sections: 861

• Detailed scenario information; 862

• Assignments, roles, and responsibilities of group or individual controllers and evaluators; 863

• Exercise safety plan; 864

• Controller communications plan (e.g., a phone list, a call-down tree, instructions for the use of 865 radio channels); and 866

• Evaluation instructions. 867

For larger, more complex exercises, planners may develop a written Evaluation Plan (EvalPlan) in lieu of, 868 or in addition to, a C/E Handbook. Like the C/E Handbook, an EvalPlan supplements the ExPlan by 869 providing evaluation staff members with guidance and instructions on evaluation or observation 870 methodology to be used as well as essential materials required to execute their specific functions. 871


Likewise, Control Staff Instructions (COSIN) may be used in lieu of a C/E Handbook for larger, more 872 complex exercises that require more coordination among the control staff. The COSIN contains guidance 873 that controllers, simulators, and evaluators need concerning procedures and responsibilities for exercise 874 control, simulation, and support. In addition to the functions of a C/E Handbook, the COSIN provides 875 guidelines for control and simulation support and establishes a management structure for these activities. 876

Controller and Evaluator (C/E) Packets 877

While C/E Handbooks contain detailed information that should be read and understood well in advance of 878 the exercise, Controller Packets and Evaluator Packets are provided immediately prior to an exercise to 879 controllers and evaluators respectively. The packets contain key information from the C/E Handbook and 880 additional information specific to the functional area in which the given controller or evaluator will be 881 working. This information is needed during exercise play in order to carry out control and evaluation 882 responsibilities. 883

Both Controller Packets and Evaluator Packets should contain the following: 884

• Essential C/E Handbook information; 885

• MSEL, including injects and events for each responsible controller and evaluator; 886

• Appropriate Exercise Evaluation Guides (EEGs); and 887

• Maps and directions. 888

Master Scenario Events List (MSEL) 889

A MSEL is typically used in drills, FEs, and FSEs and contains a chronological listing of the events that 890 drive exercise play. Each MSEL entry should contain the following at a minimum: 891

• Designated scenario time; 892

• Event synopsis; 893

• Controller responsible for delivering the inject, with controller or evaluator special instructions (if 894 applicable); 895

• Expected action (i.e., player response expected after a MSEL inject is delivered); 896

• Intended player (i.e., agency or individual player for whom the MSEL event is intended); 897

• Capability, task, or objective to be demonstrated (if applicable); and 898

• Notes section (for controllers and evaluators to track actual events against those listed in the 899 MSEL, with special instructions for individual controllers and evaluators). 900

Times listed in a MSEL should be as realistic as possible and should be based on input from SMEs. If the 901 activity occurs sooner than the MSEL writers anticipated, then controllers and evaluators should note the 902 time it occurred, but play should not be interrupted. 903

Controllers delivering MSEL injects will either be co-located with players in the venue of play, or they 904 will reside in a SimCell. A SimCell is a location from which controllers deliver messages representing 905 actions, activities, and conversations of an individual, agency, or organization that is not participating in 906 the exercise but would likely be actively involved during a real incident. Prior to the start of the exercise 907 (StartEx), the mechanisms for introducing injects into exercise play should be tested to ensure that 908 controllers are aware of the procedures for delivering MSEL injects and that any systems that will be used 909 to deliver them are functioning properly as planned. 910

The three types of descriptive MSEL events that support exercise play include: 911


1. Contextual injects introduced to a player by a controller help build the exercise operating 912 environment and/or keep exercise play moving. For example, if the exercise is designed to test 913 information-sharing capabilities, a MSEL inject can be developed to direct an actor to portray a 914 suspect by behaving suspiciously in front of a law enforcement player. 915

2. Expected action events reserve a place in the MSEL timeline and notify controllers when a 916 response action would typically take place. For example, during an FSE involving a chemical 917 agent, establishing decontamination is an expected action that the players will take without the 918 prompting of an inject. 919

3. Contingency injects are provided by a controller to players to ensure play moves forward to 920 adequately evaluate performance of activities. For example, if a simulated secondary device is 921 placed at an incident scene during a terrorism response exercise, but is not discovered, a 922 controller may want to prompt an actor to approach a player to say that he or she witnessed 923 suspicious activity close to the device location. This should prompt the responder to discover the 924 device and result in subsequent execution of the desired notification procedures. 925

MSELs are typically produced in long formats, short formats, or both. Short-form MSELs usually list 926 injects in a single row in a spreadsheet format. These can be used as a quick-reference guide during 927 exercise play or projected onto a large screen in a control cell or SimCell. Long-form MSELs are used 928 when greater detail is necessary; they include more detailed descriptions, exact scripting language for 929 actors and simulators, and more detailed descriptions of expected actions. 930

Exercise Evaluation Guides (EEGs) 931

The HSEEP offers a series of tools includes EEGs to help evaluators collect and interpret relevant 932 exercise observations. These documents are aligned to a core capability and applicable performance 933 objective, providing guidance on assessing capabilities being validated by the exercise. Each EEG 934 provides evaluators with information on what they should expect to see demonstrated or hear discussed, 935 space to record their observations, and criteria to consider after the exercise. 936

Situation Manual (SitMan) 937

SitMans are provided for TTXs and games as the core documentation that provides the textual 938 background for a facilitated exercise. The SitMan supports the scenario narrative and serves as the 939 primary reference material for all participants during conduct. 940

The SitMan’s introduction provides an overview of the exercise—including scope, capabilities and 941 objectives, structure, rules, and conduct—as well as an exercise agenda. The next section of the SitMan is 942 the scenario, which may be divided up into distinct, chronologically sequenced modules. Each module 943 represents a specific time segment of the overall scenario, based on exercise objectives and scenario 944 requirements. 945

Each module is followed by discussion questions, usually divided by organization or discipline. 946 Responses to the modules’ discussion questions are the focus of the exercise, and reviewing them 947 provides the basis for evaluating exercise results. These discussion questions can be derived from the 948 capability and associated performance objectives found within each EEG. 949

The SitMan generally includes the following information: 950

• Introduction; 951

• Schedule of events; 952

• Exercise purpose, scope, capabilities, tasks, and objectives; 953

• Exercise structure (i.e., order of the modules); 954


• Instructions for exercise players and observers; 955

• Exercise assumptions and artificialities; 956

• Exercise rules; 957

• Exercise scenario background (including scenario location information); 958

• Discussion questions and key issues; and 959

• Reference appendices with relevant supporting information, which may include, but not be 960 limited to: 961

- Entity- and threat-specific information; 962 - Material Safety Data Sheet or agent fact sheet, when applicable; 963 - Relevant documents regarding plans, standard operating procedures (SOPs), etc.; and 964 - A list of reference terms. 965

Facilitator’s Guide 966

A Facilitator’s Guide is designed to help facilitators manage a seminar, workshop, TTX, game, or 967 planning meeting. It usually outlines instructions and key issues for discussion during the event and 968 provides background information to help the facilitator answer questions from participants or players. 969

Multimedia Presentation 970

Multimedia presentations are often used to illustrate the general scenario for participants. They are given 971 at StartEx and support the SitMan. The presentation should concisely summarize information contained in 972 the written documentation. Like the SitMan, the multimedia presentation is also divided into distinct, 973 chronologically segmented modules that, when combined, create the entire scenario. 974

This presentation typically contains, at a minimum, the following information: 975

• Introduction; 976

• Background or history on the threat and the scenario; 977

• Exercise capabilities, tasks, and objectives; 978

• Exercise play rules and administrative information; and 979

• Modules that describe the scenario. 980

The presentations are intended to help focus and drive the exercise as well as add realism. A/V 981 enhancements to a presentation include video or sounds that convey information to participants. 982

Extent of Play Agreements (XPAs) 983

XPAs can be used during the development of exercise objectives to define the exercise scope and the 984 subject of the evaluation. These agreements are formed between exercise participants and can be vital to 985 the planning of an exercise, recruitment of evaluators, and development of support requirements. They 986 define which organizations will participate in the exercise as well as their extent of play (e.g., one 987 firehouse for 8 hours, county Emergency Operations Center activated at level A for 24/7 exercise 988 operations). 989

Exercise Plan (ExPlan) 990

ExPlans are general information documents that help drills, FEs, and FSEs run smoothly by providing 991 participants with a synopsis of the exercise. They are published and distributed to the participating 992 organizations following development of most of the critical elements of the exercise. In addition to 993 addressing exercise objectives and scope, ExPlans assign activities and responsibilities for exercise 994


planning, conduct, and evaluation. The ExPlan is intended to be seen by the exercise players and 995 observers—therefore, it does not contain detailed scenario information that may reduce the realism of the 996 exercise. Players and observers should review all elements of the ExPlan prior to exercise participation. 997

An ExPlan typically contains the following sections: 998

• Administrative handling instructions and security designation; 999

• Purpose, scope, and objectives; 1000

• Duration, date, and time of exercise and schedule of events; 1001

• Exercise planning team and control staff organization description; 1002

• Roles and responsibilities; 1003

• Rules of conduct; 1004

• Safety issues, notably real emergency codes and phrases, safety controller responsibilities, 1005 prohibited activities, and weapons policies; 1006

• Logistics; 1007

• Security of and access to the exercise site; 1008

• Communications (e.g., radio frequencies or channels); and 1009

• Maps and directions. 1010

Player Handout 1011

The Player Handout provides key information to exercise players. A Player Handout can supplement the 1012 SitMan or ExPlan by providing a quick-reference guide to logistics, agenda or schedule, and key contact 1013 data for players. 1014

Participant Feedback Form 1015

Players and observers receive a Participant Feedback Form after the end of the exercise that asks for input 1016 regarding observed strengths and areas for improvement that players identified during the exercise. 1017 Providing Participant Feedback Forms to players during the exercise Hot Wash allows them to provide 1018 evaluators with their insights into decisions made and actions taken. A Participant Feedback Form also 1019 provides players the opportunity to provide constructive criticism about the design, control, or logistics of 1020 the exercise to help enhance the planning of future exercises. At a minimum, the questions on this 1021 feedback form solicit the following: 1022

• Strengths of player actions or policies, plans, and SOPs; 1023

• Areas for improvement identified in player actions or policies, plans, and SOPs; and 1024

• Impressions about exercise conduct and logistics. 1025

Information collected from feedback forms contributes to the issues, observations, recommendations, and 1026 corrective actions in the AAR/IP. Feedback forms can be supplemented by the conduct of a Hot Wash 1027 immediately following the exercise, during which facilitators, controllers, and evaluators capture player 1028 perspectives on the key strengths and areas for improvement identified during the exercise. 1029

Waiver Forms 1030

Each victim actor should receive a waiver form prior to the day of the exercise. Signing this form waives 1031 liability for all exercise planners and participants. Exercising entities should use discretion when 1032 recruiting actors under the age of 18 because of additional challenges and concerns related to liability. If 1033


the exercise requires volunteers younger than age 18, parents or legal guardians should sign their waiver 1034 forms. 1035

Weapons and Safety Policy 1036

All exercises, where applicable, should employ a written weapon and safety policy that has been 1037 approved by the senior officials of the participating organizations prior to exercise conduct. For the 1038 purpose of this policy, a weapon includes all firearms, knives, explosive devices, less-than-lethal weapons 1039 or devices, and any other object capable of causing bodily harm. Specifically included in this policy are 1040 defensive aerosols (i.e., Mace®, pepper sprays, tear gas); batons; kubatons; and Tasers®. 1041

Qualified personnel (e.g., law enforcement, security, military) with legal authority to carry weapons who 1042 have an assigned exercise role (e.g., responder, tactical team) and have the potential for interaction with 1043 other exercise participants will NOT carry a loaded weapon within the confines of the exercise play area. 1044

Qualified personnel (e.g., law enforcement, security, military) with legal authority to carry and use 1045 weapons who are utilized to provide real-world security for the exercise may continue to carry loaded 1046 weapons as part of their normal scope of duty. Personnel providing real-world security for the exercise 1047 may not participate in any exercise activities. Real-world security personnel should not directly interact 1048 with participants except for the specific purpose of enforcing the laws which they are authorized to 1049 enforce or to protect themselves or those in their presence from imminent harm. 1050

Media or Public Affairs Guidance 1051

Members of the media have the unique ability to fulfill an important function before, during, and after an 1052 exercise. Prior to an exercise, they inform the public that an exercise will take place, and they make the 1053 public aware that the community is preparing for disasters. During an exercise, they can facilitate the 1054 validation of public information plans and procedures. Following an exercise, the media may release 1055 details to the host community on the state of its preparedness, if the exercise planning team leader 1056 provides such information. Therefore, exercise sponsors should work to incorporate media-related issues 1057 into exercise planning. 1058

Press Release 1059

Prior to an exercise, the exercise planning team should develop a written press release to disseminate to 1060 media outlets. This release informs the media and the public about general exercise information. 1061 Additionally, this information can be distributed to observers, senior officials, and other VIPs. This 1062 release should not contain detailed scenario information, such as the type of threat or hazard, nor should it 1063 contain information that might hinder exercise outcomes if a participant were to see it. 1064

Typically, the contents of a media or public information release for an exercise include the following: 1065

• Introduction, including sponsor and exercise program information; 1066

• Purpose and expected outcomes; 1067

• Scope and duration; 1068

• General scenario information (e.g., site/venue, goals, objectives); and 1069

• Participating agencies or disciplines. 1070

Public Announcement 1071

Public announcements should be made prior to any exercise involving public space or that will be 1072 viewable by the public. This precaution helps avoid confusion on the part of the public. It will also help 1073 the public avoid congestion near the exercise site by providing suggestions for alternate routes. 1074


Announcements can be made through local media, through mass mailings or pamphlets, and/or on signs 1075 near the exercise site. 1076

Media Policy 1077

The agency or organization sponsoring the exercise should decide whether to invite media representatives 1078 to the exercise. If invited, media representatives should have an opportunity prior to the exercise to 1079 conduct interviews with key planners and participants. 1080

At seminars, workshops, TTXs, or games, media representatives should not be present during the 1081 discussion of any potentially sensitive information, and filming exercise conduct should be avoided so as 1082 not to inhibit or hinder discussion or the flow of play. 1083

During drills, FEs, or FSEs, media representatives may be allowed to film certain activities but should be 1084 cautioned not to interfere with exercise play or film any sensitive operations. Unless media 1085 representatives are invited to participate in the exercise, a guide—typically a public information officer or 1086 designee—should escort media representatives at all times. If mock media representatives—exercise 1087 controllers simulating the real-world media—are employed during an exercise to test public affairs 1088 training, they should be kept completely separate from any real-world media representatives who may be 1089 observing the exercise. 1090

Exercise Development 1091

Exercise development involves planning for the critical elements of exercise conduct: logistics, control, 1092 and evaluation. 1093

Planning for Exercise Logistics 1094

Logistical details are important, but often overlooked, aspects of an exercise. They can make the 1095 difference between a smooth, seamless exercise and one that is confusing or even unsafe. 1096

Venue 1097

Facility and Room 1098

Meetings, briefings, and exercises should be conducted in facilities that are appropriate for the exercise 1099 scope and attendance. Facilities should be reserved solely for exercise purposes and should be accessible 1100 to all participants and free from distractions. 1101

When selecting a facility and room for exercise planning or conduct, planners should account for the 1102 following considerations: 1103

• Ensuring there are enough tables and chairs for every relevant participant; 1104

• Having a table arrangement best suited to the meeting or exercise (e.g., a U-shaped layout for 1105 exercises requiring facilitation and participant interaction); 1106

• Selecting a facility with room acoustics that facilitate ease of discussion; and 1107

• Accessibility of parking and restrooms for all participants, including those with and without 1108 disabilities who have access and functional needs. 1109

Audio/Visual (A/V) Requirements 1110

A/V requirements are identified during the design phase including individuals assigned to ensure 1111 equipment is properly functioning. 1112

Supplies, Food, and Refreshments 1113


Exercise planners should not assume participants will bring necessary supplies with them. Writing 1114 utensils, notepads, easels, copies of plans and procedures to be discussed, name badges, and any other 1115 equipment deemed necessary should be procured prior to exercise conduct and provided to participants. 1116

Food and refreshments should also be provided to participants and observers. For tabletop exercises, 1117 seminars, and workshops, it is often beneficial to have a working lunch provided to minimize disruption 1118 to play. For drills, functional exercises, and full-scale exercises, hydration of participants is an important 1119 consideration. 1120

Badging and Identification 1121

For security purposes, all exercise participants should wear some form of identification. Although some 1122 players may wear their uniforms, badges are typically used to identify each exercise participant by name 1123 and organization. Where appropriate, name tents should be placed on tables prior to StartEx to ensure 1124 proper seating arrangements. Additionally, each table should have a table tent identifying the organization 1125 or functional area seated at that table. 1126

Registration and Table/Breakout Identification 1127

Participants register upon arrival, for both identification and security reasons. Each participant should, at 1128 minimum, provide their name, organization, telephone number, and e-mail address. The exercise planning 1129 team retains copies of the sign-in sheets, so that participants can receive follow-up correspondence such 1130 as thank-you notes, certificates of completion, copies of the AAR/IP, and invitations to future planning 1131 meetings and exercises. 1132

Victim Actors 1133

Volunteer victim actors provide added realism and prompt players to provide simulated victim care. 1134 Exercise planning team members can recruit them from local colleges and universities, drama clubs, 1135 theaters, civic groups, emergency response academies, medical classes, and Federal and State military 1136 units. Consideration should be given to soliciting volunteer victim actors from within the functional 1137 and/or access needs population to provide an opportunity to practice meeting the needs of these 1138 individuals in a disaster situation. 1139

Prior to the exercise, victim actors should receive the following: 1140

• Actor instructions including information on when to arrive, where to report, and other logistical 1141 details; 1142

• Symptomatology cards containing the signs and symptoms the actor will portray as well as 1143 information for medical providers; and 1144

• Waiver forms clearing liability for exercise planners and participants. 1145

Parking, Transportation, and Designated Areas 1146

Established parking areas should be clearly labeled for use by observers, media, actors, controllers, 1147 evaluators, and players arriving in personal vehicles. If required, law enforcement personnel should be 1148 available to help direct vehicles to proper parking areas. 1149

Drills, functional exercises, and full-scale exercises may also have several key areas for exercise conduct, 1150 including: 1151

• Assembly Area. This is a gathering place for all deployable resources that will be playing in an 1152 exercise. The purpose of the assembly area is to gather all resources and personnel near the 1153 exercise site prior to StartEx for safety briefings, weapons checks, and to ensure that resources 1154 and personnel are transported in a safe and unhurried manner. 1155


• Operations Area. This is a large space where tactical operations—such as decontamination, 1156 triage, or render-safe procedures—take place. 1157

• Response Route. This is the path traveled by responding emergency units from the Assembly 1158 Area to the exercise site during a response-focused exercise. 1159

• Observer/Media Area. This is a designated area that provides observers and real-world media 1160 representatives with a view of the exercise but prevents them from interfering with exercise play. 1161

Planning for Exercise Control 1162

Exercise control maintains exercise scope, pace, and integrity during conduct under safe and secure 1163 conditions. Key elements of exercise control include controller staffing, structure, training, 1164 communications, and safety and security. 1165

Staffing 1166

The planning team identifies how many controllers will be required during exercise conduct to deliver 1167 and track information. As a guiding principle, at least one controller is at every venue whenever possible. 1168 In addition to controlling the flow of information and release of MSEL events, a controller at every site 1169 helps ensure the exercise is conducted safely with proper security controls. 1170

The control staff during a TTX is the facilitator. If participants divide into groups for part of conduct, a 1171 table facilitator is assigned to each group. A complex multijurisdictional FSE, on the other hand, may 1172 require hundreds of controllers at field and headquarters play sites as well as additional controllers in 1173 control cells to coordinate among the various play sites or serve in a SimCell. Resource constraints may 1174 make placing a controller at every site challenging. Multitasking personnel to serve as both a controller 1175 and an evaluator can help. While not desirable, exercise planners may also assign selected players to serve 1176 as controllers. Such players/controllers would need to understand clearly how to separate the roles to 1177 avoid feeding advance information into play or otherwise harming exercise integrity. 1178

Structure and Simulation Cell (SimCell) 1179

The control structure is the framework that allows controllers to communicate and coordinate with other 1180 controllers at other play sites or at a control cell to deliver and track exercise information. For TTXs, 1181 games, and small drills, the structure is usually minimal. For FEs and FSEs, however, the control 1182 structure may need to be fairly substantial to allow for proper coordination. 1183

In an exercise involving field and headquarters play among multiple organizations in one location, a 1184 control cell serves as a central node for sharing information among controllers at the various sites and for 1185 putting all of the information together to form a common exercise picture. If an exercise contains multiple 1186 jurisdictions, particularly multiple levels of government in different geographic locations, it may be 1187 beneficial to establish multiple venue control cells (VCCs) that communicate and coordinate with each 1188 other through a master control cell. When an exercise does require establishment of multiple control cells, 1189 it is important to define their roles and relationships, including their decisionmaking hierarchy. 1190

When developing a control cell, exercise planners should consider their resource environment. Ideally, a 1191 control cell will contain a point of contact or a liaison representing each participating organization. In 1192 exercises involving a mix of classified and unclassified information, it may be required to separate control 1193 cells, with appropriate security firewalls set up to handle classified and unclassified information. 1194 Moreover, if an exercise uses a SimCell to drive exercise play, a determination needs to be made how to 1195 staff and integrate it into the broader control structure. 1196

A SimCell is used to generate injects for, receive player responses for, and provide information in place 1197 of nonparticipating organizations that would likely participate actively if exercise events were real. 1198 Physically, the SimCell is a working location for a number of qualified professionals who portray these 1199


nonparticipating organizations. These professionals are knowledgeable of the organizations they are 1200 portraying, and they deliver MSEL injects in a realistic fashion. Depending on the type of exercise, the 1201 SimCell may require a telephone, computer, e-mail account, radios, or other means of communication. 1202

Controller Training 1203

If all exercise controllers can be recruited from the exercise planning team, there is little need to develop 1204 and provide special training for controllers. If, however, controllers are recruited from participating 1205 entities or other sources outside the planning team, it is very beneficial to provide some level of advance 1206 training to ensure that controllers understand the exercise, their role in it, and what they need to do. 1207

The training generally includes a basic primer on the exercise design and all of the developed aspects of 1208 exercise control, including the scenario, information delivery methods, control staff, structure, and 1209 communications plan. Controllers are also trained to use the documents (e.g., MSEL) and the facilities 1210 (e.g., SimCell) that will help them control the exercise. 1211

Communications Plan 1212

The best-designed exercise control structure staffed by the most experienced exercise practitioners will 1213 fail if controllers cannot communicate effectively and efficiently. A communications section in the C/E 1214 Handbook or COSIN serves as a communications plan by telling controllers who to communicate with, 1215 what they need to communicate, and how they will communicate. This communications section may 1216 include: 1217

• Controller Communications. Controllers at field or headquarters play sites may need to 1218 communicate with controllers at other sites or only with a control cell. Control cells will need to 1219 be able to communicate with all controllers at field or headquarters play sites, internally, and with 1220 other control cells if appropriate. Controllers and control cells may also need to communicate 1221 directly with players through means other than face-to-face interaction. 1222

• Timing and Content of Communications. While controllers should communicate exercise 1223 events as they occur, establishing a communications battle rhythm with defined information 1224 requirements will help to ensure effective information flow. 1225

• Communications Methodology. Communications may occur by phone, radio, e-mail, over a 1226 networked system, or a mix. Controllers and control cells will need to be equipped to use the 1227 designated method(s) of communication. 1228

Safety and Security 1229

Controllers also play an important role in ensuring that the exercise is conducted safely in a secure 1230 environment. In exercises involving potentially dangerous field play or the use of classified materials, the 1231 control team designates a safety and/or security controller(s) to focus on those areas of control. 1232

Safety 1233

Safety is the most important consideration in planning any exercise. For these types of exercises, the 1234 following actions can help ensure a safe environment: 1235

• Identify a safety controller(s). 1236

• Dedicate non-exercise participating advanced life support or basic life support ambulance unit(s) 1237 for real-world emergencies that may occur during the exercise. 1238

• Identify real-world emergency procedures with a code word or phrase. 1239

• Identify safety requirements and policies. 1240


• Consider other safety issues outside the scope of exercise control (e.g., weather, heat stress, 1241 hypothermia, fire/pyrotechnics, weapons, animals/K-9s, use of force, use of personal protective 1242 equipment). 1243

Security 1244

Because of the sensitive nature of many exercises, it is important for the exercise site to be secure. Local 1245 law enforcement should provide site security where appropriate. Exercises often also involve sensitive or 1246 classified information or procedures. For all exercises involving sensitive or classified information, 1247 exercise planners should identify and adhere to appropriate security standards to ensure that this 1248 information is not compromised. Such measures can include conducting registration prior to a TTX, 1249 ensuring that uninvited or uncleared individuals do not participate, or having law enforcement or security 1250 guards monitor and control access to a play site for the duration of the exercise. 1251

Planning for Exercise Evaluation 1252

Thorough planning and organization prior to an exercise is imperative to effective and successful exercise 1253 evaluation. This process should include: 1254

• Appointing a lead evaluator; 1255

• Defining evaluation requirements based on exercise objectives; 1256

• Recruiting, training, and assigning evaluators; 1257

• Finalizing the evaluation sections of the C/E Handbook or EvalPlan; and 1258

• Conducting a C/E Briefing. 1259

Through this process, an evaluation team can organize itself appropriately and develop a comprehensive 1260 plan to address how the exercise will be evaluated. 1261


4. Exercise Conduct 1262

Overview 1263

Exercise conduct involves activities such as preparing for exercise play, managing exercise play, and 1264 conducting immediate exercise wrap-up activities. For a seminar, workshop, tabletop exercise (TTX), or 1265 game, conduct also entails presentation, facilitation, and discussion. For a drill, functional exercise (FE), 1266 or full-scale exercise (FSE), conduct encompasses all operations occurring between the designated start of 1267 the exercise (StartEx) and end of the exercise (EndEx). Core elements of exercise conduct include 1268 preparing for exercise play, conducting the exercise, and wrap-up. Exercise practitioners are encouraged 1269 to apply and adapt HSEEP doctrine on exercise conduct to meet their specific needs. 1270

Preparing for Exercise Play 1271

Setup for Seminars, Workshops, Tabletop Exercises (TTXs), and Games 1272

Members of the exercise planning team assigned to set up should visit the exercise site at least 1 day prior 1273 to the event to arrange the room, test audio/video (A/V) equipment, and discuss administrative and 1274 logistical issues. On the day of the exercise, planning team members should arrive several hours before 1275 StartEx to handle setup activities and arrange for registration. 1276

Prior to exercise conduct, the exercise planning team must deliver the necessary exercise materials and 1277 equipment, which may include the following: 1278

• Situation Manuals (SitMans) or other written materials for exercise participants; 1279

• Multimedia presentation; 1280

• Appropriate A/V equipment including televisions, projectors, projection screens, microphones, 1281 and speakers; 1282

• Table tents for each table; 1283

• Name tents for each participant; 1284

• Badges identifying the role of each exercise participant; 1285

• Sign-in sheets; and 1286

• Participant Feedback Forms. 1287

Setup for Drills, Functional Exercises (FEs), and Full-Scale Exercises (FSEs) 1288

The appropriate exercise planning team members should arrange and begin event setup as many days 1289 prior to the event as necessary, depending on the scope of the simulated environment. Setup entails 1290 arranging briefing rooms and testing A/V equipment, placing props and effects, marking the appropriate 1291 areas and their perimeters, and checking for potential safety issues. On the day of the exercise, all exercise 1292 planning team members should arrive several hours before StartEx to handle any remaining logistical or 1293 administrative items pertaining to setup, arrange for registration, and conduct a communications check. 1294


Briefings 1295

Held before an exercise, briefings educate participants about their roles and responsibilities. By 1296 scheduling separate briefings for controllers and evaluators, actors, and players, exercise planning team 1297 members can avoid giving extraneous material to different groups. 1298

Player Briefing 1299

Shortly before StartEx, a controller conducts a briefing for all players to address individual roles and 1300 responsibilities, exercise parameters, safety, security badges, and any remaining logistical exercise 1301 concerns or questions. Following the exercise, controllers ensure that appropriate players attend the post-1302 exercise Hot Wash in their respective functional area. 1303

Controller/Evaluator (C/E) Briefing 1304

The C/E Briefing is generally conducted before a drill, FE, or FSE. It begins with an exercise overview 1305 and then reviews location and area, schedule of events, scenario, control concept, controller and evaluator 1306 responsibilities, instructions on completing Exercise Evaluation Guides (EEGs), and any miscellaneous 1307 information. 1308

Victim Actor Briefing 1309

The Actor Briefing should be conducted before the exercise, prior to the actors taking their positions. The 1310 victim actor controller leads this briefing and includes the following information: exercise overview, 1311 safety, real emergency procedures, symptomatology, acting instructions, and schedule. Identification 1312 badges and symptomatology cards are distributed before or during this briefing. 1313

Observer Briefing 1314

An Observer Briefing is generally conducted the day of an exercise and informs observers and very 1315 important persons (VIPs) about program background, scenario, schedule of events, observer limitations, 1316 and any other miscellaneous information. Often, observers will be unfamiliar with public safety 1317 procedures and will have questions about the activities they see. Designating someone, such as a public 1318 information officer, to answer questions prevents observers from asking questions of players, controllers, 1319 or evaluators. Participant Handouts and Exercise Plans (ExPlans) or SitMans, depending on the type of 1320 exercise being conducted, are often distributed during this briefing. 1321

Conducting the Exercise 1322

The following table defines the exercise roles generally associated with exercise conduct. The table also 1323 identifies which exercise types the role generally participates in. 1324

Role Definition Exercise

Type

Exercise Director

The exercise director oversees all exercise functions during exercise conduct; oversees and remains in contact with controllers and evaluators; debriefs controllers and evaluators following the exercise; and oversees setup and cleanup of the exercise as well as positioning of controllers and evaluators.

All

Facilitator During a seminar, workshop, tabletop exercise (TTX), or game, the facilitator(s) is responsible for keeping participant discussions on track with exercise objectives and ensuring all issues and objectives are explored as thoroughly as possible within time constraints. If an exercise uses breakout groups, more than one facilitator may be needed.

Seminar, Workshop, TTX, Game



Type

Evaluator Evaluators, selected from participating agencies, are chosen based on their expertise in the functional areas they will observe. Evaluators use Exercise Evaluation Guides (EEGs) to measure and assess performance, capture unresolved issues, and analyze exercise results. Evaluators should passively assess and document players’ performance against established emergency plans and exercise evaluation criteria without interfering in exercise flow.

All

Lead Evaluator

The lead evaluator should participate fully as a member of the exercise planning team and should be a senior-level individual familiar with all relevant issues associated with the exercise, including plans, policies, and procedures of the exercising entity; Incident Command and decisionmaking processes of the exercising entity; and interagency and/or interjurisdictional coordination issues relevant to the exercise. The lead evaluator must have the management skills needed to oversee a team of evaluators over an extended process as well as the knowledge and analytical skills to undertake a thorough and accurate analysis of all capabilities being tested during an exercise.

All

Controller In a game, drill, functional exercise (FE), or full-scale exercise (FSE), controllers plan and manage exercise play, set up and operate the exercise incident site, and possibly take the roles of individuals and agencies not actually participating in the exercise. Controllers direct the pace of exercise play, provide key data to players, and may prompt or initiate certain player actions and injects to the players as described in the Master Scenario Events List (MSEL) to ensure exercise continuity. Controllers issue exercise materials to players as required, monitor the exercise timeline, and monitor the safety of all exercise participants. Controllers are the only participants who should provide information or direction to players. All controllers should be accountable to one senior controller.

Game, Drill, FE, FSE

Senior Controller

The senior controller is responsible for the overall organization of the exercise. The senior controller monitors exercise progress and coordinates decisions regarding deviations or significant changes to the scenario caused by unexpected developments during play. The senior controller monitors actions by controllers and ensures they implement designated and modified actions at the appropriate time. The senior controller debriefs controllers and evaluators after the exercise and oversees the setup and takedown of the exercise.

Game, Drill, FE, FSE

Safety Controller

The safety controller is responsible for monitoring exercise safety during exercise setup, conduct, and cleanup. All exercise controllers assist the safety controller by reporting any safety concerns. The safety controller should not be confused with the safety officer, who is identified by the incident commander during exercise play.

Drill, FE, FSE

Assembly Area Controller

The assembly area controller is responsible for the logistical organization of the assembly area, including placement locations for units entering the assembly area, the release of dispatched units into the field, and coordination of pathways and overall safety within the assembly area.

FSE



Type

Simulator

Simulators are control staff personnel who role play as nonparticipating organizations or individuals. They most often operate out of the Simulation Cell (SimCell), but they may occasionally have face-to-face contact with players. Simulators function semi-independently under the supervision of SimCell controllers, enacting roles in accordance with instructions provided in the MSEL. All simulators are ultimately accountable to the exercise director and senior controller.

Drill, FE, FSE

Observer Observers do not directly participate in the exercise; rather, they observe selected segments of the exercise as it unfolds, while remaining separated from player activities. Observers view the exercise from a designated observation area and are asked to remain within the observation area during the exercise. A dedicated controller or public information officer should be assigned to manage these groups. In a seminar, workshop, TTX, or game, observers may support the development of player responses to the situation during the discussion by delivering messages or citing references.

All

Player Players have an active role in preventing, responding to, or recovering from the risks and hazards presented in the scenario, by either discussing or performing their regular roles and responsibilities. Players initiate actions that will respond to and/or mitigate the simulated emergency.

All

Actor Actors are typically volunteer personnel responsible for simulating a specific role in an exercise. Actors are vital to creating a realistic scenario and can play a variety of roles.

Drill, FSE

Conduct for Seminars, Workshops, Tabletop Exercises (TTXs), and Games 1325

Multimedia Presentation 1326

The multimedia presentation is a crucial vehicle for conveying information to the players. The 1327 presentation typically starts with brief remarks by representatives from the exercise planning team or 1328 sponsoring organization, and/or senior officials from the governing jurisdiction. After the opening 1329 remarks, the presentation moves into a brief introductory and explanatory phase led by a facilitator. 1330 During this phase, attendees will be introduced to any other facilitators, controllers, or evaluators; given 1331 background on the exercise process; and be advised about their individual roles and responsibilities. 1332

The facilitator generally presents the multimedia briefing, which describes the scenario and any relevant 1333 background information. The facilitator also leads the discussion, introduces spokespersons, poses 1334 questions to the audience, and ensures that the schedule remains on track. 1335

Facilitated Discussion 1336

Facilitated group discussions occur at individual tables organized by discipline or agency/organization, 1337 and ideally, someone with functional area expertise facilitates them. The facilitator is responsible for 1338 keeping the discussion on track with the exercise objectives and making sure all issues are explored 1339 within the time allotted. A good facilitator should possess: 1340

• The ability to keep side conversations to a minimum, keep discussions on track and within 1341 established time limits, control group dynamics and strong personalities, and speak competently 1342 and confidently about the subject without dominating the conversation; 1343

• Functional area expertise or experience; 1344


• Awareness of appropriate plans and procedures; and 1345

• The ability to listen well and summarize player discussions. 1346

If feasible and/or appropriate, co-facilitators who are knowledgeable about local issues, plans, and 1347 procedures may assist the lead facilitator. Also, designating a recorder to take notes allows the facilitator 1348 to focus on key discussion issues. 1349

Moderated Discussion 1350

Moderated discussions generally follow breakout facilitated discussions. In moderated discussions, a 1351 representative from each group presents all participants with summarized results from a group’s 1352 facilitated discussion. This spokesperson is selected before the facilitated discussion so that he or she can 1353 prepare to speak on behalf of the group. During moderated discussions, spokespersons summarize the 1354 facilitated discussion, present key findings and issues, and discuss any unresolved issues or questions. At 1355 the end of the moderated discussion period, the floor becomes open for questions. 1356

Time for both facilitated and moderated discussion is generally scheduled at the end of each module, with 1357 another, longer period for each at the conclusion of the exercise. During the moderated discussion, groups 1358 should focus only on the material presented in a given module. 1359

Exercise Data Collection 1360

During the exercise, evaluators use EEGs to record data for critical topics and subjects that the lead has 1361 assigned them to assess. During seminars, workshops, TTXs, and games, facilitators help evaluators 1362 collect useful data by keeping discussions focused on capabilities and activities relevant to the questions 1363 provided in the EEGs. 1364

Conduct for Drills, Functional Exercises (FEs), and Full-Scale Exercises (FSEs) 1365

During conduct of drills, FEs, and FSEs, the exercise planning team leader normally serves as the senior 1366 controller or exercise director. This person is responsible for commencing exercise play by announcing 1367 StartEx. 1368

The exercise area for an FE is limited to the control or command centers and their onsite staff members. 1369 All other activity and deployment of resources outside of these locations is notional and is simulated by 1370 the SimCell control staff. All exercise communications going in and out of these exercise locations must 1371 either originate or terminate at the SimCell. This is done to ensure that no accidental deployment of 1372 resources occurs. To accomplish this, players should be supplied with an exercise directory that provides 1373 contact information for each of the simulated organizations, which will be portrayed by simulators in the 1374 SimCell. 1375

To prevent unintended deployments, all exercise communications must be easily identifiable as such. This 1376 can be accomplished by displaying the phrase “exercise material only” prominently on all typed or 1377 printed communications, and by beginning each verbal communication by stating, “This is an exercise.” 1378

Exercise play rules should also be disseminated to establish the parameters that participants must follow. 1379 These rules help players understand their roles in the exercise environment, enabling the tasks they 1380 perform to be effectively evaluated. These rules also describe appropriate behavior, establish guidelines 1381 for physical contact, and aim to prevent physical harm to individuals or damage to property. Written rules 1382 must be reviewed and approved by appropriate authorities and then provided in advance to all parties. 1383

All exercise operations should be performed in the operational area. Controllers and evaluators report key 1384 activities to the senior controller. The senior controller announces EndEx, either at the conclusion of the 1385 scenario, after a certain period of time has passed, or when he or she has determined that all exercise 1386 objectives have been met. 1387


Control 1388

Because of the great deal of simulated activity that occurs during FEs, these exercises require a robust and 1389 detailed MSEL and close communication between the site controller(s) and the SimCell. Site controllers 1390 should advise the SimCell on the pace of exercise play, and request more or fewer injects as necessary to 1391 maintain an appropriate pace. 1392

During an FSE, the assembly area controller remains in close communication with other controllers 1393 throughout the exercise to ensure safe and realistic dispatch of units. The assembly area controller is 1394 responsible for its logistical organization, including placement locations for units and coordination of 1395 exiting patterns for dispatched units. It is imperative for the exercise planning team to create a 1396 deployment timetable based on realistic response times. Failure to do so will result in a compromised and 1397 disorganized exercise. 1398

The assembly area controller must be informed about any updates to the exercise that may require 1399 changes to the deployment timetable. Should such changes be required, the assembly area controller 1400 would update the deployment timetable. When the most current information is used, appropriate units are 1401 dispatched and arrive on schedule, allowing the remainder of the exercise to proceed smoothly and 1402 realistically. 1403

The assembly area controller “takes attendance” to ensure all players are present when a unit arrives at the 1404 assembly area. Units are positioned according to their deployment times, and qualified individuals 1405 perform a weapons check to guarantee the tagging of all inspected weapons to indicate they are safe for 1406 exercise play. 1407

Exercise Data Collection 1408

During the exercise, each evaluator should use the EEGs to record both quantitative and qualitative data 1409 for capabilities and their associated performance measures as assigned by the lead evaluator. During 1410 drills, FEs, or FSEs, evaluators should be strategically pre-positioned in locations at which they can 1411 gather useful data, and they should track and record participant actions carefully. 1412

Contingency Process 1413

In order to prevent jeopardizing mission performance in response to real-world events, the exercise 1414 planning team should maintain a contingency process to halt, postpone, or cancel an exercise as 1415 necessary. Should the execution of the exercise put at risk any efforts to respond to real-world events or 1416 should real-world events severely degrade the ability to conduct the exercise, the exercise planning team 1417 should convene, in coordination with leadership from participating organizations, to determine the 1418 appropriate course of action. Following decision on a final course of action, the Exercise Director should 1419 communicate that course of action to all exercise planners, participants, and other key stakeholders 1420 through all relevant communications mechanisms. 1421

Wrap-Up Activities 1422

Performing thorough exercise wrap-up will ensure that all relevant data is collected to support effective 1423 evaluation and improvement planning. 1424

Debriefings 1425

Immediately following the exercise, a short debriefing should be conducted with exercise planning team 1426 members to ascertain their level of satisfaction with the exercise, discuss any issues or concerns, and 1427 propose improvements. Planners should collect exercise attendance lists, provide copies to the exercise 1428 planning team leader, collect Participant Feedback Forms, and develop debriefing notes that can be 1429 integrated into the meeting minutes. 1430


Player Hot Wash 1431

For drills, functional, and full-scale exercises, a Hot Wash should be conducted for each functional area 1432 by that functional area’s controller or evaluator immediately following an exercise. The Hot Wash should 1433 be led by an experienced facilitator who can ensure that the discussion remains brief and constructive and 1434 who can focus conversation on strengths and areas for improvement while these issues remain fresh in 1435 players’ minds. It also provides an opportunity for them to gain clarification on exercise play that was 1436 outside of their view. The information gathered during a Hot Wash can be used during the After Action 1437 Report (AAR)/Improvement Plan (IP) process, and exercise-specific suggestions can be used to improve 1438 future exercises. Hot Washes also provide opportunities to distribute Participant Feedback Forms, which, 1439 when completed by players, can be used to help generate the AAR/IP. 1440

Controller/Evaluator (C/E) Debriefing 1441

The C/E Debriefing provides a forum for functional area controllers and evaluators to review the exercise. 1442 The exercise planning team leader facilitates this debriefing, which provides each controller and evaluator 1443 with an opportunity to provide an overview of the functional area they observed and to discuss both 1444 strengths and areas for improvement. During the debriefing, controllers and evaluators complete and 1445 submit their EEGs and their Participant Feedback Forms. Debriefing results are captured for inclusion in 1446 the AAR/IP. 1447


5. Evaluation 1448

Overview 1449

The HSEEP evaluation methodology promotes an analytical process to assess performance against 1450 identified core capabilities and exercise-specific objectives. Effective exercise evaluation involves 1451 planning evaluation activities, collecting evaluation data during exercise conduct, and analyzing collected 1452 data to assess performance. Exercise practitioners are encouraged to apply and adapt the program’s 1453 doctrine and methodology on exercise evaluation to meet their specific needs. 1454

Plan the Exercise Evaluation 1455

Exercise evaluation truly begins during exercise planning, not after an exercise is complete. Setting clear 1456 evaluation requirements, based on the exercise objectives, early in the planning process will ensure that 1457 the design, development, and execution of the exercise best support an effective evaluation. Therefore, 1458 thorough planning and organization prior to an exercise is imperative to effective and successful exercise 1459 evaluation. Planning an exercise evaluation typically includes: 1460

• Selecting a lead evaluator and evaluation team; 1461

• Defining evaluation requirements based on exercise scope, exercise objectives, and core 1462 capabilities; 1463

• Recruiting, training, and assigning evaluators; 1464

• Finalizing evaluation documentation; and 1465

• Conducting a Controller and Evaluator (C/E) Briefing. 1466

Through this process, an evaluation team can develop a thorough plan to address how the exercise will be 1467 evaluated. 1468

Select Lead Evaluator and Evaluation Team 1469

Early in the exercise planning process, the exercise planning team leader should appoint a lead evaluator 1470 who will oversee all facets of the evaluation process. The lead evaluator participates fully as a member of 1471 the exercise planning team and should be familiar with the exercise’s objectives. A lead evaluator should 1472 have the management skills needed to oversee a team of evaluators as well as the knowledge and 1473 analytical skills to undertake a thorough and accurate analysis of exercise performance. 1474

The exercise planning team and lead evaluator should determine the structure of the exercise evaluation 1475 team based on factors such as the exercise scope, exercise objectives, and the associated core capabilities 1476 to be assessed during the exercise. Exercises that involve multiple jurisdictions and/or multiple playing 1477 locations should consider assigning jurisdiction leads or site leads, as illustrated by the example provided 1478 in Figure D-1. These individuals support the lead evaluator and manage the activities of other evaluators 1479 assigned to that location. 1480


1481 Figure D-1: Example Exercise Evaluation Team Organization 1482

Define Evaluation Requirements 1483

To ensure effective evaluation, the exercise planning team must define exercise evaluation requirements 1484 by considering exercise scope, core capabilities to be exercised, and exercise-specific objectives. 1485

Exercise Scope 1486

Exercise scope consists of, but is not limited to, the days and hours of the exercise, the locations/sites for 1487 exercise play, the number of exercise participants, and the type of exercise. Defining the scope helps 1488 inform the number of evaluators needed and where evaluators should be placed for observation. 1489

Exercise Objectives and Core Capabilities 1490

Effective evaluation assesses performance against the exercise objectives and documents core capability. 1491 Lead evaluators should therefore have an in-depth understanding of an exercise’s specific objectives and 1492 associated core capabilities. Understanding this information will enable lead evaluators to define the types 1493 of exercise data that evaluators should collect and how evaluators will analyze and assess that data. 1494 Understanding the exercise purpose, the capabilities to be assessed, and the associated capability 1495 performance objectives supports evaluation planning, design, and selection of appropriate evaluators. 1496

For seminars and workshops, consideration of the exercise’s objectives and capabilities can inform the 1497 development of a Situation Manual (SitMan), which provides the exercise facilitator with suggestions for 1498 how to steer exercise discussion to the objectives and capabilities being evaluated. 1499

For other types of exercises, evaluators can consider developing Exercise Evaluation Guides (EEGs). 1500 EEGs provide a template for collecting and evaluating exercise data in relation to objectives and 1501 associated core capabilities. EEGs typically identify key metrics and targets for exercise objectives and 1502 core capabilities and enable evaluators to capture structured and unstructured data regarding exercise 1503 performance. Evaluators should develop and customize EEGs to meet the unique objectives of their 1504 exercise and to reflect jurisdiction-specific capability targets. Sample evaluation materials and templates 1505 are available on the HSEEP Web site (www.hseep.dhs.gov). 1506

http://www.hseep.dhs.gov/�


As with exercise design, individuals with and without disabilities who have access and functional needs 1507 and applicable support agencies/organizations must also be included from the beginning of the evaluation 1508 planning process. Doing so ensures that exercise planners and evaluators understand their perspectives 1509 and can structure the evaluation to capture issues relevant to these individuals and agencies and 1510 incorporate them into AAR/IP accordingly. 1511

By defining requirements early in the exercise and evaluation planning process, the exercise planning 1512 team and lead evaluator can develop the appropriate evaluation documentation and tools and ensure 1513 evaluators are trained and prepared. 1514

Recruit, Assign, and Train Evaluators 1515

Once evaluation requirements have been defined, the lead evaluator oversees recruiting, assigning, and 1516 training evaluators. The evaluation requirements already discussed play a critical role in determining how 1517 many evaluators must be recruited, what kind of subject-matter expertise they must possess, how they are 1518 assigned during an exercise, and what kind of training or instruction they must receive prior to the 1519 exercise. 1520

Recruiting and Assigning Evaluators 1521

Whenever possible, evaluators should have experience and subject-matter expertise in the functional area 1522 they are assigned to observe. Evaluators should be assigned to different exercise play areas on the basis of 1523 their subject-matter expertise. Evaluator assignments should be decided upon, recorded, and 1524 communicated to evaluators prior to exercise conduct. The number of evaluators assigned to each 1525 exercise play location should be informed by the number of objectives and capabilities being evaluated 1526 and the overall size of the exercise. For seminars, workshops, and tabletop exercises, the number of 1527 evaluators typically depends on the number and organization of the players. 1528

Training Evaluators 1529

Effective evaluator training ensures that exercise evaluators have a shared understanding of key data to be 1530 collected and how that data will contribute to the assessment of performance. Evaluator training typically 1531 includes reviewing the following: 1532

• General information about the exercise, including scope, schedule, scenario, objectives, and 1533 associated core capabilities; 1534

• Relevant evaluator documentation (e.g., SitMan, C/E Handbook); 1535 • Specific plans, policies, procedures, agreements, or other information that are the focus of the 1536

exercise; and 1537 • Evaluation documentation and tools (e.g., EEGs). 1538

Evaluator training materials and other documents can be found on the HSEEP Resource Center Web site. 1539

Develop and Finalize Evaluation Documentation 1540

Once exercise requirements have been defined and evaluation planning to meet those requirements has been 1541 completed, the lead evaluator should finalize the evaluation section of the C/E Handbook. The C/E 1542 Handbook typically contains the following evaluation-related information: 1543

• Exercise-Specific Details: Exercise scenario, schedule of events, and evaluation schedule; 1544

• Controller and Evaluator Team Organization, Assignments, and Locations: A list of 1545 controller and evaluator locations, a map of the exercise site(s), and a control and evaluation team 1546 organizational chart; 1547

• Controller and Evaluator Instructions: Step-by-step instructions for controllers and evaluators 1548 regarding what to do before the exercise, during the exercise, and following its conclusion; and 1549


• Control and Evaluation Tools: As necessary, exercise-specific EEGs, the MSEL, and blank 1550 paper for taking notes. 1551

In less complex exercises, the lead evaluator may communicate this C/E Handbook information verbally. 1552 For more complex exercises, lead evaluators should seek to fully document information in the C/E 1553 Handbook before distributing to evaluators. 1554

Conduct a Pre-Exercise Controller and Evaluator (C/E) Briefing 1555

Before exercise play begins, the lead evaluator typically meets with the controllers and/or evaluators to 1556 verify roles, responsibilities, and assignments and to provide any significant updates (e.g., last-minute 1557 changes to the scenario, new assignments). The C/E Briefing also provides an opportunity for evaluators 1558 to ask questions and to ensure complete understanding of their roles and responsibilities. Depending on a 1559 variety of factors—including exercise scope, objectives, and scenario—it may be necessary to conduct 1560 separate briefings for exercise evaluators and for exercise controllers. 1561

Collect Exercise Data 1562

Effective exercise evaluation requires thorough collection of data from exercise play. Collecting exercise 1563 data can differ depending on the type of exercise. Common to all types of exercises is a focus on 1564 objectives-driven and capabilities-based evaluation. 1565

Observing a Seminar, Workshop, Tabletop Exercise (TTX), or Game 1566

Seminars, workshops, tabletop exercises (TTXs), and games use facilitated discussions to assess 1567 performance against objectives and associated core capabilities. Facilitators help evaluators collect useful 1568 data by keeping discussions focused on the exercise objectives. Strategies for keeping discussion focused 1569 and constructive may be recorded in a SitMan or appendix to the SitMan, which is provided to facilitators 1570 prior to an exercise. 1571

Exercise evaluators should collect data, in a non-attribution environment, in accordance with the 1572 evaluation training, documentation, and tools they have received. Evaluators will generally record the 1573 following types of information from player discussions: 1574

• The plans, policies, and procedures players would implement to prevent, protect against, mitigate, 1575 respond to, or recover from the incident described in the exercise scenario; 1576

• How clearly defined the authorities, roles, and responsibilities of the various government agencies 1577 and private organizations are; 1578

• The decisions that players make and how those decisions are made; 1579

• What information about the scenario, the hazard, the victims, and the risks to participants and the 1580 public is collected and who is collecting it; 1581

• How and what information is shared with other agencies and with the public; 1582

• What the roles of Federal and State agencies are and how resources are requested, distributed, and 1583 controlled; 1584

• The mutual aid agreements that exist and how they would be activated; 1585

• The recommendations for improvements that are made by the group; 1586

• The issues which are unresolved or require follow-up; and 1587

• The actions that players plan to take to address outstanding issues. 1588


A debriefing/Hot Wash with the exercise planning team, facilitators, and evaluators is typically held 1589 immediately following an exercise. The purpose of this debriefing is to collect observations and thoughts 1590 about exercise conduct. The debriefing also provides evaluators the opportunity to clarify points or collect 1591 any missing information. Following an exercise, evaluators may also supplement evaluation data with 1592 follow-up discussions with exercise participants, reviews of Participant Feedback Forms, and facilitator 1593 notes. 1594

Observing a Drill, Functional Exercise (FE), or Full-Scale Exercise (FSE) 1595

Drills, functional exercises (FEs), and full-scale exercises (FSEs)—assess performance based on actual 1596 operational results. During drills, FEs, and FSEs, evaluators are strategically pre-positioned in locations at 1597 which they can gather useful data, and must track and record participant actions carefully. After an 1598 exercise, the information recorded by evaluators is used to analyze whether or not exercise objectives 1599 were successfully met and whether core capabilities were successfully demonstrated. 1600

Exercise evaluators should collect data in accordance with the evaluation training, documentation, and 1601 tools they have received. As players make decisions and take actions, evaluators should take notes that 1602 capture the following types of information: 1603

• The name or position of the person who performed the action or made the decision; 1604

• The observed action; 1605

• The location where the action or decision took place; 1606

• The time when the action was completed; 1607

• Why the action took place and/or why the decision was made; and 1608

• How the action was performed and how the decision was made. 1609

Additional evaluator observations and notes should include, but are not limited to, the following: 1610

• Initiation and unfolding of scenario events; 1611

• Deviations from plans or procedures; 1612

• Timeliness and other performance measures; 1613

• Effectiveness of, or shortcomings in, command and control; 1614

• Creative player problem-solving activities; and 1615

• Equipment issues that affect player efforts. 1616

Because numerous events may be occurring simultaneously, evaluators may not be able to record all of 1617 the action. Knowing which events are important allows for manageable action recording, eliminates 1618 unnecessary information, and provides the most useful data for exercise evaluation. Effective evaluator 1619 training is critical to ensuring a shared understanding of critical exercise actions. 1620

Evaluators should be able to recognize the occurrence of the following events, as appropriate: 1621

• An inject is information—including directives, instructions, and decisions—provided by exercise 1622 controllers to exercise players in order to drive exercise play toward the achievement of 1623 objectives. 1624

• A message in occurs when an individual or group receives information from someone outside of 1625 his or her physical location. Messages can be received as part of player-to-player activity or as a 1626 controlled exercise inject. 1627


• A message out occurs when an individual sends information to another individual or group of 1628 players outside of his or her physical location. 1629

• A discussion is a conversation involving several players. 1630

• A decision occurs when an individual or group arrives at a conclusion or makes a specific 1631 determination. 1632

• A directive is a specific order or direction given to one or more players. 1633

• Movement occurs when an individual, group, or piece of equipment relocates. 1634

Evaluators should not interfere with exercise play. However, evaluators may need to interact with players 1635 during the exercise to ask follow-up questions about observations. 1636

Conducting a Player Hot Wash 1637

Immediately after a drill, FE, or FSE, evaluators may conduct an exercise debriefing with players and 1638 controllers. This facilitated discussion, referred to as a Hot Wash, allows players to engage in a 1639 self-assessment immediately following exercise play. The Hot Wash also provides evaluators with the 1640 opportunity to clarify points or collect any missing information from players before they leave the 1641 exercise venue. 1642

As part of the Hot Wash, evaluators may distribute Participant Feedback Forms (see the HSEEP Web site 1643 for examples), which can provide details on player perceptions regarding exercise performance. 1644 Participant Feedback Forms can also serve to solicit general feedback on exercise quality, which can be 1645 provided to the exercise planning team to help implement improvements in future exercises. 1646

Collecting Supplemental Data 1647

As necessary, the lead evaluator may assign evaluators to collect supplemental data immediately after the 1648 exercise. Such data may be critical to fill in exercise evaluation gaps. For example, one useful source of 1649 supplemental evaluation data might include records produced by automated systems or communication 1650 networks and written records, such as duty logs and message forms. These records can help evaluators 1651 validate or provide more context regarding exercise observations. 1652

Analyze Data 1653

During data analysis, the evaluation team consolidates the data collected during the exercise, assessing 1654 performance against exercise-specific objectives and associated core capabilities. From this data analysis, 1655 evaluators identify key strengths and areas for improvement and may draft a quick-look After Action 1656 Report (AAR). Exercise stakeholders can then work to develop a full AAR and Improvement Plan (IP). 1657

Analyzing Data from a Tabletop Exercise (TTX) or Game 1658

Following the exercise, evaluators should review their notes of the discussion and begin to develop 1659 preliminary analysis of the exercise. Preliminary analysis typically involves developing a chronological 1660 narrative of relevant discussion for each exercise objective and associated core capability, highlighting 1661 strengths and areas for improvement. 1662

The evaluators’ analyses should help answer the following types of critical evaluation questions: 1663

• Were the objectives of the exercise met? 1664

• Did discussion suggest that exercise stakeholders could deliver core capabilities? If not, why? 1665


• Did discussion suggest that personnel need additional training in plans, procedures, and protocols 1666 to complete the activities and tasks needed to demonstrate a capability? 1667

• Did discussion identify any resource shortcomings? 1668

• Do the current authorities, plans, policies, and procedures support performance objectives? 1669

• Do personnel from multiple agencies or jurisdictions need to work together to perform a core 1670 capability? If so, are the agreements or relationships in place to support the coordination 1671 required? 1672

• What should be learned from this exercise? 1673

• What strengths and areas for improvement were identified? 1674

As appropriate, evaluators should seek to identify the root cause(s) of any areas for improvement. 1675 Uncovering root causes enables exercise stakeholders to develop actionable recommendations that will 1676 address capability gaps. 1677

Analyzing Data from a Drill, Functional Exercise (FE), or Full-Scale Exercise (FSE) 1678

Following the exercise, evaluators should review their exercise evaluation documentation and begin to 1679 develop preliminary analysis of the exercise. In conducting their analyses, evaluators should make use of 1680 all available data, including, as appropriate, data from EEGs filled out during exercise play, other notes or 1681 records from the exercise, notes from the post-exercise Hot Wash, notes from the C/E Debriefing, and any 1682 other relevant materials. Evaluators should focus their analysis on assessing performance against exercise 1683 objectives and core capabilities. This analysis should seek to identify strengths, areas for improvement, 1684 and core capability gaps. 1685

The lead evaluator may work with the evaluation team to reconstruct a timeline of exercise events as they 1686 occurred—an approach similar to the reconstruction of events that is completed by many agencies and 1687 organizations following an actual incident. The final product of this reconstruction should be a master 1688 timeline that captures all key exercise events relevant to the objectives and capabilities being exercised. 1689

The evaluators’ analyses should help answer the following types of critical evaluation questions: 1690

• Were the objectives of the exercise met? 1691

• Did discussion suggest that exercise stakeholders could deliver core capabilities? If not, why? 1692

• What did evaluators observe? 1693

• What should evaluators have observed according to policies, plans, and procedures? 1694

• Is there a difference? If so, why? 1695

• What is the consequence of that difference? 1696

• What should be learned from this exercise? 1697

• What strengths and areas for improvement were identified? 1698

Develop Quick-Look Report 1699

Based on initial analyses described above, the evaluation team may develop and release a Quick-Look 1700 Report to the exercise participants. This Quick-Look Report can be a valuable way to quickly articulate 1701 strengths, areas for improvement, lessons learned, and capability gaps immediately following the 1702 exercise. The Quick-Look Report should provide a quick snapshot of the high-priority, mission-critical 1703 issues and actions resulting from the exercise. Mission-critical findings should encompass those issues 1704


that require immediate action by an organization’s senior leadership to mitigate vulnerabilities that, if not 1705 corrected, could cause significant damage to critical infrastructure, degrade national security, or cause 1706 loss of life. Evaluation teams typically aim to produce the Quick-Look Report within 7 days of the 1707 exercise. 1708

Develop the Draft After Action Report (AAR)/Improvement Plan (IP) 1709

The AAR/IP is the final exercise document that summarizes key exercise-related evaluation information. 1710 The length and format of AAR/IPs depend on the exercise type and scope and is determined by the 1711 exercise planning team. Key elements of the AAR/IP should include: 1712

• Exercise Overview. This includes identifying information, such as the exercise name, date, 1713 duration, and participants. 1714

• Exercise Design Summary. This includes the exercise objectives and core capabilities and a 1715 summary of key scenario events. 1716

• Analysis of Objectives and Core Capabilities. This includes an assessment of exercise 1717 performance in relation to exercise objectives and core capabilities and highlights relevant areas 1718 for improvement, strengths, lessons learned, and capability gaps. 1719

• IP. The IP identifies specific corrective actions from the exercise, assigns these actions to 1720 responsible parties, and establishes target dates for action completion. 1721

A sample HSEEP AAR template is available on the HSEEP Resource Center Web site 1722 (www.hseep.dhs.gov). 1723

Summary Report Content for a Seminar or Workshop 1724

Some exercises—such as seminars and workshops—may produce a Summary Report. Generally, 1725 Summary Reports will provide the following information: 1726

• Overview of guest and keynote speaker(s) presentations; 1727

• Summary of discussion points including outstanding issues to be addressed; and 1728

• Summary of results and recommendations.1729

http://www.hseep.dhs.gov/�


6. Improvement Planning 1730

Overview 1731

Improvement planning is the process by which the areas for improvement identified in the draft After 1732 Action Report (AAR) are resolved through development of concrete corrective actions that are prioritized, 1733 tracked, and analyzed as part of a continuous corrective action process. Completed corrective actions can 1734 be validated through subsequent exercises or real-world events, informing future exercise program 1735 management activities. Exercise practitioners are encouraged to apply and adapt HSEEP doctrine on 1736 improvement planning to meet their specific needs. 1737

Conduct After Action Conference (AAC) 1738

Following completion of the draft AAR, the exercise planning team, evaluation team, and other exercise 1739 stakeholders identified in the AAR meet for an After Action Conference (AAC) to review and refine the 1740 draft AAR. 1741

Before the conference, the draft AAR can be distributed to conference participants for review. Allowing 1742 participants to see the AAR in its entirety in advance can maximize the value of the AAC by ensuring that 1743 key individuals are familiar with the AAR’s content and will come prepared to discuss and confirm key 1744 issues. Conference organizers should ensure including representatives of organizations expected to be 1745 assigned corrective actions receive invitations and attend the AAC. 1746

The AAC should be an interactive session, providing attendees the opportunity to discuss and validate the 1747 analytical findings in the draft AAR. Conference participants should ensure that there is consensus 1748 regarding the assessment of strengths, areas for improvement, and capability gaps. This discussion and 1749 validation process is also important for those findings nominated as potential lessons learned in the AAR. 1750 Exercise participants should discuss these lessons learned to ensure that the lessons identified have truly 1751 been “learned.” Exercise stakeholders can submit AAR lessons learned to the U.S. Department of 1752 Homeland Security (DHS) Lessons Learned Information Sharing System (www.LLIS.gov) for wider 1753 distribution, as appropriate. 1754

Develop Improvement Plan (IP) 1755

In addition to refining the draft AAR, the AAC should be devoted to discussing the specific areas for 1756 improvement and appropriate responsibility to inform the development of the Improvement Plan (IP). The 1757 IP identifies specific corrective actions needed to address areas for improvement identified in the AAR. 1758 The IP also details those organizations responsible for implementing the corrective action and the 1759 timeframe for implementation. Each exercise stakeholder organization should identify a point of contact 1760 (POC) responsible for reporting progress on assigned corrective action milestones and timelines. 1761

Corrective actions within the IP should contain sufficient detail. Exercise stakeholders should use the 1762 following questions as a guide for developing corrective actions: 1763

• What changes need to be made to plans and procedures to improve performance? 1764

• What changes need to be made to organizational structures to improve performance? 1765

• What changes need to be made to leadership and management processes to improve 1766 performance? 1767


• What training is needed to improve performance? 1768

• What changes to (or additional) equipment are needed to improve performance? 1769

• What lessons can be learned that will direct how to approach a similar problem in the future? 1770

• What, if any, statutory changes are needed? 1771

Not all areas for improvement in the AAR can be fully addressed at the AAC. For more complex issues, 1772 the IP may detail the first step in a long corrective action process. For example, an IP could call for 1773 creation of a working group to examine possible solutions to a complex issue and impose a deadline for 1774 the working group to select a course of action. 1775

As appropriate, corrective actions should be written to include attainable metrics that gauge progress 1776 toward full implementation. Examples of metrics include, but are not limited to, the number of personnel 1777 trained in a core capability or the percentage of equipment that is National Incident Management System 1778 resource typed. These metrics should be defined against concrete deadlines so that gradual progress 1779 toward attainment of corrective actions can be tracked. 1780

Each exercise program should have a program manager responsible for tracking corrective actions to 1781 completion. Following IP development, this individual should subsequently monitor progress toward 1782 implementation of each corrective action. The program manager should also ensure that implemented 1783 corrective actions inform future exercise design and development activities so that improvements are 1784 exercised and validated. 1785

Finalize After Action Report (AAR) and the 1786

Improvement Plan (IP) 1787

Following the AAC, exercise stakeholders should finalize the AAR and the IP. Finalizing the AAR and 1788 the IP involves incorporating the corrections, clarifications, and other feedback provided by participants 1789 at the AAC. Once these inputs have been incorporated, the AAR and IP can be distributed to members of 1790 the exercise planning team for review and validation. 1791

Once the exercise planning team has validated the AAR/IP and all necessary authorities have approved 1792 the AAR/IP, the document is considered final. To protect potentially sensitive information, the exercise 1793 planning team should agree on a distribution list for the final AAR/IP. The finalized corrective actions 1794 captured in the AAR/IP should be tracked through continual updates as part of the corrective action 1795 process. 1796

Track Corrective Action Implementation 1797

Exercise stakeholders should establish an ongoing process to track the implementation of corrective 1798 actions. By tracking corrective actions to completion, exercise stakeholders will be able to demonstrate 1799 that exercises have yielded tangible preparedness improvements. 1800

Event Points of Contact (POCs) 1801

A successful exercise program should have a designated event POC who is responsible for continuously 1802 tracking implementation of the corrective actions identified and assigned in AAR/IPs. This individual will 1803 be the central POC for exercise improvements and is responsible for compiling corrective actions, 1804 following the corrective action process, and generating reports on the progress of those corrective actions. 1805


Organization Points of Contact (POCs) 1806

Stakeholders participating in the exercise should identify a POC who is responsible for managing the 1807 corrective actions assigned to the stakeholder organization and assigning individual action officers to 1808 complete specific corrective actions. The organization POC can collect information from assigned action 1809 officers on the progress of corrective actions and provide regular updates on this progress to the event 1810 POC. 1811

Action Officers 1812

Action officers should be assigned to each corrective action and are responsible for completing the 1813 prescribed action. Action officers should provide regular updates to their organization’s POC, who can 1814 systematically compile the updates to produce periodic progress reports on the status of all corrective 1815 actions identified in an exercise’s AAR/IP. These reports can track whether the performance metrics 1816 defined for corrective actions in the AAR/IP are achieved on schedule. 1817

Continual Improvement 1818

The progress reports on corrective actions should illustrate a consistent trend of progress. Because the 1819 AAR/IP ties these corrective actions to specific exercise objectives and core capabilities, these reports 1820 ultimately demonstrate the concrete ways in which exercises build and maintain core capabilities. Once 1821 exercise stakeholders have had time to fully implement post-exercise corrective actions, a new cycle of 1822 exercise activities can test and validate these corrective actions and improved capabilities. 1823

1824

PRE-DECISIONAL DRAFT Terms and Acronyms-1

Glossary of Terms and 1825

Acronym List 1826

This document provides descriptions for many of the commonly-used terms of the Homeland 1827 Security Exercise Evaluation Program (HSEEP). 1828

A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z 1829

1830 A

Term Description Actor Actors are volunteers who simulate specific roles, such as disaster casualty

victims, in order to add realism to an exercise. Actor Briefing An Actor Briefing is generally conducted before the exercise and provides

actors with an overview of the exercise, including: exercise overview, safety, real emergency procedures, symptomatology, acting instructions, and schedule. Identification badges and symptomatology cards are distributed before or during this briefing.

Administration/ Finance Section

The Administration/Finance Section of the exercise planning team provides financial management and administrative support throughout exercise development, including exercise registration support and scheduling.

After Action Conference (AAC)

The AAC is a meeting held among the lead evaluator, members of the evaluation team, and exercise stakeholders to debrief the exercise and to review and refine the draft AAR. The AAC should be an interactive session, providing attendees the opportunity to discuss and validate the analytical findings in the draft AAR.

After Action Report (AAR)

The AAR is summarizes key exercise-related evaluation information, including the exercise overview, exercise design summary, and analysis of objectives and core capabilities. The AAR is usually developed in conjunction with an IP. The lead evaluator and exercise planning team draft the AAR and submit it to conference participants before the After Action Conference.

Analytical Red Teaming

In prevention exercises, analytical red teaming is a discussion-based technique used to employ an adversary’s perspective to advance security by providing an alternative view of threats, vulnerabilities, and countermeasures.

Assembly Area The assembly area is a gathering place for all deployable resources that will be playing in an exercise. The purpose of the assembly area is to gather all resources and personnel near the exercise site prior to StartEx for safety briefings, weapons checks, and to ensure that resources and personnel are transported in a safe and unhurried manner.

Assembly Area Controller

The assembly area controller is responsible for the logistical organization of the assembly area, including placement locations for units entering the assembly area, the release of dispatched units into the field, and coordination of pathways and overall safety within the assembly area.


A Term Description

Attack Tree The attack tree is a tool used during prevention exercises that provides the exercise planning team with a visual representation of the anticipated and potential paths an adversary can take to execute an attack. It is useful for both planning and evaluating prevention exercises.

1831


B Term Description

Best Practices Best practices are peer-validated techniques, procedures, and solutions that prove successful and are solidly grounded in actual experience in operations, training, and exercises. After Action Reports (AARs) should identify lessons learned and highlight best practices. Lessons learned and best practices from exercises can be submitted to Lessons Learned Information Sharing (LLIS.gov), the Department of Homeland Security’s national network for lessons learned and best practices.


C

Term Description Capabilities-Based Planning

Capabilities-based planning is defined as planning, under uncertainty, to build capabilities suitable for a wide range of threats and hazards while working within an economic framework that necessitates prioritization and choice. Capabilities-based planning is the basis for guidance such as the National Preparedness Goal.

Capability A capability may be delivered with any combination of properly planned, organized, equipped, trained, and exercised personnel to achieve an intended outcome.

Concept and Objectives (C&O) Conference

A C&O Conference is the formal beginning of the exercise planning process. It is held to identify the type, scope, objectives, and purpose of the exercise. For less complex exercises and for organizations with limited resources, the C&O Conference can be conducted in conjunction with the IPC.

Contextual Inject Contextual injects are introduced to a player by a controller to help build the exercise operating environment and/or keep exercise play moving. For example, if the exercise is designed to test information-sharing capabilities, a MSEL inject can be developed to direct an actor to portray a suspect by behaving suspiciously in front of a law enforcement player.

Contingency Inject Contingency injects are events that a controller provides to a player if the players get off track or do not take an action that is necessary for the continuation of the exercise. This ensures that play moves forward, as needed, to adequately evaluate performance of activities. For example, if a simulated secondary device is placed at an incident scene during a terrorism response exercise, but is not discovered, a controller may want to prompt an actor to approach a player to say that he or she witnessed suspicious activity close to the device location.

Control Staff Instructions (COSIN)

The COSIN contains guidance that controllers, simulators, and evaluators need concerning procedures and responsibilities for exercise control, simulation, and support. The COSIN provides guidelines for control and simulation support and establishes a management structure for these activities.

Controller/Evaluator (C/E) Briefing

The C/E Briefing is a preexercise overview for controllers, evaluators, and the exercise administrative staff. The briefing summarizes the C/E Handbook (or the Control Staff Instructions [COSIN] and Evaluation Plan [EvalPlan]) and focuses on explaining the roles and responsibilities of controllers and evaluators.

Controller/Evaluator (C/E) Debriefing

The C/E Debriefing provides a forum for functional area controllers and evaluators to review the exercise. The exercise planning team leader facilitates this debriefing, which provides each controller and evaluator with an opportunity to provide an overview of the functional area they observed and to discuss both strengths and areas for improvement. During the debriefing, controllers and evaluators complete and submit their EEGs and their Participant Feedback Forms. Debriefing results are captured for inclusion in the AAR/IP.


C Term Description

Controller/Evaluator (C/E) Handbook

The C/E Handbook specifically describes the roles and responsibilities of exercise controllers and evaluators and the procedures they must follow. Because the C/E Handbook contains information about the scenario and about exercise administration, it is distributed to only those individuals specifically designated as controllers or evaluators. For larger, more complex exercises, planners may develop a written EvalPlan in lieu of, or in addition to, a C/E Handbook.

Controllers In a game, drill, functional exercise, or full-scale exercise, controllers plan and manage exercise play, set up and operate the exercise incident site, and possibly take the roles of individuals and agencies not actually participating in the exercise. Controllers direct the pace of exercise play, provide key data to players, and may prompt or initiate certain player actions and injects to the players as described in the Master Scenario Events List (MSEL) to ensure exercise continuity. Controllers issue exercise materials to players as required, monitor the exercise timeline, and monitor the safety of all exercise participants. Controllers are the only participants who should provide information or direction to players. All controllers should be accountable to one senior controller.

Core Capabilities Distinct critical elements necessary to achieve the National Preparedness Goal.

Corrective Action Corrective actions are the concrete, actionable steps outlined in an IP that are intended to resolve preparedness gaps and shortcomings experienced in exercises or real-world events.

Critical Path The critical path is the map and timeline of moves and countermoves throughout the course of a prevention exercise. The critical path’s use in this capacity begins during exercise design and development, when planners define the “planned” critical path of the exercise scenario based upon an attack tree. At the end of the exercise, the exercise critical path can be compared to the planned critical path, highlighting any deviations.


D

Term Description Drill A drill is a coordinated, supervised activity usually employed to validate a

specific operation or function in a single agency or organization. Drills are commonly used to provide training on new equipment, develop or validate new policies or procedures, or practice and maintain current skills.


E

Term Description Evaluation Plan (EvalPlan)

The EvalPlan is typically used for exercises of a large scope and scale. An Evaluation Plan provides evaluation staff with guidance and instructions on evaluation or observation methodology to be used as well as essential materials required to execute their specific functions.

Evaluation Team The evaluation team consists of evaluators trained to observe and record participant actions. These individuals should be familiar with the exercising jurisdiction’s plans, policies, procedures, and agreements.

Evaluator Evaluators, selected from participating agencies, are chosen based on their expertise in the functional areas they will observe. Evaluators use Exercise Evaluation Guides to measure and assess performance, capture unresolved issues, and analyze exercise results. Evaluators passively assess and document players’ performance against established emergency plans and exercise evaluation criteria, in accordance with HSEEP standards and without interfering with exercise flow.

Event Within the Master Scenario Events List, an event is an expected action that is anticipated to take place during an exercise.

Exercise An exercise is an instrument to train for, assess, practice, and improve performance in prevention, protection, mitigation, response, and recovery capabilities in a risk-free environment. Exercises can be used for testing and validating policies, plans, procedures, training, equipment, and interagency agreements; clarifying and training personnel in roles and responsibilities; improving interagency coordination and communications; identifying gaps in resources; improving individual performance; and identifying opportunities for improvement.

Exercise Director

The exercise director oversees all exercise functions during exercise conduct; oversees and remains in contact with controllers and evaluators; debriefs controllers and evaluators following the exercise; and oversees setup and cleanup of the exercise as well as positioning of controllers and evaluators.

Exercise Evaluation Guide (EEG)

EEGs provide a template for collecting and evaluating exercise data in relation to objectives and associated core capabilities. EEGs typically identify key metrics and targets for exercise objectives and core capabilities and enable evaluators to capture structured and unstructured data regarding exercise performance. Evaluators should develop and customize EEGs to meet the unique objectives of their exercise and to reflect jurisdiction-specific capability targets.

Exercise Plan (ExPlan)

ExPlans are general information documents that help drills, FEs, and FSEs run smoothly by providing participants with a synopsis of the exercise. They are published and distributed to the participating organizations following development of most of the critical elements of the exercise. In addition to addressing exercise objectives and scope, ExPlans assign activities and responsibilities for exercise planning, conduct, and evaluation. The ExPlan is intended to be seen by the exercise players and observers—therefore, it does not contain detailed scenario information that may reduce the realism of the exercise.


E Term Description

Exercise Planning Team

The exercise planning team is responsible for the successful execution of all aspects of an individual exercise. The planning team works with exercise stakeholders to determine or refine exercise objectives based on core capabilities, creates a realistic scenario to achieve the exercise objectives, and develops documents to guide exercise conduct and evaluation. HSEEP recommends that the planning team’s organization and management principles reflect those of the National Incident Management System (NIMS), with clearly defined roles and responsibilities and a manageable span of control.

Exercise Play Area The exercise play area is the site or facility where the bulk of tactical player activities and tasks are demonstrated during an exercise.

Exercise Play Rules Exercise play rules are the parameters that exercise participants follow during the exercise. Exercise play rules describe appropriate exercise behavior, particularly in the case of real-world emergencies.

Exercise Program Management

Exercise program management is the act of overseeing a variety of individual exercises and supporting activities sustained over time. An effective exercise program helps whole community stakeholders maximize efficiency, resources, time, and funding by ensuring that individual exercises are part of a coordinated, integrated approach to building, maintaining, and delivering core capabilities.

Exercise Program Manager

The exercise program manager develops a self-sustaining exercise program through program budget management oversight, exercise conduct, and improvement tracking monitoring and reporting.

Exercise Project Management

Exercise project management is the act of engaging in design and development, conduct, evaluation, and improvement planning for an individual exercise. Effective project management ensures that all aspects of planning and executing an individual exercise are done efficiently and are grounded in common approaches and best practices.

Expected Action Event

An expected action event serves as a holding place in the Master Scenario Events List and notifies controllers of when an expected action should occur.


F Term Description

Facilitated Discussion

A facilitated discussion is the focused discussion of specific issues through a facilitator with functional area or subject-matter expertise.

Facilitator During a seminar, workshop, tabletop exercise, or game, the facilitator(s) is responsible for keeping participant discussions on track with exercise objectives and ensuring all issues and objectives are explored as thoroughly as possible within time constraints. If an exercise uses breakout groups, more than one facilitator may be needed.

Final Planning Conference (FPC)

The FPC is the final forum for reviewing exercise processes and procedures. An FPC should be conducted for all exercises to ensure that all elements of the exercise are ready for conduct. Prior to the FPC, the exercise planning team receives final drafts of all exercise materials. No major changes to exercise’s design, scope, or supporting documentation should take place at or following the FPC. The FPC ensures that all logistical requirements have been met, outstanding issues have been identified and resolved, and exercise products are ready for printing.

Full-Scale Exercise (FSE)

FSEs are typically the most complex and resource-intensive type of exercise. They involve multiple agencies, organizations, and jurisdictions and validate many facets of preparedness. FSEs often include many players operating under cooperative systems such as the Incident Command System or Unified Command.

Functional Exercise (FE)

Functional exercises are designed to validate and evaluate capabilities, multiple functions and/or subfunctions, or interdependent groups of functions. FEs are typically focused on exercising plans, policies, procedures, and staff members involved in management, direction, command, and control functions. In FEs, events are projected through an exercise scenario with event updates that drive activity at the management level. An FE is conducted in a realistic, real-time environment; however, movement of personnel and equipment is usually simulated.


G Term Description

Game A game is a simulation of operations that often involves two or more teams, usually in a competitive environment, using rules, data, and procedures designed to depict an actual or hypothetic situation. Games explore the consequences of player decisions and actions and are therefore excellent tools to use when validating or reinforcing plans and procedures or evaluating resource requirements.

Ground Truth The ground truth is comprised of the detailed elements of a prevention exercise scenario that must remain consistent during exercise development and conduct to ensure that realism is maintained and objectives may be met in the unscripted move-countermove exercise environment. The ground truth includes the scenario timeline, local threat environment, UA threat group, and individual adversary profiles and relationships. Once composed, the ground truth is used as the basis for MSEL development and red team operations planning, if applicable.

Ground Truth Advisor

In prevention exercises, the ground truth advisor tracks how the adversary (i.e., red team) and exercise participant moves and countermoves change the fabric of the exercise environment, potentially creating additional truths. To ensure consistency, each unscripted MSEL inject should be vetted by the ground truth advisor.


H

Term Description Homeland Security Exercise and Evaluation Program (HSEEP)

HSEEP is a capabilities-based and objectives-driven exercise program that provides standardized policy, doctrine, and terminology for the program management and project management (including design and development, conduct, evaluation, and improvement planning) of homeland security exercises.

Hot Wash A Hot Wash is a facilitated discussion held immediately after an exercise among exercise players from each functional area. It captures feedback about any issues, concerns, or proposed improvements players may have about the exercise. The Hot Wash is an opportunity for players to voice their opinions on the exercise and their own performance.

HSEEP Toolkit The HSEEP Toolkit is an interactive, online tool for exercise scheduling, design, development, conduct, evaluation, and improvement planning. The HSEEP Toolkit can be accessed from the HSEEP Web site and includes the following subcomponent systems:

• National Exercise Schedule (NEXS) System • Design and Development System (DDS) • Exercise Evaluation Guide (EEG) Builder • Master Scenario Events List (MSEL) Builder • Corrective Action Program (CAP) System


I Term Description

Improvement Plan (IP)

The IP identifies specific corrective actions, assigns them to responsible parties, and establishes target dates for their completion. The IP is developed in conjunction with the After Action Report.

Improvement Planning Workshop (IPW)

An Improvement Planning Workshop provides a forum to identify training and exercise priorities based on a review of corrective action items from exercises and real-world events. IPW activities can inform TEPW activities or can be integrated into the TEPW. IPW activities typically include identifying and reviewing strengths and areas for improvement from previous real-world events and exercises through a review and analysis of After-Action Report/Improvement Plan data. Building on those identified strengths and areas for improvement, IPWs allow stakeholders to identify and discuss potential training and exercise priorities.

Information Sharing Environment Analysis (ISEA)

Before a prevention exercise, jurisdictions can use this process to identify, describe, and depict their State or local information-sharing environment as it pertains to standard operating procedures, policies, and systems.

Initial Planning Conference (IPC)

The IPC marks the beginning of the exercise development phase. An IPC’s purpose is to determine exercise scope by gathering input from the exercise planning team; design requirements and conditions (e.g., assumptions and artificialities); objectives; extent of play; and scenario variables (e.g., time, location, hazard selection). The IPC is also used to develop exercise documentation by obtaining the planning team’s input on exercise location, schedule, duration, and other relevant details.

Inject Injects are Master Scenario Events List events that prompt players to implement the plans, policies, and procedures that planners want the exercise to validate. Exercise controllers provide injects to exercise players to drive exercise play toward achievement of objectives. Injects can be written, oral, televised, and/or transmitted via any means (e.g., fax, phone, e-mail, voice, radio). Injects can be contextual or contingency.


L Term Description

Lead Evaluator The lead evaluator should participate fully as a member of the exercise planning team and should be a senior-level individual familiar with all relevant issues associated with the exercise, including plans, policies, and procedures of the exercising entity; Incident Command and decisionmaking processes of the exercising entity; and interagency and/or interjurisdictional coordination issues relevant to the exercise. The lead evaluator must have the management skills needed to oversee a team of evaluators over an extended process as well as the knowledge and analytical skills to undertake a thorough and accurate analysis of all capabilities being tested during an exercise.

Lessons Learned Lessons learned are knowledge and experience (both positive and negative) derived from observations and historical study of actual operations, training, and exercises. Exercise AAR/IPs should identify lessons learned and highlight best practices. Lessons learned and best practices from exercises can be submitted to Lessons Learned Information Sharing (LLIS.gov), the Department of Homeland Security’s national network for lessons learned and best practices.

Logistics Section The Logistics Section provides the supplies, materials, facilities, and services that enable the exercise to function smoothly without outside interference or disruption. This section consists of two subsections: service and support. The service subsection provides transportation, barricading, signage, food and drinks, real-life medical capability, and exercise security. The support subsection provides communications, purchasing, general supplies, management of very important persons (VIPs) and observer processing, and recruitment and management of actors.


M

Term Description Master Scenario Events List (MSEL)

The MSEL is a chronological timeline of expected actions and scripted events to be injected into exercise play by controllers to generate or prompt player activity. It ensures necessary events happen so that all objectives are met. Larger, more complex exercises may also use a procedural flow, which differs from the MSEL in that it contains only expected player actions or events. The MSEL links simulation to action, enhances exercise experience for players, and reflects an incident or activity meant to prompt players to action.

Master Scenario Events List (MSEL) Conference

The MSEL Conference may be held in preparation for more complex exercises to review the scenario timeline and focus on MSEL development. A MSEL Conference can be held in conjunction with or separate from the Midterm Planning Conference to review the scenario timeline for the exercise.

Midterm Planning Conference (MPC)

The MPC is a planning conference for exercises. It is used to discuss exercise organization and staffing concepts; scenario and timeline development; and scheduling, logistics, and administrative requirements. It is also a session to review draft documentation.

Mitigation The capabilities necessary to reduce loss of life and property by lessening the impact of disasters.

Moderated Discussion

A moderated discussion is a facilitated, discussion-based forum where a representative from each functional area breakout presents to participants a summary and results from a group’s earlier facilitated discussion. During moderated discussions, spokespersons summarize the facilitated discussion, present key findings and issues, and discuss any unresolved issues or questions. At the end of the moderated discussion period, the floor is open for questions.

Multiyear Training and Exercise Plan (TEP)

The Multiyear TEP is the foundation document guiding a successful exercise program. The Multiyear TEP articulates overall exercise program objectives and outlines a multiyear schedule of training and exercise activities designed to meet those objectives.


N Term Description

National Exercise Program (NEP)

The NEP’s mission is to serve as the principal exercise mechanism for examining the preparedness and measuring the readiness of the United States across the entire homeland security enterprise by designing, coordinating, conducting, and evaluating a progressive cycle of exercises that rigorously test the Nation’s ability to perform missions or functions that prevent, protect against, respond to, recover from, and mitigate all hazards.

National Incident Management System (NIMS)

The NIMS standard was designed to enhance the ability of the United States to manage domestic incidents by establishing a single, comprehensive system for incident management. It is a system mandated by Homeland Security Presidential Directive 5 (HSPD-5) that provides a consistent, nationwide approach for Federal, State, local, and tribal governments; the private sector; and nongovernmental organizations to work effectively and efficiently together to prepare for, respond to, and recover from domestic incidents, regardless of cause, size, or complexity.

National Preparedness Goal (Goal)

The National Preparedness Goal defines the core capabilities necessary to prepare for the specific types of incidents that pose the greatest risk to the security of the Nation. The Goal emphasizes actions aimed at achieving an integrated, layered, and all-of-Nation preparedness approach that optimizes the use of available resources. Specifically, the Goal defines success as: A secure and resilient Nation with the capabilities required across the whole community to prevent, protect against, mitigate, respond to, and recover from the threats and hazards that pose the greatest risk.

National Preparedness System

The National Preparedness System is an integrated set of guidance, programs, and processes that will enable the Nation to meet the National Preparedness Goal.


O

Term Description Objectives The cornerstone of exercise design is the selection of capabilities to be

assessed and determination of exercise objectives based on those capabilities. The exercise planning team determines exercise capabilities and objectives based on the overall exercise program objectives. The objectives should reflect the exercise sponsor’s specific needs, environment, plans, and procedures, while providing a framework for scenario development and a basis for evaluation. Planners should create objectives that are simple, measurable, attainable, realistic, and task-oriented (SMART) and should limit the number of exercise objectives to enable timely exercise conduct, facilitate reasonable scenario design, and support successful evaluation.

Observer Observers do not directly participate in the exercise; rather, they observe selected segments of the exercise as it unfolds, while remaining separated from player activities. Observers view the exercise from a designated observation area and are asked to remain within the observation area during the exercise. A dedicated controller or public information officer should be assigned to manage these groups. In a seminar, workshop, TTX, or game, observers may support the development of player responses to the situation during the discussion by delivering messages or citing references.

Observer Briefing An Observer Briefing is generally conducted the day of an exercise and informs observers and very important persons (VIPs) about program background, scenario, schedule of events, observer limitations, and any other miscellaneous information. Participant Handouts and ExPlans or SitMans, depending on the type of exercise being conducted, are often distributed during this briefing.

Observer/Media Area This is a designated area that provides observers and real-world media representatives with a view of the exercise but prevents them from interfering with exercise play.

Operations Section The Operations Section provides most of the technical or functional expertise for both scenario development and evaluation. This includes development of the Master Scenario Event List.


P

Term Description Participant Participants are the overarching group that includes all players, controllers,

evaluators, and staff members involved in conducting an exercise. Participant Feedback Form

Players and observers receive a Participant Feedback Form after the end of the exercise that asks for input regarding observed strengths and areas for improvement that players identified during the exercise. Providing Participant Feedback Forms to players during the exercise Hot Wash allows them to provide evaluators with their insights into decisions made and actions taken. A Participant Feedback Form also provides players the opportunity to provide constructive criticism about the design, control, or logistics of the exercise to help enhance future exercises. Information collected from feedback forms contributes to the issues, observations, recommendations, and corrective actions in the After Action Report/Improvement Plan.

Planning Conferences

Effective exercise design and development involve a combination of exercise planning conferences. These conferences bring together the full range of exercise stakeholders to discuss and agree on key aspects of the exercise’s design and development. Various factors—including exercise scope, type, and complexity—inform the types of conferences needed, and exercise planners should tailor the planning conference schedule to suit the particular nature of the exercise.

Planning Section The Planning Section is responsible for compiling and developing all exercise documentation. To accomplish this effectively, the Planning Section also collects and reviews policies, plans, and procedures that will be assessed in the exercise. This group is also responsible for planning exercise evaluation. During the exercise, the Planning Section may be responsible for developing simulated actions by agencies not participating in the exercise and for setting up a SimCell as required.

Player Players have an active role in preventing, responding to, or recovering from the risks and hazards presented in the scenario, by either discussing or performing their regular roles and responsibilities. Players initiate actions that will respond to and/or mitigate the simulated emergency.

Player Briefing A Player Briefing is held immediately before an exercise and addresses individual roles and responsibilities, exercise parameters, safety, badges, and any other logistical items. For a drill or full-scale exercise, Player Briefings typically occur in the assembly area.

Preparedness The actions taken to plan, organize, equip, train, and exercise to build and sustain the capabilities necessary to prevent, protect against, mitigate the effects of, respond to, and recover from those threats that pose the greatest risk to the security of the Nation.

Presidential Policy Directive 8 (PPD-8)

PPD-8: National Preparedness describes the Nation’s approach to preparing for the threats and hazards that pose the greatest risk to the security of the United States.

Prevention The capabilities necessary to avoid, prevent, or stop a threatened or actual act of terrorism.

Prevention Exercises Prevention exercises may focus on issues that pertain to information and intelligence sharing, credible threats, surveillance, and/or opposing force or Red Team activity.


P Term Description

Procedural Flow (ProFlow)

The ProFlow outlines a sequential flow of actions anticipated from participating organizations in response to a hypothetical situation during a prevention exercise. The ProFlow is used in conjunction with the MSEL to allow controllers and evaluators to track and monitor expected actions to ensure their completion at designated times. The ProFlow differs from the MSEL in that it contains only expected player actions and not controller-delivered injects.

Props Props are nonfunctional replications of objects. The presence or discovery of props requires certain actions by exercise players. Examples of props include simulated bombs, bomb blast debris (shrapnel), mannequins or body parts, and foam bricks and beams. Simulants that mimic the effects of chemical or radiological hazards or that cause a positive reading of an actual detection device are also considered props.

Protection The capabilities necessary to secure the homeland against acts of terrorism and manmade or natural disasters.


R

Term Description Recovery The capabilities necessary to assist communities affected by an incident to

recover effectively. Red Team A red team is the capability-based analytical or physical manifestation of the

adversary, which serves as an opposing force to exercise participants. Red Team Handbook The Red Team Handbook supplements the Controller/Evaluator Handbook

and the ExPlan and contains more detailed information about the red team, its planned operations, and the Universal Adversary it is built to physically represent.

Red Team Operators Red Team operators portray the physical entity of the adversary in a prevention exercise.

Red Team Planning Conference

The Red Team Planning Conference, conducted in preparation for a prevention exercise, is held to confirm safety redundancies, rules of exercise play, operational plans, and the exercise timeline.

Response The capabilities necessary to save lives, protect property and the environment, and meet basic human needs after an incident has occurred.

Root Cause Analysis When evaluating exercises, root cause analysis involves not merely identifying what issues emerged, but rather discovering the root causes of those issues. Root cause analysis enables exercise stakeholders to target how best to address areas for improvement and close capability gaps.

Rules of Exercise Play

In prevention exercises, the rules of exercise play is a comprehensive list of rules that provides specific guidance for the behavior of all exercise participants. These rules define the boundaries for exercise play; establish limits on red team activity; ensure that useful information is gathered; and most importantly, ensure participant safety.


S Term Description

Safety Controller The safety controller is responsible for monitoring exercise safety during exercise setup, conduct, and cleanup. All exercise controllers assist the safety controller by reporting any safety concerns. The safety controller should not be confused with the safety officer, who is identified by the incident commander during exercise play.

Scenario A scenario provides the storyline that drives an exercise to test objectives. The scenario selected for an exercise should be informed by the actual threats and hazards faced by the exercise stakeholders. The exercise scenario should realistically stress the delivery of core capabilities, providing a mechanism for testing objectives and assessing core capability levels and gaps.

Scope Scope is an indicator of the level of government or private-sector participation in exercise play, regardless of participant size. Scope levels include local, multilocal, regional (within a State), State, multistate, Federal, national, international, and private sector.

Seminar Seminars generally orient participants to, or provide an overview of, authorities, strategies, plans, policies, procedures, protocols, resources, concepts, and ideas. As a discussion-based session, seminars can be valuable for entities that are developing or making major changes to existing plans or procedures. Seminars can be similarly helpful when attempting to gain awareness of, or assess, the capabilities of interagency or interjurisdictional operations.

Senior Controller The senior controller is responsible for the overall organization of the exercise. The senior controller monitors exercise progress and coordinates decisions regarding deviations or significant changes to the scenario caused by unexpected developments during play. The senior controller monitors actions by controllers and ensures they implement designated and modified actions at the appropriate time. The senior controller debriefs controllers and evaluators after the exercise and oversees the setup and takedown of the exercise.

Simulation (1) An electronic simulation is a method for predicting the results of implementing a model over time. (2) Simulation of nonparticipating personnel and agencies is a technique for increasing realism in exercises.

Simulation Cell (SimCell)

A SimCell is used to generate injects for, receive player responses for, and provide information in place of nonparticipating organizations that would likely participate actively if exercise events were real. Physically, the SimCell is a working location for a number of qualified professionals who portray these nonparticipating organizations.

Simulators Simulators are control staff personnel who role play as nonparticipating organizations or individuals. They most often operate out of the SimCell, but they may occasionally have face-to-face contact with players. Simulators function semi-independently under the supervision of SimCell controllers, enacting roles in accordance with instructions provided in the MSEL. All simulators are ultimately accountable to the exercise director and senior controller.

Situation Manual (SitMan)

SitMans are provided for TTXs and games as the core documentation that provides the textual background for a multimedia, facilitated exercise. The SitMan supports the scenario narrative and serves as the primary reference material for all participants during conduct.


S Term Description

Special Effects Special effects are technical, mechanical, or electronic scenario enhancements. Special effects typically require trained and licensed personnel, special permission for use, and additional safety and/or security precautions. Examples include use of pyrotechnics or explosives.

Sponsor The sponsor is the primary funding organization for an exercise. Subject-Matter Expert (SME)

SMEs add functional knowledge and expertise in a specific area or in performing a specialized job, task, or skill to the exercise planning team. They help make the scenario realistic and plausible and ensure jurisdictions have the appropriate capabilities to respond. SMEs are ideal for the positions of controllers and evaluators.

Support Staff The exercise support staff includes individuals who are assigned administrative and logistical support tasks during the exercise (e.g., registration, catering).

Symptomatology Card

Symptomatology cards are provided to each actor in a response-focused exercise. Each card is unique, containing the signs and symptoms the actor will portray as well as information for medical providers. At a minimum, symptomatology cards should include vital signs; symptoms; trauma injuries; acting instructions (e.g., disorientation, emotional distress); and special needs (e.g., language barriers, physical limitations).


T

Term Description Tabletop Exercise (TTX)

A TTX is typically held in an informal setting intended to generate discussion of various issues regarding a hypothetical, simulated emergency. TTXs can be used to enhance general awareness, validate plans and procedures, rehearse concepts, and/or assess the types of systems needed to guide the prevention of, protection from, mitigation of, response to, and recovery from a defined incident. Generally, TTXs are aimed at facilitating conceptual understanding, identifying strengths and areas for improvement, and/or achieving changes in attitudes.

Training and Exercise Planning Workshop (TEPW)

A TEPW is usually conducted to create a Multiyear TEP. At a TEPW, stakeholders work together in a collaborative workshop environment to identify and set exercise program objectives based on core capabilities. Based on these program objectives, TEPW stakeholders develop a multiyear schedule of specific training and exercises.


U Term Description

Universal Adversary (UA)

In prevention exercises, the UA is a fictionalized adversary created by compiling known terrorist motivations, doctrine, and TTPs in live, virtual, and constructive simulations. The UA is based on realistic threats, but it is designed not to compromise actual intelligence.

Universal Adversary (UA) advisor

In prevention exercises, the UA advisor is responsible for providing the MSEL and red team developers with realistic and scenario guidance on the characteristics of the UA.


V

Term Description Venue A venue is the primary location of exercise conduct. Venue Control Cell (VCC)

The VCC is the location from where controllers run the exercise, in the event that the exercise occurs at multiple venues.


1 W

Term Description Whole Community A focus on enabling the participation in national preparedness activities of a

wider range of players from the private and nonprofit sectors, including nongovernmental organizations and the general public, in conjunction with the participation of Federal, state, and local governmental partners in order to foster better coordination and working relationships. Used interchangeably with “all-of-Nation.”

Workshop Although similar to seminars, workshops differ in two important aspects: participant interaction is increased, and the focus is placed on achieving or building a product. Effective workshops entail the broadest attendance by relevant stakeholders. Products produced from a workshop can include new standard operating procedures, emergency operations plans, continuity of operations plans, and mutual aid agreements. To be effective, workshops should focus on a specific issue, and the desired outcome, product, or goal must be clearly defined.

2 3


Acronyms and Abbreviations 4

Acronym/Abbreviation Description A/V Audio/Visual AAC After Action Conference AAR After Action Report AATW Advanced Analytic Techniques Workshop C/E Controller/Evaluator C&O Concept and Objectives CAP Corrective Action Program COOP Continuity of Operations Plan COSIN Control Staff Instructions DHS U.S. Department of Homeland Security EEG Exercise Evaluation Guide EndEx End of Exercise EOC Emergency Operations Center EvalPlan Evaluation Plan ExPlan Exercise Plan FE Functional Exercise FEMA Federal Emergency Management Agency FPC Final Planning Conference FSE Full-Scale Exercise HSEEP Homeland Security Exercise and Evaluation Program ICS Incident Command System IP Improvement Plan IPC Initial Planning Conference IPW Improvement Planning Workshop LLIS Lessons Learned Information Sharing ISEA Information Sharing Environment Analysis MPC Midterm Planning Conference MSEL Master Scenario Events List NED National Exercise Division NEP National Exercise Program NGO Nongovernmental Organization NIMS National Incident Management System POC Point of Contact PPD-8 Presidential Policy Directive 8 ProFlow Procedural Flow SimCell Simulation Cell SitMan Situational Manual


Acronym/Abbreviation Description SMART Simple, Measurable, Achievable, Realistic, and Task-Oriented SME Subject-Matter Expert SOP Standard Operating Procedure StartEx Start of Exercise THIRA Threat and Hazard Identification and Risk Assessment TEP Training and Exercise Plan TEPW Training and Exercise Planning Workshop TTP Tactics, Techniques, and Procedures TTX Tabletop Exercise UA Universal Adversary VIP Very Important Person XPA Extent of Play Agreement

5 6

homeland security exercise and evaluation …montanadma.org/sites/default/files/hseep...

Documents