development of training themes for joint, interagency,..(2010) ross, k. us army research

U.S. Army Research Institute For the Behavioral and Social Sciences

Research Note ####

Development of Training Themes for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operations

Karol G. Ross Cognitive Performance Group

Anna Grome

Klein Associates Division, ARA

Meagan C. Arrastia Cognitive Performance Group

Brooke Schaab

US Army Research Institute

James Ong Stottler Henke Associates, Inc.

David Spangler

Global Innovation and Design, Inc.

May 2010

Approved for public release; distribution is unlimited.

U.S. Army Research Institute for the Behavioral and Social Sciences A Directorate of the Department of the Army Deputy Chief of Staff, G1 Authorized and approved for distribution:

MICHELLE SAMS, Ph.D.

Director

Technical review by

NOTICES

REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.

1. AGENCY USE ONLY (Leave blank)

2. REPORT DATE

February 2010

3. REPORT TYPE AND DATES COVERED

October 15, 2006 through October 31, 2009

4. TITLE AND SUBTITLE

Development of Training Themes for Joint, Interagency, Intergovernmental and

Multinational (JIIM) Operations

5. FUNDING NUMBERS

W91WAW-07-C-0067

6. AUTHORS - Karol G. Ross, Anna Grome, Meagan C. Arrastia, Brooke Schaab, James Ong, David Spangler

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

Stottler Henke Associates, Inc.

951 Mariner’s Island Blvd., Suite 360

San Mateo, CA 94404

8. PERFORMING ORGANIZATION REPORT NUMBER

Phase II Research Report No. 345

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

ERDC Contracting

Champaign Office

PO Box 9005

Champaign, IL. 61826

10. SPONSORING/MONITORING AGENCY REPORT NUMBER

11. SUPPLEMENTARY NOTES

NONE

12a. DISTRIBUTION/AVAILABILITY STATEMENT

Distribution authorized to US Government only; Proprietary Information (DFARS – SBIR Data Rights). Other requests for this document shall be referred to the DoD Controlling Office or the DoD SBIR Program Office.

12b. DISTRIBUTION CODE

13. ABSTRACT (Maximum 200 words)

The contemporary operating environment requires a new emphasis on collaboration across services, across government agencies and with non-governmental agencies and host nations to create global stability in the interest of national security. This report documents an effort to capture the expertise of experienced military and non-military players, combined with operational lessons learned and previous research to insure that training and education will prepare our forces for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operations. First, we reviewed documents that reflected lessons learned and insights into these operations. Second, we conducted interviews with experts in collaboration between the military and other organizations, as well as with host nation officials and citizens. Third, we analyzed interview transcripts to derive themes that expressed the high-level cognitive skills in these operations. We identified these themes: Understand the situation within its historical, regional, and cultural context; Understand the other participants; Shift perspective; Establish and maintain common ground; Build capability to affect the situation; Visualize the operation; Support information exchange; and Maintain flexibility. We describe our six-step thematic analysis method and provide analysis tables to clarify the process. Finally, we administered surveys and held focus groups to validate the themes with experienced Civil Affairs officers.

14. SUBJECT TERMS

Stability Operations, Joint Interagency Intergovernmental and Multinational Training, Training Themes, HASE, JIIM, Thematic Analysis

15. NUMBER OF PAGES

186

16. PRICE CODE

17. SECURITY CLASSIFICATION OF REPORT

UNCLASSIFIED

18. SECURITY CLASSIFICATION OF THIS PAGE

UNCLASSIFIED

19. SECURITY CLASSIFICATION OF ABSTRACT

UNCLASSIFIED

20. LIMITATION OF ABSTRACT

UNLIMITED

NSN 7540-01-280-5500 Computer Generated STANDARD FORM 298 (Rev 2-89)

ii

Research Note ####

Development of Training Themes for Joint, Interagency,

Intergovernmental and Multinational (JIIM) Operations

Karol G. Ross

Cognitive Performance Group

Anna Grome

Klein Associates Division, ARA

Meagan C. Arrastia

Cognitive Performance Group

Brooke Schaab

US Army Research Institute

James Ong

Stottler Henke Associates, Inc.

David Spangler

Global Innovation and Design, Inc.

Fort Leavenworth Research Unit

Stanley M. Halpin, Chief

U.S. Army Research Institute for the Behavioral and Social Sciences

2511 Jefferson Davis Highway, Arlington, Virginia 22202-3926

May 2010

iii

DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY,

INTERGOVERMENTAL AND MULTINATIONAL (JIIM) OPERATIONS

ACKNOWLEDGEMENTS

The authors would like to thank Partners International Foundation who organized the

Health African Scenarios Exercise workshop and shared workshop data with us. We also

thank faculty at Joint Forces Staff College, members of the 91st Civil Affairs Battalion at

Ft. Bragg, the 350th

Civil Affairs Command and others who shared their expertise and

perspectives on Joint, Interagency, Intergovernmental, and Multinational stability and

support operations.

iv



EXECUTIVE SUMMARY

Research Requirement:

The contemporary operating environment for U.S. military forces requires a new

emphasis on collaboration across services, across government agencies, with non-

governmental agencies, and with host nations in order to create global stability in the

interest of our national security. While operations are driven by U.S. national objectives,

our interests are increasingly dependent on defining common goals with others to support

the security and stability of a wide range of regions and nations in which we currently are

engaged or soon will find ourselves conducting operations. The skills to determine

relevant objectives and take effective actions in this collaborative environment have long

been a part of U.S. military capabilities, but these skills are now more in the foreground

of operations and are required of a wider range of personnel from the tactical to the

strategic level. New and emerging doctrine directs the nature of these interactions for the

U.S. military. While doctrine provides guidance and structure, it does not capture the

expertise required to successfully apply that doctrine. The purpose of this research effort

was to identify the high-level cognitive skills required for effective performance in Joint,

Interagency, Intergovernmental and Multinational (JIIM) environments to serve as

themes to support training development.

Procedure:

The approach to theme development included three steps. First, we identified and

analyzed relevant documents that reflected lessons learned or other insights into

operations in JIIM environments. Second, we conducted a series of interviews with

subject matter experts whom we had identified as experienced in operations that required

collaboration among the military and other organizations, as well as host nation officials

and citizens. Third, we analyzed the interview transcripts to derive themes that expressed

the high-level cognitive skills evident in the experiences. To represent our findings, we

combined interview findings with insights gained from the documents to produce a

matrix of themes.

The method for theme validation included a quantitative and qualitative approach.

First a survey instrument consisting of 54 items was developed from the previous

interviews with JIIM operators in order to explore the degree of consensus among a

different set of SMEs on the value of the eight themes identified. The items are

statements describing the skills that make up each theme with the answer choices on a 5-

point Likert scale ranging from very important to mission success to very unimportant to

mission success. In addition to the survey, a focus group was conducted to discuss the

themes with a subset of the individuals that have actually had to employ such skills in

v

JIIM environments. Feedback about the themes in their own words, as well as examples

of each theme, added richness to the data that allowed the themes to be further explored.

Findings:

A total of eight themes emerged from the data. The themes are listed below. Each

theme is presented as a cognitive act that the practitioner in the domain must perform. In

addition, the set of themes was integrated into a descriptive model to show their inter-

relationships and application over the course of an operation.

1. Understand the situation within its historical, regional, and cultural context

2. Understand the other participants

3. Shift perspective

4. Establish and maintain common ground

5. Build capability to affect the situation

6. Visualize the operation

7. Support information exchange

8. Maintain flexibility

The themes were validated as important to the vast majority of the SMEs surveyed.

They recommended few changes to the wording of the definitions and accompanying

cognitive challenges.

Utilization and Dissemination of Findings:

This research produced a set of high-level cognitive skills in the form of themes that

will be used to build a training product. The scenario-based tutorials in the product will

support the development of a common understanding of successful collaboration in JIIM

environments in a training audience that varies in levels of expertise. Use of the product

will prepare participants to engage in more difficult training and exercises based on the

understanding of the doctrine, processes and skills needed for successful collaboration

which they will acquire.

vi


INTERNATIONAL AND MULTINATIONAL (JIIM) OPERATIONS

CONTENTS

INTRODUCTION .............................................................................................................. 1

DEVELOPMENT OF TRAINING THEMES.................................................................... 2

Method ............................................................................................................................ 3

Research Question ...................................................................................................... 3

Participants .................................................................................................................. 3

Data Collection Procedure .......................................................................................... 4

Analysis Plan .............................................................................................................. 4

Results ............................................................................................................................. 5

VALIDATION OF THEMES............................................................................................. 8

Method ............................................................................................................................ 8

Research Question ...................................................................................................... 8

Participants .................................................................................................................. 8

Data Collection Procedure .......................................................................................... 8

Analysis Plan .............................................................................................................. 9

Results ............................................................................................................................. 9

Survey Results ............................................................................................................ 9

Focus Group Results ................................................................................................. 10

DISCUSSION ................................................................................................................... 10

REFERENCES ................................................................................................................. 12

APPENDIX A: LIST OF PARTICIPANTS AND DOCUMENTS ............................... A-1

APPENDIX B: EXAMPLES OF DATA ANALYSIS TABLES ................................... B-1

APPENDIX C: THEMES FOR OPERATIONS IN JIIM ENVIRONMENTS .............. C-1

APPENDIX D: THEME VALIDATION SURVEY ...................................................... D-1

APPENDIX E: THEME VALIDATION FOCUS GROUP GUIDE ............................... E-1

APPENDIX F: THEME VALIDATION SURVEY RESULTS ..................................... F-1

vii

LIST OF FIGURES

Figure 1. Model of High-Level Cognitive Skills Required for Joint, Interagency,

Intergovernmental and Multinational Operations ............................................................... 7

LIST OF TABLES

Table 1. Summary of Survey Findings for Validation of Themes .................................... 10

1



INTRODUCTION

The contemporary operating environment for U.S. military forces requires a new

emphasis on collaboration across services, across government agencies, with non-

governmental agencies, and with host nations in order to create global stability in the

interest of our national security. While operations are driven by U.S. national objectives,

our interests are increasingly dependent on defining common goals with others to support

the security and stability of a wide range of regions and nations in which we currently are

engaged or soon will find ourselves conducting operations. The skills to determine

relevant objectives and take effective actions in this collaborative environment have long

been a part of U.S. military capabilities, but these skills are now more in the foreground

of operations and are required of a wider range of personnel from the tactical to the

strategic level.

New and emerging doctrine directs the nature of these interactions for the U.S.

military.1 While doctrine provides guidance and structure, it does not capture the

expertise required to successfully apply that doctrine. The purpose of this research effort

was to identify the high-level cognitive skills required for effective performance in Joint,

Interagency, Intergovernmental and Multinational (JIIM) environments.

We addressed this requirement by identifying the expertise of experienced members

of the military and non-military players in these settings. We combined that

understanding of expertise with operational lessons learned and previous research in

order to build an integrated set of themes that reflect the skills required for collaboration

in the JIIM environment. Expertise in operations in JIIM environments is concentrated

within certain elements of the military forces such as Civil Affairs, combat arms units

who have successfully engaged in counterinsurgency operations, members of Provincial

Reconstruction Teams, and those who have served in Joint or State Department positions.

Additionally, insights into expert performance are also found in non-government

organizations that are part of the collaborative efforts these operations require. Our

approach to understanding expert performance was to conduct interviews across this

range of experienced people using a critical incident approach and then to identify themes

that can inform the focus of training and education. The purpose of this report is to

document the results which will be used for the development of a training product

intended to build an understanding of this domain across a diverse audience and to

improve collaboration in the JIIM environment. The process by which we turned the

expertise into training themes and then validated the themes is described here in detail to

stimulate a consistent approach to developing themes as the basis of training development

in complex domains. We present the themes in both a matrix and integrated into a model

that indicates the application of the themes in operation.

1 See for example U.S. Army Field Manual 3-07 Stability Operations, October 2008; Joint Publication 3-

07.3, Peace Operations, 17 Oct 07; and DoD Directive 3000.05, Military Support for Stability, Security,

Transition, and Reconstruction (SSTR) Operations, 28 Nov 05.

2

DEVELOPMENT OF TRAINING THEMES

The concept of using themes to guide cognitive development through situated

learning was introduced in the research literature as part of the constructivist approach to

instruction. The goal of theme-based instruction is to go beyond superficial familiarity

with concepts and facts to mastering the conceptual complexity of an area. Thematic

instruction aids the learner in going over the same problem from different viewpoints.

This instructional strategy is said to promote cognitive flexibility in a domain of practice

and avoids counter productive training, i.e., training that inhibits transfer of knowledge to

field performance. Examples of counter-productive training include oversimplification of

concepts, linear presentation of material which ignores the inter-related nature of

concepts, using one exemplar leading to a student perception of one right answer for

complex problems, and using simple analogies for complex systems (Spiro et al, 1992).

Theme-based instruction when applied as a method to help the student explore situations

from different perspectives produces a more cognitively complex understanding of a

domain more quickly, as well as more flexible transfer of concepts to field performance.

Theme-based training of cognitive skills has since been successfully implemented

for the military. Spiro and Jehng (1990) provided a model of Cognitive Flexibility to

guide learning which also proposed the concept of a deeper level of cognitive insight

gained by theme-based exploration of situations or scenarios. They asserted that this

method of learning supported the ability to spontaneously restructure the knowledge

gained in adaptive response to changing situations. Spiro, Feltovich, Jacobson, &

Coulson, (1992) demonstrated this approach with adult learners in the area of tactical

thinking. They used one case study, the battle of Chancellorsville—a case study often

used in U.S. Army tactical education—to support theme-based learning. Their efforts

were focused on demonstrating that a deeper conceptual understanding of tactics could be

gained and generalized to new settings by using a theme-based structure to examine a

scenario or case in depth. During the development of the ―Think like a Commander‖

(TLAC) training environment, a number of high-level cognitive skills common across

expert tacticians were summarized to guide deliberate practice in this domain. The

approach of TLAC is to explore a situation from the multiple perspectives. In line with

the earlier examples of this type of instructional strategy cited above, the high-level

constructs were dubbed ―themes‖ (Ross, & Lussier, 1999). This successful theme-based

approach to training thinking skills (Lussier, Shadrick, & Prevou, 2003) has since been

used to produce training for military crisis management thinking skills in the Red Cape

tool (Shadrick, Schaefer, & Beaubien, 2007).

Theme-based training as an approach to situated or scenario-based training and

education is dependent on generating acceptable themes that reflect the high-level

cognitive skills in a domain. While a theme-based approach has been successful in

military training in limited applications, the approach for identifying relevant and useful

themes to guide such training has not been documented.

Thematic analysis is generally used to pull meaning from the data in order to

understand a phenomenon or specific aspects of a phenomenon that are psychological

(such as the experience of choosing cosmetic surgery or choosing treatment during

3

serious illness) or social in nature (such as patterns of drug abuse in a particular segment

of society). Our use of thematic analysis is phenomenological in nature as we seek to

understand particular situations from the point of view of subject matter experts, but is

aimed specifically at understanding their high-level cognition for the purpose of

supporting learning that will go beyond the procedural. In the qualitative research

literature, ―[t]hematic analysis is widely used, but there is no clear agreement about what

thematic analysis is and how you go about doing it‖ (Braun, & Clarke, 2006, p. 79).

There are a number of methods that can generally be thought of as thematic analysis

(such as conversation analysis, interpretative phenomenological analysis, discourse

analysis, and narrative analysis). Our approach is detailed below.

Method

Our approach included three steps. First, we identified and analyzed relevant

documents that reflected lessons learned or other insights into JIIM operations. Second,

we conducted a series of interviews with subject matter experts whom we had identified

as experienced in operations that required collaboration among the military and other

organizations, as well as host nation officials and citizens. Third, we analyzed interview

transcripts to derive themes that expressed the high-level cognitive skills evident in the

experiences. To represent our findings, we combined interview findings with insights

gained from the documents we had reviewed to produce a matrix of themes.

Research Question

The focus of this research effort is on the high-level cognitive skills that underlie

successful performance in JIIM operations. Our intention was to reflect the findings of

critical incident accounts and the perceptions of the individual as they pertained to

successful performance.

Participants

We recruited interview participants with experience in some aspect of JIIM

environments. Our goal was to compare multiple perspectives at tactical and operations

levels. Participants in the military or associated with military organizations were

volunteers. Participants from other organizations were compensated for their time as

subject matter experts (SMEs). We interviewed a total of 15 military or military-related

SMEs; three African nationals, one affiliated with a non-government organization and

two with a government health organization; and five SMEs with Department of State

experience for a total of 23 interviews. Data from one additional military interviewee was

not used.

Prior to the interviews we reviewed a number of relevant documents that capture the

skills required of participants in JIIM operations. (See Appendix A for a complete list.)

We reviewed 1) JIIM lessons learned (Agrait, & Loughran, 2007); 2) transcripts of three

days of meetings of the HASE (Healthy Africa Scenarios Exercise) Workshop in Ghana,

Africa held January 21-25, 2008; 3) a briefing on the Joint, Interagency, and

4

Multinational Planner‘s Course from the Joint Forces Staff College; 4) a report on

cognitive challenges in operations other than war (Miller et al., 2003); 5) a report that

reviewed the cognitive challenges of working across military organizations (O‘Dea et al.,

2006); 6) a report on the elements of cross-cultural competence required for military

operations (Ross, 2008); and 7) a report on modeling cross-cultural competence in the

U.S. Army (McCloskey, Grandjean, & Ross, in publication).

Data Collection Procedure

A semi-structured interview protocol was used that was based on the Critical

Decision Method (CDM; Crandall, Klein, & Hoffman, 2006; Hoffman, Crandall, &

Shadbolt, 1998). All interviews were digitally recorded, and all interviews were

transcribed for analysis.

Analysis Plan

The guiding principle in our analysis was to find the elements of performance that

are key to successful planning and operations in JIIM environments. Key aspects were

not determined by a simple prevalence count, i.e., how many times certain performance

elements were mentioned, but by the emphasis placed on how the operations are

accomplished or how they fail, i.e., expert strategies. Existing documents that had

previously examined operations in terms of expertise or lessons learned helped us

determine the emphasis to place on elements of performance.

Our approach was inductive, i.e., the themes identified are linked strongly to the data

gathered specifically for this project or to reports that are based on data about

performance in JIIM environments. We developed questions to help us indirectly assess

the elements of expertise by gathering rich examples of performance and insights and

strategies that drove performance. In this approach, the themes may bear little relation to

the actual questions asked in the interviews. One assumption of the research is that as

expertise develops, people are not generally able to articulate the how and why of their

cognitive performance. Therefore, the analysis is grounded in the specifics of the data.

The data has been collected so as to expose the cognitive challenges and strategies

inherent in situations without directly asking participants why they did things the way

they did. The inductive process does not try to fit the data into a pre-conceived

framework, but creates the framework from the data. The process also provides a rich

description of the cognitive challenges and strategies as opposed to a description of the

procedures involved in planning and operations. Analysis requires interpretation of

specific data to general themes. Specifics are coded into categories first within each

individual data item (interview or document) and then summarized across the data set.

The first step in the process was to read through the data set to immerse the analysis

team in the findings. The research question informs how reading proceeds. Both analysts

made notes of interesting ideas in the data and documented those that might be converted

into coding categories. Review of the selected documents was interspersed with reviews

of the interview transcripts. The selected documents were not theoretical background, but

5

were previous findings based on interviews that had been analyzed in a thematic manner

and lessons learned generated by experienced participants. Two analysts experienced in

thematic analysis for cognitive performance and experienced in the subject area

conducted the first step. The outcome was a set of preliminary themes from each analyst

and notes to connect each potential theme to the interviews and documents. During this

step more themes were generated than are retained during the final analysis.

The second step was to generate initial themes of expert high-level cognitive skills.

The two analysts independently reviewed each other‘s themes and rationale for each and

then discussed the overlap and wording for each theme. The outcome of this step was an

agreed upon set of codes that were to serve as preliminary themes.

The third step was to convert the preliminary themes into a representation to help

the team understand the nature of each theme. A three-column table was constructed to

present an initial name for each theme, a one-paragraph definition of the theme, and a list

of the cognitive challenges associated with the theme. An example of this can be seen in

Appendix B.

The fourth step was to review the themes more exhaustively against the data set to

identify areas needing refinement. In this step, both analysts re-read the data set. A third

analyst was added to the team and also reviewed each data item (report or interview).

Each item was coded according to the themes that had been developed, and an additional

field was created for ―other‖ themes or interesting elements that emerged during the

complete review. The number of instances of a theme found in the data set, or prevalence,

does not necessarily mean a theme is more crucial. Our inclusion of other documents in

the data set allowed us to examine the nature of performance in JIIM environments in

general across a wide range of data and previous analysis in order to make judgments

about the criticality of any one aspect of performance and judge whether to retain

inclusion of a theme in the representation.

The fifth step was to refine the name of each theme and the corresponding definition and

cognitive challenges for each. Our goal was to have a name for each theme that was easy

to remember and to create definitions and explanations of cognitive challenges that were

drawn from the data, i.e., using the words of the experienced interview participants or

findings in the documentation to the extent possible to define and explain the themes.

The sixth step was to create a model of expertise in JIIM planning and operations that

reflected the high-level cognitive skills of experienced practitioners as opposed to a

procedural representation. Our goal was to reflect the flow of cognitive performance and

inter-relationship among the themes.

Results

The interview participants were grouped into military (15), Africa Nationals (3), and

Department of State personnel (4). In the interviews, we attempted to concentrate on

experiences that were based in relief, development, or reconstruction operations. Some

experiences also contained aspects of security operations. Our primary focus was on

6

understanding military performance in JIIM operations which accounted for the majority

of interview participants.

Our analysis resulted in a total of eight themes. The themes are listed below. Each

theme is presented as a cognitive act that the practitioner in the domain must perform.

9. Understand the situation within its historical, regional, and cultural context

10. Understand the other participants

11. Shift perspective

12. Establish and maintain common ground

13. Build capability to affect the situation

14. Visualize the operation

15. Support information exchange

16. Maintain flexibility

The themes are presented in a table format including the cognitive challenges for

each at Appendix C.

The construction of our representation was heavily influenced by our previous work

with Klein‘s macrocognition framework for understanding high-level cognitive

performance (Klein et al. 2003). The elements of macrocognition were used to help

describe the cognitive challenges we discovered in this domain. In the cognitive

challenges column we entered macrocognitive labels for the overarching abilities we

believed were associated with each theme or we used elements found to be important to

cross-cultural competence (e.g., sensemaking, decision making or willingness to engage)

as documented in McCloskey et al, in publication. We constructed items for each

challenge that are specific manifestations of the macrocognitive or cultural elements in

terms of skills and challenges in this specific domain (e.g., ―Understand local/regional

and organizational power structures‖ or ―See the situation from another‘s point of view to

influence and predict behavior‖).

We indicated the high-level skills in the order in which they occur and labeled their

relationships in performance. The themes representation also indicates a progression in

complexity from initial understanding of the situation to maintaining flexibility during

operations.

We created a graphic descriptive model of the themes as a high-level depiction of the

cognitive skills that support effective performance in the JIIM environment. The model

captures how the high-level skills occur in progression as planning and implementation

proceed, but also are mutually supportive and require reiteration throughout operations.

In addition, some actions are precursors of others in that they set the stage for success in

applying the subsequent skill. See Figure 1 below.

7

Figure 1. Model of High-Level Cognitive Skills Required for Joint, Interagency, Intergovernmental and Multinational

Operations

8. Maintain flexibility across interactions to understand and adapt new

approaches and to be resilient when attempts to understand others or

implement plans are not immediately successful.

Provide knowledge to

3. Shift perspective –

Understand how others

see specific aspects of the

situation and see your

organization.

4. Establish and maintain

common ground – What

goals can you work on

together that are mutually

beneficial?

1. Understanding the

situation within its

historical, regional, and

cultural context – Learn

to read these types of

situations.

6. Visualize the

operation including 2d

and 3d order

consequences

7. Support of

information exchange

– understand

communication

preferences

5. Build the capacity

to affect the situation

by using all assets

and expertise across

organizations


other participants in the

specific situation –

Who are they? How do

they do business? What

has their role been in

this situation?

Is the basis for effective

Must be implemented as you Provide knowledge to

Provide

knowledge to

8

VALIDATION OF THEMES

Method

The method for this phase of the project had two steps, one quantitative and one

qualitative. The intent was to investigate the validity of the eight themes for successfully

operating in a JIIM environment. First a survey instrument consisting of 54 items was

developed from the previous interviews with participants in JIIM environments in order

to explore the degree of consensus among a different set of SMEs on the value of the

eight themes identified. See Appendix D for a complete list of the survey items. The

items are statements describing the skills that make up each theme with the answer

choices on a 5-point Likert scale ranging from very important to mission success to very

unimportant to mission success. In addition to the survey, a focus group was conducted to

discuss the themes with a subset of individuals that have actually had to employ such

skills in JIIM environments. Feedback about the themes in their own words, as well as

examples of each theme, added richness to the data that allowed the themes to be further

explored. The survey data was analyzed in order to find out whether or not the themes are

meaningful to experts who have worked in JIIM environments and if there was consensus

about the importance of the themes to operations. The focus group transcripts were

analyzed for the denial or affirmation of the themes‘ utility as descriptions of effective

performance in JIIM.

Research Question

Our research question was ―Do these themes have utility as descriptions of effective

performance in the JIIM environment?‖ This question was addressed with the survey

described above. The focus group was intended to provide us with incidents in which the

skills described were crucial to mission success.

Participants

We recruited Civil Affairs personnel with experience in many JIIM environments.

Eighteen reservist Civil Affairs officers and NCOs were surveyed. The mean age of the

sample was 47 years and the average number of years in service was 23. The sample

consisted of three Colonels, eight Lieutenant Colonels, two Majors, three Sergeants First

Class, one Master Sergeant, and one Staff Sergeant with 72% of the sample being

officers. Of the 18 participants, six were asked to volunteer to participate in a focus group

based on the extent of their experience working in JIIM environments.

Data Collection Procedure

All participants signed the informed consent form, which were collected before the

administration of the surveys, as well as the beginning of the focus groups. A semi-

structured interview protocol was used for the focus groups which consisted of first

describing the themes followed by probing questions to determine the importance of each

9

theme and the relevance of the statements used to describe theme. Critical incidents in

which the themes were crucial to the success of missions were also elicited. The focus

groups were digitally recorded and transcribed for analysis.

Analysis Plan

The survey data was analyzed to determine the degree of consensus among the

respondents on the value of the eight themes. This analysis consisted of five steps:

1. The data was entered into SPSS, data-analysis software.

2. The data was cleaned and checked for any errors during data entry.

3. The mean answer choice and standard deviation for each question was calculated.

A table was constructed listing the frequency of each answer choice, the mean,

and the standard deviation of each question.

4. The individual questions were combined into their thematic groups. Once this was

done the themes could be reported on instead of the individual questions.

5. The mean answer choice and standard deviation for each theme was calculated.

This indicated the general consensus of the importance of each theme to mission

success within a JIIM environment. See Table 2 below.

The six participants who subsequently formed the focus group were asked to identify

incidents that illustrated one or more themes. After the focus group recording was

transcribed:

1. The themes were read through to re-familiarize the researcher.

2. The transcripts of the focus groups were read through to familiarize the

researcher with the data.

3. The transcripts were analyzed for each theme, one at a time. Incidents mentioned

were coded and catalogued under each theme they described.

Results

Survey Results

Each of the eight themes was validated with the use of the 54 items in the survey.

Each item was rated on a scale from 1 to 6 with 1 being very important and 6 being very

unimportant. Each item was rated as important on average with means ranging from 1.1

to 2.1. See Appendix F for a full listing of means and standard deviations. The rate of

importance for each theme ranged from 91-100% with Shifting Perspective being

reported as important or very important by an average of 91% of the sample and

Maintaining Flexibility being reported as important or very important by 100% of the

sample. The percentage of participants that found each theme important or very important

is listed in Table 2 below. Although the sample size does not allow for generalization of

the results, the findings provide some confidence that these themes are applicable for

JIIM training.

10

Table 1. Summary of Survey Findings for Validation of Themes

Theme Related

Item

Numbers

Total

Items

% of Participants that

Found the Theme ―Very

Important‖ or ―Important‖

for JIIM Operations

Understand the situation within its

historical, regional, and cultural

context

1-9 9 95.1%

Understand the other participants 10-16 7 96.8%

Shift perspective 17-24 8 91.0%

Establish and maintain common

ground

25-31 7 99.2%

Build capacity to affect the situation 32-38 7 95.2%

Visualize the operation 39-44 6 94.4%

Support information exchange 45-49 5 91.1%

Maintain Flexibility 50-54 5 100%

Focus Group Results

In order to further investigate the validity of the JIIM themes and improve them we

went over the phrasing of the themes and other data in the matrix, as well as gathered

example. Our analysis of the focus groups‘ transcripts confirmed that the challenges and

training needs reflected in the themes were meaningful to experienced practitioners. We

also revised the wording, but generally the wording was accepted by the focus group

members. The eight themes were accepted by all the members of the Civil Affairs sample

and the participants of the focus groups expanded on each theme by relating an example

from their experience in theater. They agreed that in order to better work in JIIM

environments in the future, training needs to be targeted towards specific issues that fall

within these different themes.

DISCUSSION

The themes we derived from interviews with experienced personnel and documents

in the area of JIIM operations are appropriate for guiding training development relating to

the high-level cognitive skills needed for effective performance in a collaborative

environment. Themes are used in situated learning designs to increase the learner‘s

ability to see a situation from multiple perspectives. Secondly, they are used to reinforce

habits of thought that are consistent with expert performance to guide deliberate practice

of these thought processes to accelerate expertise. While theme-based situated learning

has been used successfully to teach advanced concepts, we believe that the potential for

accelerating the acquisition of entry-level knowledge is equally as important. Too often

entry-level knowledge is taught in a manner that is boring and decontextualized. These

methods result in a lack of retention and a lack in the student‘s ability to generalize

11

knowledge to new settings and to higher-level learning, essentially requiring the student

to re-learn introductory information later in the education or training process. The themes

will be used to support the development of scenario-based tutorials which will support

learning for a training audience of mixed levels of experts. Building scenarios around the

themes will help learners at different levels understand the basis for successful

implementation of doctrine and processes.

12

REFERENCES

Agrait, R. & Loughran, J. J. (2007). Lessons Learned from Joint, Interagency

International and Multinational (JIIM) Workshops and Papers. Vienna, VA:

ThoughtLink, Inc.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative

Research in Psychology, 3, 77-101.

Crandall, B., Klein, G., & Hoffman, R. R. (2006). Working minds: A practitioner’s guide

to cognitive task analysis. Cambridge, MA: The MIT Press.

Hoffman, R. R., Crandall, B. W., & Shadbolt, N. R. (1998). Use of the critical decision

method to elicit expert knowledge: A case study in cognitive task analysis

methodology. Human Factors, 40(2), 254-276.

Klein, G., Ross, K. G., Moon, B., Klein, D. E., Hoffman, R. R., Hollnagel, E. (May/June

2003). Macrocognition. IEEE Intelligent Systems, 81-85.

Lussier, J. W., Shadrick, S. B., & Prevou, M. I. (2003). Think Like a Commander

prototype: Instructor’s guide to adaptive thinking. (Research Product 2003-02).

Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social

Sciences.

McCloskey, M., Grandjean, A., & Ross, K. (in publication). Assessing learning and

development in Army cross-cultural competence. (Phase 1 SBIR Report.)

Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences.

Miller, T. E., Phillips, J. K., Battaglia, D. A., Wiggins, S. L., Baxter, H., Mills, J. A., &

Ross, K. G. (2003). Collaborative online training for operations other than war

(OOTW) (Final Technical Report for Contract #F33165-00-C-6003 Air Force

Research Laboratory, Mesa, AZ). Fairborn, OH: Klein Associates Inc.

O'Dea, A., Ross, K.G., McHugh, A, Phillips, J.K., Throne, M.H., & McCloskey, M. &

Mills, J.A (2006). Global teams: Enhancing the performance of multinational staffs

through collaborative online training (Research Report 1849, AD A469425).

.Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social

Sciences

Ross, K. G. (2008, May). Toward an operational definition of cross-cultural competence

from interview data. Patrick AFB, FL: Defense Equal Opportunity Management

Institute (DEOMI). (Available from the Cognitive Performance Group

([email protected]).

mailto:[email protected]

13

Ross, K. G., & Lussier J. W. (1999). A Training Solution for Adaptive Battlefield

Performance. Proceedings of the Interservice/Industry Training, Simulation &

Education Conference (I/ITSEC), 1999. Orlando FL.

Shadrick, S. B., Schaefer, P. S., & Beaubien, J. (2007). Development and Content

Validation of Crisis Response Training Package Red Cape: Crisis Action Planning

and Execution (Research Report 1875). Alexandria, VA: U. S. Army Research

Institute for the Behavioral and Social Sciences.

Spiro, R. J., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1992). Cognitive

Flexibility, constructivism, and hypertext: Random access instruction for advanced

knowledge acquisition in ill-structured domains. In T. Duffy, & D. Jonassen (Eds.).

Constructivism and the Technology of Instruction: A Conversation (pp.57-75).

Hillsdale, NJ: Lawrence Erlbaum Associates.

Spiro, R.J., Feltovich, P.J., Coulson, R.L., Jacobson, M.J., Durgunoglu, A., Ravlne, S., &

Jehng, J. (1992). Knowledge acquisition for application: Cognitive flexibility and

transfer of training in ill-structured domains (ARI Research Note 92-21).

.Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social

Sciences.

A-1

APPENDIX A: LIST OF PARTICIPANTS AND DOCUMENTS

A-2

Experience Base Organization Point of View

1 Iraq, Afghanistan Dutch

Marines

NATO

Multinat‘l Ops; Mentoring/Training of

Iraqi Ministry of Interior/Defense

Leadership; Gov‘t infrastructure building,

2 Mozambique US Army Working in Mozambique with the local

NGO and host nation to build HIV/AIDS

laboratories. Embassy Affairs; Security

Cooperation; NGO/IGO Coordination;

Reconstruction Project Mgmt.

3 Afghanistan US Army Studied police reform in Afghanistan

through interagency interaction.

4 Iraq US Army Worked as Chief of Transportation in Iraq.

5 Afghanistan US Army Worked with the UN and the host nation

to stabilize the area.

6 Iraq, Afghanistan US Marines

7 Ethiopia, Kenya US Army CAO working with the host nations in

Ethiopia and Kenya to provide

humanitarian assistance.

8 Kenya US Army Working in Kenya with the embassy and

host nation to fix water sanitation.

9 Djibouti, Ethiopia,

Yemen,

Philippines

US Army CAO worked with local tribes to provide

humanitarian assistance.

10 Afghanistan,

Niger, Kenya, Iraq

US Army COA working in Afghanistan with host

nation to provide humanitarian assistance.

11 Iraq, Chad US Army CAO working in Chad and Iraq with DOD

and the State Department.

12 Iraq, Afghanistan US Army CAO working in Iraq and Afghanistan

with NGOs.

13 Iraq, Kenya,

Afganistán,

Senegal, Cameron

US Army COA working in Afghanistan with host

nation to provide humanitarian assistance.

14 Afghanistan, HOA US Army In Afghanistan, he worked with a local

NGO to provide a medical clinic. In the

HOA, he worked with the State Dept. and

the Dept. of Defense.

15 Afghanistan, Iraq US Army Worked with a local NGO in Afghanistan

to provide for Pakistan refugee camps and

educate villagers on irrigation and

agriculture. Worked with another NGO in

Iraq to provide aid in Baghdad.

16 Honduras US Army

Corps of Eng.

Reconstruction and Stabilization Project

Mgmt.

A-3

17 Afghanistan Dutch

Marines, NL

MOD

Multinat‘l Strategic Ops and Intel; Int‘l

Political Advisement

18 2 African Nations,

Central and S.

America

State Dept. State Dept representative that met to

develop COA with military personnel.

19 Haiti, Iraq UN , State

Dept.

Representative

Worked with the UN on environmental

missions in Haiti and Iraq.

20 Central Asia State Dept.

Representative

Worked with the host nations on

environmental missions in Central Asia.

21 Iraq State Dept.

Representative

State Dept. Representative on a

multinational committee to exchange

information in the Green Zone of Iraq.

22 Lithuania US Embassy Worked with the host nation and the State

Department.

23 Rwanda International

Non-

Government

Organization

Worked in a political-military

organization with local NGOs to obtain

humanitarian assistance in Rwanda during

the genocide of 1994.

24 Ghana African

Health

Research

Center

African national working with NGOs and

US Army in health research laboratory.

25 Ghana International

Non-

Government

Organization

African national working with US Army

in health research laboratory.

Other Materials

1 ―Lessons Learned from Joint, International, and Multinational (JIIM) Workshops and

Papers‖– ThoughtLink, Inc. (Produced as part of this project)

2 Transcript of the Healthy Africa Scenarios Exercise Workshops January 21-25, 2008

3 JIMPC General Overview Briefing – Joint Forces Staff College

4 ―Global Teams‖ Project Report (interview-based themes)

5 ―Operations Other Than War (OOTW)‖ Project Report (interview-based themes)

6 Cross-Cultural Competence Operational Definition Report (interview-based competence

modeling)

7 Cross-Cultural Competence Assessment Report (interview-based competence modeling)

B-1

APPENDIX B: EXAMPLES OF DATA ANALYSIS TABLES

Interview with Military Personnel

B-2


Experience in Iraq

Theme Title

1. Understand the

situation within its

historical, regional,

and cultural

context

2. Understand the

other participants When in Iraq… ―The first reaction you have as a civilian walking

into that is how do you find your way around? In my case, having not

been a diplomat and not having come out of the State Dept., not having

NOT been a part of an embassy country team before, and I had a steep

learning curve just on the diplomatic side, on the State Dept., country

team side to figure out how all those processes and people worked and

interacted, etc. On top of that, though, I had to look at MNFI, and MNCI,

and MNDB, and all of those other HQ that are layered there and figure

out how to effectively integrate my staff with that group of people.‖

(analyst note: this is about understanding the other players, your own

role, and how you fit into the mix)

―Some of these guys have very, very distinct impressions on whether

the civilians are of any use at all in a battle space. And so culture is a big

deal. We need to get these guys to seek out the civilians, find and know

what their capabilities are and exploit those capabilities in the battle space

so they don‘t have to do those things that the civilians can do. And then

collaborate…..So anything that we can do that gives the military guys a

sense of a positive contribution civilians can do that helps them

understand that they‘re a different culture, that it helps them understand

that we have different communications processes and capabilities, and

that our op tempos are different.‖

(analyst note: This overlaps with ―building capability: theme – e.g.,

identifying and leveraging expertise, using all available assets)

3. Shift Perspective

4. Establish and

maintain common

ground

―I found frequently, that the civilian side of the equation was

competing with the military side. There was lack of common objective

and understanding of the long-term desired end state. A specific example,

front page of the Washington Post here 6, 8, 10 months ago, DoD‘s

Current State, Undersecretary Brinkley is in Iraq, came in and briefed us

and said we want a stand-up, state-owned enterprise—state-owned

enterprise that‘s a failed legacy of Saddam‘s socialist sort of approach to

economics—lots of employees, very little output, very poor quality, and

the embassy and the State dept. had established as an end state, stamp out


B-3

Theme Title

all state-owned enterprises, privatize and make it a competitive economy

that can merge into the international arena. So you have specific

assignment of civil affairs officers, under Gen. Carelli, working under

Brinkley with millions of dollars of funding, to stamp out state-owned

enterprise, ostensibly you get people off the streets, get them employed

because that reduces the number of potential candidates for insurgents.

And the other side you have a bunch of civilians beating themselves to

death, trying to stamp out state-owned enterprise and to create privatized

business to build the economy.‖

(analyst note: This is an example of lack of common ground between

civilian and military players)

Plane crash incident: ―…it was a civilian airplane, with six people on

board, German registered, one Iraqi, five Germans, the airplane

disappeared, it was overdue for fuel, couldn‘t find it, and the question

then is ‗well who is responsible for search and rescue?‘ And MNFI and

MNCI maintained that they were not. There was no good answer for then

who was…..there was no acceptance and no knowledge of who was

responsible for search and rescue in that country where we owned the

airspace.

My initial reaction was one of disbelief and frustration that MNFI, which

owned the airspace, and which had air assets available didn‘t

acknowledge any role or responsibility….Then sort of aghast that there

wasn‘t some contingency plan that would have prescribed the process and

steps that a person would take. So we created it on the fly….and it took

so long…And the process was not visible to me. So part of the challenge

is that the military, the MNFI processes, are not really visible to a large

population of the civilian leadership and planners who have to depend on

it. It was not collaborative; it was two separate stove pipes coming in and

trying to work together.

…finally the FRAGO got disseminated out to MNCI, which then resulted

in aircraft availability. Which in the end we didn‘t use because __ in Iraq

had already taken off and gone. We lost face with the government of Iraq.

It set us back in terms of our ability to work with them because they felt

we intentionally didn‘t respond, that we intentionally didn‘t give them the

information, that we wanted them to look bad, etc…‖

(analyst note: This is an example about lack of common ground, as far as

who is responsible for what. This overlaps with ―knowing the other

participants‖ and their roles and responsibilities. Following the crash,

there was no shared knowledge of who is responsible for search and


B-4

Theme Title

rescue. No understanding of who‘s in charge and who owns that

role/responsibility).

―If I were to walk in that office again…I‘d ask what‘s our plan?

What are our capabilities? Whose responsibility is it? What are the assets

that are dedicated to that response? And what is the communication

network that is employed to make that work smoothly?‖

(analyst note: This is about building capability, common ground,

knowing participants and roles/responsibilities)

5. Build capability

to affect the

situation

―One of the things I fought for, was to get them to identify

transportation as a critical, or essential… I think they called it an

Essential Services—oil, water, electricity, they were all designated

essential and transportation was not. Well transportation infrastructure is

critical and you can‘t distribute oil in Iraq without transportation, you

can‘t create autonomy without transportation, you can‘t move guns…all

the other things for war, much less a civil economy without highways and

rail. And so it should have been that determination should have been

made early that transportation was critical. And then how would you

leave out the senior person in transportation from the campaign planning?

(analyst note: this is about making your own role and expertise clear to

others. Convincing other stakeholders of the importance of your input)

6. Visualize the

operation

―…there‘s no continuity from incumbent to incumbent, there are no

archives, and no records in the civilian side, and which weren‘t while I

was there. I inherited no records, I had no overlap from my predecessor,

very, very minimal plan existence so there‘s no continuity.‖

(analyst note: This is about a lack of continuity. No visioning of how

what you do (or don‘t do) as far as record-keeping and continuity impacts

the future of the operation)

7. Support

information

exchange

Re Gen. Casey‘s Campaign Plan…‖I hadn‘t seen it…. you would

think that if transportation infrastructure is critical to our success in Iraq,

you would have thought that there would have been an overt attempt to

get Transportation and Secretary of Transportation‘s fingerprints on that

campaign plan. There had been no effort that I‘m aware of to do any of

that. And so I got that plan and I sat down and I wrote a significant

number of comments specifically related to this footprint issue. And I

tried to run that up channel, they never got their copy of the plan back

either, when I left it was probably still in the safe. But, because I had not

seen it, and had not been given the opportunity to coordinate on it, to

provide input, and I am suspicious that very few people on the civilian

side had a chance to act on it.…This is a clear case of two stovepipes


B-5

Theme Title

competing instead of collaborating. I didn‘t see the campaign plan until

very late. My ability to provide input to it was greatly constrained, and

I‘m not even sure today, how many of my comments got to anybody who

would‘ve read them and been able to do anything with them…But the

bigger issue is how do you coordinate serious documents, serious plans

across that spectrum of all those people, all the relative stakeholders.‖

(analyst note: This is an example of poor info exchange. He had no

involvement in Casey‘s campaign plan, no input.)

―We can‘t collaborate if everything is done on SIPR because the

civilians don‘t have access. So we need to look at ways to do… I mean,

first of all, how do you share information? And what are the tools for

sharing that information? And what are the impediments for sharing that

information? And I said SIPR is one of them. Over classification and then

not having a common IT or… the other side of that is that you have the

military guys who are very far advanced, IWS and all those other tools.

They don‘t interface with the civilians on that.‖

(analyst note: this is about recognizing impediments to information

exchange with certain players)

C-1

APPENDIX C: THEMES FOR OPERATIONS IN JIIM ENVIRONMENTS

C-2

TITLE

DEFINITION

COGNITIVE CHALLENGES

1. Understand the

situation within

its historical,

regional, and

cultural context

Do your homework. Understand the

context and history of the situation

and the participants. Identify

drivers/causes of the current

situation. Recognize that chaos is

typical of relief situations and some

development situations. A clear

framework for seeing the situation

provides a stable basis for

assessment and decision making.

Learn how to read situations from

others who can see more in the

situation than you can.

Declarative knowledge,

development of a general mental

model of relief/development

situations, sensemaking

Understand and

differentiate long-term and

short-term issues (for

example, know and use the

Capital Analysis and

Performance Strategy

(CAPS) framework).

Know and apply the

Tactical Conflict

Assessment Framework

(TCAF) when conflict is a

part of the situation.

Understand the timeline up

to the current situation.

Identify and understand

critical events on the

timeline.

Identify and understand

regional and cultural factors

that have influenced the

current situation.

―Read‖ novel situations to

focus on priority indicators

(leading up to and during

operations).

2. Understand the

other participants

Know the types of organizations and

entities that can be involved and

identify those who are involved in

the specific situation. Be aware of

their historical roles and

relationships, as well as their

priorities and their willingness to

collaborate. Also recognize there

may be differences in how they do

business. There can be differences in

work pace, work hours; integration

into the community; maintenance of

clear cut roles and responsibilities

versus diffuse roles; communication

styles; metrics for progress; comfort


development and application of a

mental model of relief/development

operations, cultural competence,

willingness to engage others to

understand their organization

Consider both

organizational culture and

culture of host nation to

understand how others

conduct business.

Understand local/regional

and organizational power

structures.

Anticipate impacts of

C-3

TITLE

DEFINITION


with and availability of technology;

authority structures; and whether

they concentrate on tasks or

relationships to do business.

Anticipate how these differences can

affect goal focus, information

sharing, planning, and decision

making in order to work more

effectively together. Understand

local power structures to anticipate

how decisions will be made,

approved and supported in the host

nation/local region.

cultural and organizational

values and differences on

decision making and action.

Understand the language

used in other organizations

to describe situations and

priorities.

Understand Interagency

Management System

designed for post

conflict/stabilization

situations.

3. Shift

Perspective

Shift your perspective to see the

situation from another person's point

of view. Shift Perspective to

understand, predict, and influence

behavior and foster communication.

Be aware of how you and your

organization appear to other

organizations and the host

nation/region. Be able to explain

your organization's position.

Problems are multi-dimensional and

aspects of the situation that are

salient to you may be background to

others and vice versa. Be aware of

the tendency to interpret others‘

decisions and actions based on your

own experience base, beliefs,

assumptions, and value system.

When seeking to understand others‘

rationale for decision-making,

consider their intent, their priorities,

patterns of living, and long-term

goals.

Cultural competence, prediction,

persuasion, negotiation,

collaboration, sensemaking, rapport

building, relationship building

See yourself as others see you.

Present yourself and your

organization in a way that is

meaningful to others.

See the situation from

another‘s point of view to

influence and predict

behavior.

Anticipate how your own

actions and decisions will

be interpreted by host

nation in order to mitigate

negative fallout.

4. Establish and

maintain common

ground

Establish common ground as a basis

for common goals and unity of

effort. There are usually many

diverse organizations and countries

with which we interact in non-

combat operations, each with their

own goals and cultures. Building in

the time necessary to find common

Collaboration, rapport building,

relationship building, problem

identification, solution

identification

Be willing to engage others

and build relationships.

Maintain relationships over the

course of the operation.

C-4

TITLE

DEFINITION


ground, and actually discovering it

becomes a significant challenge.

Establishing common ground

requires a willingness to engage and

build relationships with people who

may be very different from you.

Building and maintaining these

relationships over the course of the

operation takes time. Self-regulation

to maintain control and openness

when dealing with others takes

practice. The resulting relationships

create the basis for establishing

common goals and coordinated

actions.

Engage in self-regulation.

Discover common interests in

the situation.

Negotiate goals and solutions.

5. Build

capability to

affect the

situation

Capability is gained by knowing and

using all assets to address common

goals. It is difficult to anticipate who

will be involved in a situation and

what assets will be available.

Different functional cells of the

military organization may have to

self-organize in response to the

situation in the early stages of

operation to leverage assets and

opportunities. The diverse set of

resources and expertise that may

exist in your own organization and

other participant organizations is

often not explicit. They can go

untapped if not deliberately sought

out. Explore who has expertise

around different issues or problems.

Recognize the military can bring

technical skill and disciplined

decision making to the situation and

how that can best be leveraged

without "shutting out" others who

have ownership in the situation and

specific expertise the military does

not. Understand own boundaries for

action and manage expectations.

Contribute to the creation of a

solution process to which multiple

players can contribute.


envisioning asset employment,

recognizing and leveraging options

Identify available expertise.

Know and use all assets;

understand how resources

are accessed and deployed.

Match assets to the ―right‖

problems, not just those

most salient to your

organization.

Develop workarounds where

resource constraints or

barriers to access exist.

Manage expectations.

C-5

TITLE

DEFINITION


6. Visualize the

operation

Anticipate the need to transition to

the next phase including the

resources needed for different

players. Realize your phase of

operations may be different than

those of others. A well-developed

exit strategy up front is crucial. It

guides both large-scale and smaller,

incremental decision making.

Visualize the transition from relief to

development. Recognize the risks to

different players in the situation in

relation to a decision or action.

Visualize risks and mitigating

actions in a dynamic manner. Some

issues develop rapidly and some over

time. Crisis situations are likely to

contain both kinds of problems.

Some solutions can be affected

rapidly and some can only be

accomplished over the long-term.

Know the difference and don't let

short-term solutions overcome long-

term possibilities by ignoring second

and third order consequences.

Mental simulation,

synchronization, identification of

leverage points, prediction,

contingency thinking, decision

making, cultural competence

Understand how the dynamic

unfolding of the operations

may interact with the

region‘s historical and local

interests.

Anticipate resource needs for

the long-term; recognize

conflicts between short-

term and long-term issues.

Recognize different phases of

operation of different

players.

Develop a transition strategy

early to guide decisions and

actions.

Anticipate 2nd

and 3rd

order

effects of decisions and

actions to create lasting

solutions.

7. Support

information

exchange

Recognize different organizations‘

information needs, priorities, and

sense of urgency for information-

sharing, and be prepared to articulate

your own. Understand that the

methods and channels of

communication differ across

organizations such as method of

presentation; who shares

information; how authority for

information sharing is managed; how

complete information must be to

support decisions; central access to

information versus local access for

all involved. Recognize different

communication styles--flexible

versus standardized, embedded in

relationships versus "all business"

focus. Recognize adherence to a

Uncertainty management,

collaboration, coordination,

negotiation, persuasion, perspective

taking

Identify and respond to

different informational

needs of your collaborators.

Recognize that different

agencies may have different

priorities, and as a result,

different senses of urgency

for information.

Share your own need for

information and rationale

with collaborative partners.

Respect the different

communication channels

and methods for sharing

information.

C-6

TITLE

DEFINITION


formal chain of command; degree of

autonomy allowed to individuals;

likelihood of junior individuals to

speak out. Opportunities and limits

to information exchange can vary.

Create access to information sharing

in terms of where and how you

conduct business. Understand own

planning process and coordination

requirements for the situation.

8. Maintain

flexibility

Recognize when your approach or

stance in a situation isn‘t working,

and be willing to adapt it. Recognize

and apply behavior that is most

likely to be successful in each

situation to successfully

communicate with others. Be aware

that there are viable approaches and

solutions to problems that might be

outside the realm of the way you‘ve

approached them in the past. Be

willing to adapt your tried and true

approaches. Recognize that the first

approach or attempt may not be

successful, and that is not the same

as failure.

Problem detection, resilience,

adaptation, re-planning,

observation, flexible problem

solving, situation assessment,

decision making

Be reflective and aware of

your own and your

organization‘s biases.

Refrain from force-fitting a

certain approach.

Put more ―mental energy‖ into

understanding the situation

than in choosing a solution.

Be aware of when it‘s time to

shift your approach and try

something different.

Recognize when your style is

not effectively

communicating and have

other methods to substitute.

D-1

APPENDIX D: THEME VALIDATION SURVEY

D-2

Survey of Cognitive Training Themes for Joint, Interagency, Intergovernmental and

Multinational (JIIM) Operating Environments:

The purpose of the questionnaire is to learn what you believe are important skills and

abilities that Service Members need in order to adapt to cognitive challenges in the context of

Joint, Interagency, Intergovernmental, and Multinational Operations. Your responses will only

be used within the JTF-ITS project team.

This is a two-part questionnaire:

Part 1 Personal Information (4 items)

Part 2 Rating the Performance Abilities (55 items)

The entire survey will take approximately 60 minutes to complete. Please rate each item

according to the instructions provided for that section. If you do not understand these instructions

or the items to be rated, the facilitator will provide clarification.

Thank you for your participation. Your input is valuable for understanding how to

manage the cognitive challenges that can be found in Joint, Interagency, Intergovernmental and

Multi-national (JIIM) operating environments.

Joint Task Force, Intelligent Tutoring System (JTF-ITS), SBIR Ph. 2

Dr. Brooke Schaab, Research Sponsor, (Army Research Institute) 757-203-3306

Mr. Jim Ong, Principal Investigator, (Stottler Henke Assoc., Inc) 650-931-2700

Dr. Karol Ross (Cognitive Performance Group) 407-282-4433

D-3

Part 1 Personal Information ______ __________

Please tell us about your background and experiences by completing this section. Your responses

on this questionnaire will remain confidential, and all questionnaire results will be reported

without attribution to any of the respondents.

1. Identification: Respondent‘s ID Number _______

Age: ______ Gender: M F Rank: ___________ MOS/Branch ________

Years of Service: _______ Service: (Circle one) Army, Air Force, Navy, USMC

Current Duty Position: ___________________________________________________

Date and Location of Last Deployment: ______________________________________

Please circle one answer for the following questions.

a. Have you worked with Government Agencies or Departments as part of your civilian

occupation? YES NO Don‘t Know

b. Have you ever been assigned to a Joint Task Force staff? YES NO Don‘t Know

c. Have your military duties required you to work within a JIIM environment?

YES NO Don‘t Know

d. Have your military duties required you to work with an Allied or Host Nation military

unit? YES NO Don‘t Know

2. Most Recent Deployments: (in a Joint, Interagency, Intergovernmental, Multinational

Operation)

Location

(Country or Region)

Duty Position Date (Month/Year)

From… To…

3. Cultural Training and Education:

D-4

Please list any training or education that prepared you for a Joint, Interagency,

Intergovernmental or Multinational command or staff assignment:

Type of Training or Education Location Date (Month/Year)

4. Other Information:

Please provide other information regarding your background, operational experience, or

education/training that you believe to be relevant working with another culture.

D-5

Part 2 Rating Performance Requirements

Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the

importance of the following abilities.

Abilities

Very

Important

1

Important

2

Not

Sure

3

Unimportant

4

Very

Unimportant

5

1.

Doing homework to understand the historical context of

the situation and the participants before arrive

2. Identifying the drivers or causes of the current situation.

3. Having a clear framework to assess the situation.

4.

Having an assessing framework when the situation is

chaotic

5.

Learning to read a situation from others who understand

the situation better than I do

6.

Differentiating long-term and short-team issues when

assessing a situation

7. Establishing a timeline leading to the current situation.

8. Identifying and focusing on key indicators in a situation

9. Understanding of regional factors

10.

Knowing what types of organizations and entities could

be involved in the situation

11. Understanding the role of each agency before I arrive to a

situation

12. Understanding the willingness of different organizations

to collaborate

13.

Understanding how the agencies involved conduct

business, such as, work hours, formality of organizational

structure, pace of work, goals, and communication styles

14.

Understanding local power structures to understand how

decisions will be made, approved and supported

D-6



Abilities

Very

Important

1

Important

2

Not

Sure

3

Unimportant

4

Very

Unimportant

5

15. I need to understand the effect of culture on goals and

priorities.

16.

It is important to understand the Interagency Management

System, to conduct a successful mission.

17. Being able to understand the situation from another

person‘s point of view

18. Taking another person‘s perspective, to better predict and

influence their behavior

19. Taking another person‘s perspective so that I can

communicate with them better

20. Being aware of how the US Military appears to other

players in the situation

21. Presenting the role of the US Military in a way that is

meaningful to others

22.

Understanding that my experiences, values, and beliefs

are not the only way to interpret the situation

23. Seeing myself as others see me.

24.

Anticipating how my actions and the actions of the US

Military will be perceived to avoid negative consequences

25.

Unifying efforts by establishing common goals

26. Taking the time to find common ground with other

players in the situation

27. Building and maintain relationships

28. Maintaining emotional self control

D-7



Abilities

Very

Important

1

Important

2

Not

Sure

3

Unimportant

4

Very

Unimportant

5

29. Maintaining openness to others

30. Being willing to engage with others

31. Negotiating goals and solutions

32.

Increasing our capacity by sharing resources across

organizations.

33. Understanding and using all assets in the situation

34. Identifying who has expertise about different problems

across all players

35. Various players having ownership and contribute assets

36. Understanding the boundaries of US Military actions

37. Managing expectations about US Military actions

38. Recognizing that your phase of operation may be different

from the phase of operations of other players

39. Anticipating the resources needed for all the different

players to transition to the next phase of operations

40. A well-developed exist strategy

41.

Visualizing the transition from addressing short-term

emergency issues to long-term stability issues

42. Understanding when long-term and short-term solutions

conflict

D-8



Abilities

Very

Important

1

Important

2

Not

Sure

3

Unimportant

4

Very

Unimportant

5

43.

Creating or at least not interfering with long-term

solutions when responding to a crisis

44.

Visualizing an operation to avoid potential negative

consequences

45.

Recognizing that different organizations have different

needs, priorities, and urgencies for sharing information

46.

Understanding that different organizations have different

methods and lines of authority for sharing information

47.

Creating access for information sharing by deciding where

and how to conduct business

48.

Understanding the military‘s planning and process when

sharing information with other organizations

49.

Sharing information about the military planning and

coordination process with other players when trying to

share information with other organizations

50. Recognizing when my approach isn‘t working and

adapting it

51. Understanding the methods I have tried before may not

work a new situation

52.

Being resilient and trying new approaches when the first

attempt does not work

53. Understanding a situation in order to create flexible

solutions

54. Having more than one style of communicating

55. Understanding doctrine 3-07

D-9

Based on your training and experience, what other performance factors are important in a JIIM environment?

Thank you for your responses. This completes the JTF-ITS survey.

Please turn in this your responses to the Facilitator.

E- 1

APPENDIX E: THEME VALIDATION FOCUS GROUP GUIDE

E- 2

Personal Information ______ __________

Please tell us about your background and experiences by completing this section. Your responses

on this questionnaire will remain confidential, and all questionnaire results will be reported

without attribution to any of the respondents.

5. Identification: Respondent‘s ID Number _______

Age: ______ Gender: M F Rank: ___________ MOS/Branch ________

Years of Service: _______ Service: (Circle one) Army, Air Force, Navy, USMC

Current Duty Position: ___________________________________________________

Date and Location of Last Deployment: ______________________________________

Please circle one answer for the following questions.

e. Have you worked with Government Agencies or Departments as part of your civilian

occupation? YES NO Don‘t Know

f. Have you ever been assigned to a Joint Task Force staff? YES NO Don‘t Know

g. Have your military duties required you to work within a JIIM environment?

YES NO Don‘t Know

h. Have your military duties required you to work with an Allied or Host Nation military

unit? YES NO Don‘t Know

6. Most Recent Deployments: (in a Joint, Interagency, Intergovernmental, Multinational

Operation)

Location

(Country or Region)

Duty Position Date (Month/Year)

From… To…

E- 3

7. Cultural Training and Education:

Please list any training or education that prepared you for a Joint, Interagency,

Intergovernmental or Multinational command or staff assignment:

Type of Training or Education Location Date (Month/Year)

8. Other Information:

Please provide other information regarding your background, operational experience, or

education/training that you believe to be relevant working with another culture.

The matrix of themes was attached for discussion to gather examples of each theme.

F-1

APPENDIX F: THEME VALIDATION SURVEY RESULTS

F-2

Item

1

Very

Important

2

Important

3

Not Sure

4

Unimportant

5

Very

Unimportant

Mean

SD

1. Doing my homework

to understand the

historical context of

the situation and the

participants before

arriving

88.9%

(16)

11.1%

(2) 0% 0% 0% 1.1 .3

2. Identifying the

drivers or causes of

the current situation

77.8%

(14)

16.7%

(3) 0% 0% 0% 1.2 .4

3. Having a clear

framework to assess

the situation

61.1%

(11)

33.3%

(6)

5.6%

(1) 0% 0% 1.4 .6

4. Having an assessing

framework when the

situation is chaotic

50%

(9)

38.9%

(7)

11.1%

(2) 0% 0% 1.6 .7

5. Learning to read a

situation from others

who understand the

situation better than I

do

50%

(9)

50%

(9) 0% 0% 0% 1.5 .5

6. Differentiating long-

term and short-team

issues when assessing

a situation

50%

(9)

50%

(9) 0% 0% 0% 1.5 .5

7. Establishing a

timeline leading to

the current situation

22.2%

(4)

66.7%

(12)

5.6%

(1)

5.6%

(1) 0% 1.9 .7

8. Identifying and

focusing on key

indicators in a

situation

50% (9) 38.9%

(7) 11.1 (2) 0% 0% 1.6 .7

9. Understanding of

regional factors

72.2%

(13)

27.8%

(5) 0% 0% 0% 1.3 .5

10. Knowing what types

of organizations and

entities could be

involved in the

situation

83.3%

(15)

16.7%

(3) 0% 0% 0% 1.2 .4


role of each agency

before I arrive to a

55.6 (10) 44.4%

(8) 0% 0% 0% 1.4 .5

F-3

Item

1

Very

Important

2

Important

3

Not Sure

4

Unimportant

5

Very

Unimportant

Mean

SD

situation


willingness of

different

organizations to

collaborate

38.9%

(7)

61.1%

(11) 0% 0% 0% 1.6 .5

13. Understanding how

the agencies involved

conduct business,

such as, work hours,

formality of

organizational

structure, pace of

work, goals, and

communication styles

38.9%

(7) 55.6 (10)

5.6%

(1) 0% 0% 1.7 .6

14. Understanding local

power structures to

understand how

decisions will be

made, approved and

supported

66.7%

(12)

33.3%

(6) 0% 0% 0% 1.3 .5


effect of culture on

goals and priorities

66.7%

(12)

33.3%

(6) 0% 0% 0% 1.3 .5


Interagency

Management System,

to conduct a

successful mission

22.2%

(4)

61.1%

(11)

11.1%

(2)

5.6%

(1) 0% 2 .8

17. Being able to

understand the

situation from

another person‘s

point of view

33.3%

(6)

61.1%

(11)

5.6%

(1) 0% 0% 1.7 .6

18. Taking another

person‘s perspective,

to better predict and

influence their

behavior

33.3%

(6)

50%

(9)

16.7%

(3) 0% 0% 1.8 .7

19. Taking another

person‘s perspective

so that I can

33.3%

(6)

55.6%

(10)

5.6%

(1)

5.6%

(1) 0% 1.8 .8

F-4

Item

1

Very

Important

2

Important

3

Not Sure

4

Unimportant

5

Very

Unimportant

Mean

SD

communicate with

them better

20. Being aware of how

the US Military

appears to other

players in the

situation

55.6%

(10)

38.9%

(7) O%

5.6%

(1) 0% 1.6 .8

21. Presenting the role of

the US Military in a

way that is

meaningful to others

72.2%

(13)

22.2%

(4)

5.6%

(1) 0% 0% 1.3 .6

22. Understanding that

my experiences,

values, and beliefs

are not the only way

to interpret the

situation

66.7%

(12)

33.3%

(6) 0% 0% 0% 1.3 .5

23. Seeing myself as

others see me

44.4%

(8)

33.3%

(6)

11.1%

(2)

11.1%

(2) 0% 1.9 1

24. Anticipating how my

actions and the

actions of the US

Military will be

perceived to avoid

negative

consequences

72.2%

(13)

22.2%

(4) 0%

5.6%

(1) 0% 1.4 .8

25. Unifying efforts by

establishing common

goals

44.4%

(8)

55.6%

(10) 0% 0% 0% 1.6 .5

26. Taking the time to

find common ground

with other players in

the situation

50%

(9)

50%

(9) 0% 0% 0% 1.5 .5

27. Building and

maintaining

relationships

77.8%

(14)

22.2%

(4) 0% 0% 0% 1.2 .4

28. Maintaining

emotional self control

61.1%

(11)

38.9%

(7) 0% 0% 0% 1.4 .5

29. Maintaining openness 38.9% 55.6% 5.6% 0% 0% 1.7 .6

F-5

Item

1

Very

Important

2

Important

3

Not Sure

4

Unimportant

5

Very

Unimportant

Mean

SD

to others (7) (10) (1)

30. Being willing to

engage with others

61.1%

(11)

38.9%

(7) 0% 0% 0% 1.4 .5

31. Negotiating goals and

solutions

61.1%

(11)

38.9%

(7) 0% 0% 0% 1.4 .5

32. Increasing our

capacity by sharing

resources across

organizations

16.7%

(3)

77.8%

(14)

5.6%

(1) 0% 0% 1.9 .5

33. Understanding and

using all assets in the

situation

55.6%

(10)

44.4%

(8) 0% 0% 0% 1.4 .5

34. Identifying who has

expertise about

different problems

across all players

61.1%

(11)

38.9%

(7) 0% 0% 0% 1.4 .5

35. Various players

having ownership

and contributing

assets

50%

(9)

38.9%

(7)

11.1%

(2) 0% 0% 1.6 .7


boundaries of the US

Military‘s actions

66.7%

(12)

27.8%

(5)

5.6%

(1) 0% 0% 1.4 .6

37. Managing

expectations about

the US Military‘s

actions

55.6%

(10)

33.3%

(6)

5.6%

(1) 5.6% (1) 0% 1.6 .8

38. Recognizing that

your phase of

operation may be

different from the

phase of operations

of other players

38.9%

(7)

61.1%

(11) 0% 0% 0% 1.6 .5

39. Anticipating the

resources needed for

all the different

players to transition

to the next phase of

operation

27.8%

(5) 66.7 (12)

5.6%

(1) 0% 0% 1.8 .5

40. A well developed

exist-strategy

66.7 (12) 27.8%

(5)

5.6%

(1) 0% 0% 1.4 .6

F-6

Item

1

Very

Important

2

Important

3

Not Sure

4

Unimportant

5

Very

Unimportant

Mean

SD

41. Visualizing the

transition from

addressing short-term

emergency issues to

long-term stability

issues

22.2%

(4)

72.2%

(13) 0% 0% 0% 1.8 .4

42. Understanding when

long-term and short-

term solutions

conflict

22.2%

(4)

61.1%

(11) 11.1 (2)

5.6%

(1) 0% 2 .8

43. Creating or at least

not interfering with

long-term solutions

when responding to a

crisis

27.8%

(5)

72.2%

(13) 0% 0% 0% 1.7 .4

44. Visualizing an

operation to avoid

potential negative

consequences

44.4%

(8)

55.6%

(10) 0% 0% 0% 1.6 .5

45. Recognizing that

different

organizations have

different needs,

priorities, and

urgencies for sharing

information

33.3%

(6)

66.7%

(12) 0% 0% 0% 1.7 .5

46. Understanding that

different

organizations have

different methods and

lines of authority for

sharing information

22.2%

(4)

77.8%

(14) 0% 0% 0% 1.8 .4

47. Creating access for

information sharing

by deciding where

and how to conduct

business

33.3%

(6)

50%

(9)

16.7%

(3) 0% 0% 1.8 .7


military‘s planning

and process when

sharing information

with other

organizations

44.4%

(8)

55.6%

(10) 0% 0% 0% 1.6 .5

F-7

Item

1

Very

Important

2

Important

3

Not Sure

4

Unimportant

5

Very

Unimportant

Mean

SD

49. Sharing information

about the military

planning and

coordination process

with other players

when trying to share

information

27.8%

(5)

44.4%

(8)

16.7%

(3)

11.1%

(2) 0% 2.1 1

50. Recognizing when

my approach isn‘t

working and adapting

it

66.7%

(12)

33.3%

(6) 0% 0% 0% 1.3 .5


methods I have tried

before may not work

in a new situation

44.4%

(8)

55.6%

(10) 0% 0% 0% 1.6 .5

52. Being resilient and

trying new

approaches when the

first attempt does not

work

61.1%

(11)

38.9%

(7) 0% 0% 0% 1.4 .5

53. Understanding a

situation in order to

create flexible

solutions

55.6%

(10)

44.4%

(8) 0% 0% 0% 1.4 .5

54. Having more than

one style of

communicating

55.6%

(10)

44.4%

(8) 0% 0% 0% 1.4 .5

development of training themes for joint, interagency,..(2010) ross, k. us army research

Documents

research report

information operations

collection of information

report type

report date

previous research

social sciences research

office of management