Collaborative Collaborative Time-Sensitive TargetingTime-Sensitive Targeting

Mary (Missy) Cummings, Stacey ScottHumans and Automation Laboratory

http://halab.mit.edu{missyc, sdscott}@mit.edu

Research in the Humans and Automation Lab (HAL) focuses on the multifaceted interactions of human and computer decision-making in complex sociotechnical systems.

• Time-Sensitive Operations for Distributed Teams

• Measurement of Display Complexity

• Collaborative Human Computer Decision Making

• Integrated Sensor Decision Support

• Reduced Manning for Unmanned Vehicle Teams

• Single Operator Control of Multiple Unmanned Aerial Vehicles

Current HAL Research Projects

Human Supervisory Control in NCO

Supporting Supervisors of Teams of HSC Operators

Supporting the Control of Teams of Heterogeneous Unmanned Vehicles

Supervisory Control & Teams

• How do operators manage multiple supervisory control processes?

• What effects do human performance limitations have on the overall system?

• Reduced Manning

• Combining Shadow positions

• Multiple UV control for a single operator

• Distributed control

Multi-Modal Watch Station

Inverting Current UAV Control Schemes

• Tomahawk Strike Planning: missions and missiles

• Determine how humans and computer optimization algorithmscan complement each other

• Cognitive strategies

Collaborative Decision Making

• Decision Support for Planetary Exploration

• Astronaut traversal replanning task

• Path replanning with increasingly complex constraints

• Affect of level of automated decision support

Aiding Human-Computer Path Planning

Goal of this Research ProjectGoal of this Research Project

To identify or develop technologies to facilitate collaborative decision-making in the time-sensitive targeting task domain

Time-Sensitive Targeting is Highly Time-Sensitive Targeting is Highly CollaborativeCollaborative

Teamwork in Network-Centric OperationsTeamwork in Network-Centric Operations

• Highly connected operators, often working remotely

• Heavy reliance on commercially available collaboration technologies

- email

- instant messaging (‘chat’)

- video & desktop conferencing

Benefits of Collaboration TechnologiesBenefits of Collaboration Technologies

• Facilitate collaboration with remote colleagues

• Facilitate sharing of and access to digital media during group work (in both co-located and distributed settings)

• Provide opportunity and support for casual interactions

• Promote ad hoc collaboration & communication

• Enhance team cohesion

Disadvantages of Collaboration TechnologiesDisadvantages of Collaboration Technologies

• Require explicit effort to access / share information

- implicit or more subtle communication / interaction is often missing

- lack of non-verbal cues

• Typically convey information about completed activities, ongoing progress/status information is also important:

- Has anyone seen my request for help?

- Is someone working on the problem? What’s their status?

• Associated costs with using collaboration technologies:

- distraction

- time & mental effort of asking for or sharing information

Enhancing Teamwork in Non-Military Enhancing Teamwork in Non-Military DomainsDomains

• Research in the CSCW domain has been focused on providing technological support for teamwork in many collaborative settings:

- corporate, educational, research

• Reviews of the CSCW literature has revealed common threads across many new approaches to CTs:

- Minimizing overhead costs of sharing information about remote collaborators

- Increased use of automation to help provide this information

• Many approaches are aimed at supporting the shared activity process

Supporting the Shared Activity ProcessSupporting the Shared Activity Process

• Focused on providing team member’s activity awareness (Carroll et al., 2003):

- awareness of project work that supports group performance in complex tasks

- implies an awareness of other people’s plans and understandings

IBM’s Activity Explorer (Millen et

al., 2005)Project View IM (Scupelli et al., 2005)

Virtual School Project Timeline (Carroll et al., 2003)

Supporting the Shared Activity Process Supporting the Shared Activity Process (cont’d)(cont’d)

Approaches for supporting activity awareness:

• Feedthrough of action

• Visibility of action

Scupelli, P., Kiesler, S., Fussell, S.R., & Chen, C. (2005). Project View IM: A Tool for Juggling Multiple Projects and Teams. In Extended Abstracts of CHI 2005: ACM Conference on Human Factors in Computing Systems , pp. 1773-1776.

Project View IM (Scupelli et al., 2005)

Supporting the Shared Activity Process Supporting the Shared Activity Process (cont’d)(cont’d)

Approaches for supporting activity awareness:

• Feedthrough of action

• Visibility of action

• Important to situate this information within the context of the shared activity process

Virtual School Project Timeline (Carroll et al., 2003)

Carroll, J.M., Neale, D.C., Isenhour, P.L., Rosson, M.B. & McCrickard, D.S. (2003). Notification and Awareness: Synchronizing task-oriented collaborative activity. International Journal of Human-Computer Studies, 58(5), pp. 605-632.

Challenges Introduced by the TST DomainChallenges Introduced by the TST Domain

• Determining what constitutes relevant activities when ‘shared objects’ are not necessarily ‘files’

• Reducing the cost of providing activity-related information to team members

• Balancing awareness and information overload

• Balancing individual performance and team performance

Methodology for Developing Activity Methodology for Developing Activity Awareness Support for Collaborative TST Awareness Support for Collaborative TST OperationsOperations

• Develop representative collaborative TST task scenario

• Conduct task analysis on scenario to develop collaborative technology requirements, focusing on the shared activity process

• Develop initial team display designs for collaborative TST scenario incorporating CT requirements

• Implement team display designs

• Develop evaluation protocol and teamwork metrics for evaluation of team displays

• Design and build team testing environment

• Evaluate team displays

• Iterate team display designs based on study results

Generalized Team TST (GTT) Task

• A convoy is scheduled to passthrough a hostile region

• Team of UAV operators must protect convoy until convoy exits region

• Stakeholders:- Convoy- UAV Operations Team + UAVs- Strike Team- Intelligence- Enemy

Representative Task Scenario: GTT TaskRepresentative Task Scenario: GTT Task

convoy

Enemy (TSTs)

intel

cont

rols

TS

T B

DA

target pinpoint

UAVs

1A 2A 3A

1B1C1D

2B2C2D

3B3C3D

ISR

ISR

dete

cts

loca

tion

TST ISR

eng

ages

TST alert

orders engagement

BDA

convoy location

requests BDA

striketeam

report toupper command

UAV OperationsTeam

detects

TS

T I

SR

TST BDA

Activity Awareness Information Analysis for Activity Awareness Information Analysis for GTT Task Scenario: Inter-team InteractionGTT Task Scenario: Inter-team Interaction

Activity Awareness Information Analysis for Activity Awareness Information Analysis for GTT Task Scenario: Intra-team InteractionGTT Task Scenario: Intra-team Interaction

UAV Operations Team

controls

TST BDA (UAVs)

TST ISR (intel)

TST alerts (convoy)

orders engagement

BDA (strike team)

convoy location (intel)

requests BDA

report to command

TST ISR (UAVs)

TST BDA (intel)

authorize engagement

targ

et a

lloca

tion

mCDR

request engagement

BDA ass

ignm

ents

UAV Ops

situation update

COM

ISR – STR – BDA report – control

Activity Awareness Requirements for the Activity Awareness Requirements for the GTT GTT Task ScenarioTask Scenario

• Types of intra-team shared activity information

- relevance to current tasks / team member

- available task resources

- priority of current task activity

- operators’ availability (progress, workload, tasking availability)

- operators’ interruptibility (status, cognitive availability)

• This information helps team members address the questions:

- when / how can I help somebody else?

- when / how can I be helped?

- how do my actions impact others’ activities?

RelevantRelevant Activity Awareness Information is Activity Awareness Information is Role DependentRole Dependent

• COM and mCDR need status and workload information of the UAV Ops to ensure individuals and team are not overwhelmed

- mCDR can help UAV Ops prioritize task activities or request support from external resources

- helps COM distributed TST and BDA assignments among UAV Ops

• UAV Ops need operator & resource availability and interruptibility information

- helps minimize disruptions when someone is involved in critical tasking

- helps operators know who can feasibly help them, or who might need their help

Supporting Team Supervision: Status and Workload Supporting Team Supervision: Status and Workload Information on a Situation Display for Battlefield Information on a Situation Display for Battlefield ContextContext

Supporting Team Supervision: Status and Workload Supporting Team Supervision: Status and Workload Information on a Team Status DisplayInformation on a Team Status Display

Z1OK

CONVOY

5%Z1

UAV OP 1

UAV OP 2

UAV OP 3

1B

1D

1C

1A

2B

2D

2C

2A

3B

3D

3C

3A

Z2 Z3

ISRroute:total:

OP 1 OP 2 OP 3 total

10% 25% 10% 15%20% 50% 90% 53%

targets

ISR

STR

5

3

BDA: 2

Comms Connectivity Status

AWACSSATELLITE

STRIKE TEAM

UAVTEAM

Ongoing Work: Display Design & Ongoing Work: Display Design & ImplementationImplementation

• Iterating on the status and workload information displays

• Developing activity awareness displays to integrate into our existing UAV Op simulation environment, MAUVE, that provides operator & resource availability and interruptibility information

MAUVE Tactical Display MAUVE Decision Support Display

Ongoing: Developing Team Testing LabOngoing: Developing Team Testing Lab

• 3 Large-screen displays (A)

• 4 Reconfigurable operatorstations (B)

• 2 Mobile operatorstations (tabletPCs &handhelds) (C)

• Wireless headsetswith microphonesfor ‘always on’ audiovia Internet software

A

B

C

A

B

C

HAL Complex Operation CenterHAL Complex Operation Center

Ongoing: Developing Team Testing LabOngoing: Developing Team Testing Lab

• 3 Large-screen displays (A)

• 4 Reconfigurable operatorstations (B)

• 2 Mobile operatorstations (tabletPCs &handhelds) (C)

• Wireless headsetswith microphonesfor ‘always on’ audiovia Internet software

A

B

C

A

B

C

HAL Complex Operation CenterHAL Complex Operation Center

Project SummaryProject Summary

• Enhancing activity awareness in collaborative TST environments seems to help promise for helping teams coordinate their actions and help reduce the costs of collaborating via technology

• Overarching goal of this project is to understand the tradeoffs between supporting individual operator performance and team performance to provide overall mission effectiveness during TST operations

Thanks!

This research is funded by:

Boeing Phantom Works&

Air Force Research Lab

For more information:

http://halab.mit.edu

or

sdscott@mit.edu