team coordination and uav operations · five certt uav experiments three team members (avo, plo,...
TRANSCRIPT
![Page 1: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/1.jpg)
Team Coordination Team Coordination and UAV Operationsand UAV Operations
Nancy J. CookeNancy J. CookeCognitive Engineering Research InstituteCognitive Engineering Research Institute
National Academy of Sciences AFRL AssociateNational Academy of Sciences AFRL AssociateASU EastASU East
![Page 2: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/2.jpg)
Research Team Members
SponsorsSponsorsAir Force Office of Scientific Air Force Office of Scientific
ResearchResearchAir Force Research LaboratoryAir Force Research Laboratory
Office of Naval ResearchOffice of Naval Research
NMSU FacultyNMSU FacultyPeter FoltzPeter Foltz
AFRLAFRLDee AndrewsDee Andrews
CERTT Developer: US CERTT Developer: US PositioningPositioning
Steven M. ShopeSteven M. Shope
Research Team Members
NMSU Graduate StudentsOlena ConnorJanie DeJoode
Jamie C. GormanPreston A. Kiekel
Rebecca KeithHarry Pedersen
ASU StudentsPat Fitzgerald
Christy CaballeroPaulette Dutcher
ASU FacultyNia Amazeen Tom Taylor
![Page 3: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/3.jpg)
OverviewOverviewCERTT Lab and UAV Synthetic CERTT Lab and UAV Synthetic TaskTask
Focus on measuring team cognitionFocus on measuring team cognitionThe synthetic UAV taskThe synthetic UAV taskCERTT capabilities CERTT capabilities Task fidelityTask fidelity
Findings relevant to coordinationFindings relevant to coordinationAcquisition and retentionAcquisition and retentionDispersionDispersionWorkloadWorkloadExpert teamsExpert teamsCommunication Communication
Conclusions & Future DirectionsConclusions & Future Directions
![Page 4: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/4.jpg)
CERTT LabCERTT LabCCognitiveognitive EEngineeringngineering RResearchesearch onon TTeameam
TTasksasks
Unmanned Aerial Vehicle Synthetic Task EnvironmentIn our UAV STE three operators must coordinate in order to maneuver their UAV to take pictures of ground targets
![Page 5: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/5.jpg)
Team Cognition in PracticeTeam Cognition in Practice
![Page 6: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/6.jpg)
Team CognitionTeam Cognition
++It is more than the sum of the It is more than the sum of the cognition of individual team cognition of individual team members.members.
It emerges from the interplay of It emerges from the interplay of the individual cognition of each the individual cognition of each team member (PROCESS IS KEY)team member (PROCESS IS KEY)
Further, for heterogeneous Further, for heterogeneous teams, individual cognition is also teams, individual cognition is also heterogeneous (AVERAGING heterogeneous (AVERAGING INAPPROPRIATE)INAPPROPRIATE)
Team coordination IS team cognition.
vs.
![Page 7: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/7.jpg)
The Synthetic TaskThe Synthetic TaskDEMPC
navigator, mission planner, plans route from target to target under constraints
Air Vehicle Operatorcontrols UAV airspeed, heading, and altitude and monitors air vehicle systems
Payload Operatorcontrols camera settings, takes photos, and monitors camera systems
Based on a cognitive task analysis done on Predator operations at at Indian Springs, NV
![Page 8: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/8.jpg)
CERTT CapabilitiesCERTT CapabilitiesFour participant consoles Four participant consoles
Integration of seven task applications over local area netIntegration of seven task applications over local area netDavid Clark headsets for participants and experimenterDavid Clark headsets for participants and experimenter
One experimenter workstationOne experimenter workstationVideo and audio recording equipment (including digital audio)Video and audio recording equipment (including digital audio)Intercom and software for logging communications flowIntercom and software for logging communications flowEmbedded performance measuresEmbedded performance measuresComputer event logging capabilitiesComputer event logging capabilitiesAbility to disable or insert noise in channels of communication Ability to disable or insert noise in channels of communication intercomintercomExperimenter access to participant screensExperimenter access to participant screensExperimenter control capability of participant applicationsExperimenter control capability of participant applicationsEasy to change startEasy to change start--up parameters and waypoint library that define a up parameters and waypoint library that define a scenarioscenarioSoftware to facilitate measurement of team process behaviorsSoftware to facilitate measurement of team process behaviorsSoftware to facilitate situation awareness measurementSoftware to facilitate situation awareness measurementTraining software modules with testsTraining software modules with testsSoftware modules for offSoftware modules for off--line knowledge measurement (line knowledge measurement (taskworktaskwork ratings)ratings)Software for administering debriefing questionnaireSoftware for administering debriefing questionnaireSoftware for administering NASA TLX, SART, and other scalesSoftware for administering NASA TLX, SART, and other scalesCapability for distributed simulation (across intranet and interCapability for distributed simulation (across intranet and internet)net)
![Page 9: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/9.jpg)
Task FidelityTask Fidelity
Task based on cognitive task analysis of Task based on cognitive task analysis of Predator operationsPredator operations
Task is faithful to the aspects of the operational Task is faithful to the aspects of the operational task that pertain to team cognitiontask that pertain to team cognition
Cognitive fidelity vs. physical fidelityCognitive fidelity vs. physical fidelity
Validity: face validity and expert benchmarkingValidity: face validity and expert benchmarking
![Page 10: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/10.jpg)
Five CERTT UAV ExperimentsFive CERTT UAV Experiments
Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos
Independent Variables: knowledge sharing, workload, dispersion
Primary Measures: performance, process, cognition (teamwork knowledge, taskwork knowledge, SA)
Experiment 1: 11 teams, 10 missions
Experiment 2: 18 teams, 5 missions, shared vs. unshared
Experiment 3: 20 teams, 7 missions, 5-7 high workload, distributed vs. co-located
Experiment 4: 20 all-male teams, 5 missions, 5th high workload, distributed vs. co-located
Experiment 5: Benchmarking, 5 “expert” teams, 5 missions, 5th high workload
![Page 11: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/11.jpg)
Exp. 1 Acquisition and RetentionExp. 1 Acquisition and RetentionFindingsFindings
0
100
200
300
400
500
600
1 2 3 4 5 6 7 8 9 10
Mission
Team
Per
form
ance
Tm 1Tm 2Tm 3Tm 4Tm 5Tm 6Tm 7Tm 8Tm 9 Tm 10Tm 11
Asymptotic team performance after 4 40-min missions (robust finding)
Process improves and communication becomes more standard over time; knowledge is more stable
Skill loss after 7 weeks
![Page 12: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/12.jpg)
Exp. 1 Acquisition and Exp. 1 Acquisition and Retention FindingsRetention Findings
5510.8610.868.718.719.639.63Long (8Long (8--11 11 weeks)weeks)
446.576.574.714.715.865.86Short (4Short (4--7 7 weeks)weeks)
NNMaxMaxMinMinMeanMean(# of (# of
weeks)weeks)
Retention Retention Interval GroupInterval Group
The question of retention of C2 skill is of great practical importance, but little is known.
![Page 13: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/13.jpg)
Dispersion IS NOT Detrimental to Team Performance
Performance Scores for Colocated and Distributed Teams
0
100
200
300
400
500
1 2 3 4 5 6 7
Mission
Mea
n Sc
ore
ColocatedDistributed
Figure ?
Performance Scores for Colocated and Distributed Teams
0
100
200
300
400
500
1 2 3 4 5
Mission
Mea
n Sc
ore
ColocatedDistributed
Figure ?
Experiment 4Experiment 3
Team skill is acquired from Mission 1 –4
Increased workload (M5 and later) is detrimental to team performance
Distributed teams perform better than co-located under high workload (Exp. 3)
![Page 14: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/14.jpg)
Dispersion IS Detrimental to Team Process
Critical Incident Scores for Colocated and Distributed Teams
00.10.20.30.40.50.60.70.8
1 2 3 4 5 6 7
Mission
Mea
n Sc
ore
ColocatedDistributed
Figure ?
Experiment 3
Critical Incident Scores for Colocated and Distributed Teams
00.1
0.20.30.40.5
0.60.7
1 2 3 4 5
Mission
Mea
n Sc
ore
ColocatedDistributed
Figure ?
Experiment 4
Team process improves from Mission 1 –4
Increased workload (M5 and later) is detrimental to team process
Co-located teams demonstrate better (or different) team process than distributed teams
![Page 15: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/15.jpg)
WorkloadWorkload
High levels of workload are detrimental to team performance and process
Under high workload teams get fewer photos per minute
Co-located DEMPCs tend to perform better and perceive greater workload than distributed DEMPCs
Working memory capacity is correlated with superior DEMPC performance
![Page 16: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/16.jpg)
Exp. 5: Benchmarking StudyExp. 5: Benchmarking Study
XXXXExpExp
XXXXFl StudentFl Student
UAVUAV
XXVideoVideo
Fl Fl InstrucInstruc
TaskwkTaskwkTeamwkTeamwkProcessProcessTEAMTEAM
0
100
200
300
400
500
600
700
1 2 3 4 5
Mission
Team
Per
form
ance
Exp 1 Average
Exp 3 AverageExp 4 AverageFlight Inst.Video GameUAV TmFlight StudentsCERTT Exp
Exp 2 Average
Five Expert Teams
1) Flight Instructors
2) Video Game Team
3) UAV Design Team
4) Flight Students
5) CERTT Experimenters
![Page 17: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/17.jpg)
Team CommunicationTeam Communication
Communication is a predominant form of team interaction in command-and-controlRealReal--time, embedded in the tasktime, embedded in the taskObservable; Think aloud Observable; Think aloud ““in the wildin the wild””Rich, multidimensional (amount, flow, speech Rich, multidimensional (amount, flow, speech acts, content)acts, content)Reflects team cognition at the holistic level; for us Reflects team cognition at the holistic level; for us this this isis team cognitionteam cognition
![Page 18: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/18.jpg)
Team CommunicationTeam Communication
We focus on flow and contentWe focus on flow and content
Differences in communication patterns correspond to team performance differences
Better performing teams display more consistent patterns of communication than poorer teams
Automated communication analysis methods make this approach practical
![Page 19: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/19.jpg)
ConclusionsConclusions
The UAV-STE provides a semi-controlled realistic environment for the study of team cognition
The UAV-STE provides a fertile ground for developing and testing measures of team cognition
Our UAV-STE data suggest that much team cognition occurs in the push and pull of information among team members
![Page 20: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/20.jpg)
Future DirectionsFuture Directions
Acquisition and Retention Studies and Models
Dynamical Systems models of coordination
Additional cognitive task analyses of UAV systems
Command-and-control in emergency response
Assessment and diagnosis of team performance via communication
A new measure of coordinated situation awareness
![Page 21: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/21.jpg)
Team Situation AwarenessTeam Situation AwarenessAn ExampleAn Example
coordinated perception and action
![Page 22: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/22.jpg)
Team Situation AwarenessTeam Situation AwarenessA Holistic/Ecological Measure of Team SAA Holistic/Ecological Measure of Team SA
Change is introduced (communication Change is introduced (communication breakdown, enemy in area, storm) that will breakdown, enemy in area, storm) that will impact missionimpact mission22--3 team members are presented cues 3 team members are presented cues regarding changeregarding changeTeam members need to perceive cues in a Team members need to perceive cues in a coordinated way (i.e., connect the dots) to coordinated way (i.e., connect the dots) to identify the changeidentify the changeTeam members coordinate to take action Team members coordinate to take action relevant to the change (e.g., change altitude, relevant to the change (e.g., change altitude, communicate indirectly)communicate indirectly)
![Page 23: Team Coordination and UAV Operations · Five CERTT UAV Experiments Three team members (AVO, PLO, DEMPC) maneuver UAV to take reconnaissance photos Independent Variables: knowledge](https://reader033.vdocuments.site/reader033/viewer/2022060503/5f1d3915d3712d13be5332d7/html5/thumbnails/23.jpg)
Questions or Comments?Questions or Comments?email:
website:
www.certt.com
Cognitive Engineering Research Institute
Mesa, AZ
www.cerici.org