humanfactors - pacdeff mt.pdf · 2016. 1. 17. · dr matthew thomas human factors & safety...
TRANSCRIPT
Getting the most out of Non Technical Skills (NTS) AssessmentHumanFactors
Dr Matthew ThomasHuman Factors & Safety Management Group
University of South AustraliaLevel 7 Playford Building - City EastGPO Box 2471 Adelaide SA 5001AUSTRALIA
ph: 0438 808 808e-mail: [email protected]
Kia Ora
Adelaide
Research Context
Culture, Error and NTS on the line...
TABLE TWELVE Logistic Regression Analysis - Contributions of contextual factors and non-technical skills to crew's
failure to respond to an error during the descent-approach-landing phase.
Predictor S.E. Wald's 2 d f p e Captain as Error Origin 1.555 .608 6.531 1 .011 4.735 Assertiveness - .849 .339 6.253 1 .012 .428 Constan t .856 .979 .764 1 .382 2.354 Model 2 = 21.994, df = 2, p < .001, Nagelkerke R2 = .293, 70.8% correct classification.
Culture, NTS, and Error Management
If captain made the error, crew 4.7 times less likely to manage error
If crew scored high on assertiveness, 2.3 times more likely manage error
Culture, NTS, and Error ManagementStrong correlations between NTS markers and Error ManagementTop five markers: Leadership / Followership
VigilanceAssertivenessInquiryCommunication Environment
Instructional Use of Error and NTS...
Use of Error in Line Training
Corrected by Instructor
Directive from Instructor
Feedback from Instructor
Corrected by Trainee
None
0% 10% 20% 30% 40% 50%
45
10
5
14
26
Instructor Acknowledged Error
Debreifed in Simulator
Debriefed Post-Session
Detection of Error Discussed
Error Prevention Discussed
Error Management Discussed
0% 25% 50% 75% 100%
9
40
6
59
48
64
6
34
3
40
56
77
TEM in the Simulator
NTS
Identify
Debrief
Day One (Training) Day Two (Check) N = 277
“Gotcha” - error due to situational demands
Early recognition and management strategies.
“Random” - slip or lapse due being
humanDetection and response strategies.
“Deficit” - lack of knowledge or skill More practice!
“Mistake” - decision-making or judgement
Dig deeper and understand why. Need to identify where decision
process went wrong.
Error Type Training Focus
Summary
Some good data on the role of NTS in crew performance during normal operations
NTS are still not a real focus in recurrent training / checking programs
CRM in the classroom and NTS in the simulator / on the line are not yet aligned
TEM can be used as a focus for NTS in training
Regulatory Context
13 Jan 2009NOTICE OF FINAL RULE MAKINGImplementation of Safety Management Systems (SMS)and introduction of Human Factors (HF) Training andNon-Technical Skills (NTS) Assessment
Amendments to Civil Aviation Orders (CAOs) 82.3 and 82.5 (Air Operators Certificates - RPT
CASA - Civil Aviation Safety Authority (AUSTRALIA)
RPT AOC holders must “have a program, approved by CASA, to train and assess personnel in human factors and non-technical skills with the aim of minimising human error.”
Human Factors (HF) means the minimisation of human error and itsconsequences by optimising the relationships within systems between people,activities and equipment.
Non-Technical Skills (NTS) means specific human competencies, including criticaldecision making, team communication, situational awareness and workloadmanagement, which may minimise human error in aviation.
CASA - Civil Aviation Safety Authority (AUSTRALIA)
17 Sep 2009Project OS 09/15 - Minor Amendment of CAO 82.3 & 82.5 to remove the legislative requirement for assessment and to provide more time for Industry to conduct Human Factors TrainingProject approved.
CASA - Civil Aviation Safety Authority (AUSTRALIA)
FAA - Federal Aviation Administration (USA)
Funded some great research.
Got burnt by AQP?
Has produced some good Advisory Circulars (AC120-51E CRM).
EU-OPS and EU-FCL
Requirement for assessment of CRM performance
EASA - European Aviation Safety Agency (EUROPE)
Line checks
(i) Line checks must establish the ability to perform satisfactorily a complete line operation including pre-flight and post-flight procedures and use of the equipment provided, as specified in the Operations Manual.(ii) The flight crew must be assessed on their Crew Resource Management CRM skills in accordance with a methodology acceptable to the Authority and published in the Operations Manual. The purpose of such assessment is to:(A) Provide feedback to the crew collectively and individually and serve to identify retraining; and(B) Be used to improve the CRM training system.(iii) CRM assessment alone shall not be used as a reason for a failure of the line check.(iv) When pilots are assigned duties as pilot flying and pilot non-flying they must be checked in both functions.
EU-OPS 1.965 Recurrent Training and Checking — Pilots
NOTECHS
Summary
NTS Assessment is coming... (albeit very very slowly).
Australia is going backwards as quickly as it is going forwards.
European regulations are the most advanced, but what about implementation?
We should be setting the standards by worlds best practice rather than awaiting regulatory requirements.
Getting the most out of NTS Assessment / Evaluation
The core requirements...
Supported by standardized and customized tools.
Supported by good resources for instructors and crew.
Starts with good syllabus & scenario design for the simulator.
Provides focus for brief and debrief during training.
BEHAVIOURAL MARKERS
TRAINING & INTER-RATER RELIABILITY
INTEGRATION OF TRAINING
FOCUSSING ON SKILL DEVELOPMENT
Behavioural Markers and NTS Assessment
During the late 1990s the “Holy Grail” was a universal standardised set of
Behavioural Markers for NTS.
NOTECHS
Behavioural Markers for NTS
NOTECHS
Rating Scale
Training and Inter-Rater Reliability
Training
rwg=1-( Sχ2/σE2)
Calibration Monitoring
2 day program training records
Integration
!"!#$%#&'()*+#,-./#0-1(2(2)##31241'#
!015.677#128#92(+(1'#,'(:;
"#$%&'()*!+ ,)-!.#-'/(!"#0$1/%2!3--!*'*4-!$1(-!5#&!6-*1'472
"<=> &,0#!"!#$%#?03@
,-6AA/(28#015.677
,)-!8̋'/6!(:'6-4'/-7!7)#9/!;-4#9!9-&-!6-&'<-6!*)&#:()!54'()*!*-7*!61*1=!
-/('/--&'/(!1/14%7'7=!1/6!54'()*!7'0:41*#&!-<14:1*'#/72
$6+.B >/('/-!7:&(-!81/!#88:&!9'*)!1!7*&#/(!8̋'/6!#&!*1'49'/6!8#0$#/-/*!'5!
*1?-#55!*)&:7*!'7!7-*!;-5#&-!;&1?-!&-4-17-2!,)-&-5#&-=!*)-!,'/(!*1?-#55!
$-6:&-!'7!7*&#/(4%!16<'7-6!9)-/!8̋'/67!-@8--6!AB!?/#*7!#&!
*1'49'/67!-@8--6!CB!?/#*72
015.677#,-6AA/(28#%4(8.'(2.A
"̋'/6!(:'6-4'/-7!1&-!/#*!8#/7'6-&-6!4'0'*1*'#/72!"̋'/6!(:'6-4'/-7!1&-!
$&#<'6-6!*#!177'7*!#$-&1*#&7!'/!-7*1;4'7)'/(!*)-'&!#9/!8̋'/6!$#4'8'-72
,1?-#55!8̋'/6!(:'6-4'/-7!1&-!;17-6!#/!*)-!0#7*!16<-&7-!1'&$41/-!4#16'/(!D4'()*!
9-'()*!1/6!15*!8-/*-&!#5!(&1<'*%E!1/6!177:0-!1/!-/('/-!#:*!F,G!1/6!$&#$-&!$'4#*!
*-8)/'H:-2!G/!74'$$-&%!&:/91%7=!8̋'/6!(:'6-4'/-7!1&-!1!5:/8*'#/!#5!&:/91%!
7:&518-!8#/6'*'#/2
IJ'/67!0-17:&-6!1*!KK!5--*!DCB!0E!*#9-&!)-'()*!1/6!1$$4%!5#&!&:/91%7!CLM!5--*!
DLN0E!#&!(&-1*-&!'/!9'6*)2
II!,1?-#55!#/!:/*&-1*-6!'8-!#&!7/#9!7)#:46!#/4%!;-!1**-0$*-6!9)-/!/#!0-4*'/(!'7!
$&-7-/*2
C42/1D#,628(+(62,-6AA/(28#,6:E62.2+#
?526+A@#F
/(+*64+#G#/(+*#/(2)'.+A
O&% KP!Q!KL
J-* AN
3*1/6'/(!J1*-&Q34:7) CN
3/#9!R!S#!T-4*'/(!II AN
U8-!R!S#!T-4*'/(!II CN
HI+6;.-#"=J#>KK!
!"!#$%#&'()*+#,-./#0-1(2(2)##31241'#
!015.677#128#92(+(1'#,'(:;
"#$%&'()*!+ ,)-!.#-'/(!"#0$1/%2!3--!*'*4-!$1(-!5#&!6-*1'472
&,0#!"!#$%#?03@ "<="
L(-.I+(621'#,62+-6'
U/'*'14!&:/91%!14'(/0-/*!1/6!70##*)!7%00-*&'814!*)&:7*!1$$4'81*'#/!&-7:4*!'/!(##6!
8̋'/6!8#/*!81$1;'4'*%!6:&'/(!*1?-#552!V'()*!5#&91&6!$&-77:&-!#/!*)-!8#/*!
8#4:0/!6:&'/(!*)-!'/'*'14!$)17-!#5!*1?-#55!,!D;-4#9!1$$&#@'01*-4%!MB!?/#*7E!
'/8&-17-7!/#7-!9)--4!7*--&'/(!-55-8*'<-/-772!W/%!6-<'1*'#/!5�!*)-!8-/*-&4'/-!
6:&'/(!*)&:7*!1$$4'81*'#/!7)#:46!;-!8#:/*-&-6!9'*)!'00-6'1*-!70##*)!1/6!$#7'*'<-!
8#/*!'/$:*72!30##*)!&:66-&!8#/*!'/$:*7!8#0;'/-6!9'*)!70144!8#/*!9)--4!
'/$:*7!&-7:4*!'/!1!/#&014!*1?-#55!9'*)!/#!#<-&8#/*,'/(2!V1&(-!8#/*!9)--4!
'/$:*7!81/!)1<-!1/!16<-&7-!-55-8*!#/!6'&-8*'#/14!8#/*!/-1&!XCDT"YE!6:-!*#!*)-!
166'*'#/14!6&1(!#5!*)-!-@*-/6-6!7$#'4-&72
$6+.B J'*)!9-*!#&!74'$$-&%!&:/91%!8#/6'*'#/7=!*)-!ZT!7)#:46!('<-!7$-8'14!
1**-/*'#/!*#!-/7:&'/(!*)-!-/('/-7!)1<-!7%00-*&'8144%!;141/8-6!*)&:7*!
'/6'81*'#/72!
C6+1+(62#128#015.677
T1'/*1'/!9'/(7!4-<-4!6:&'/(!*)-!*1?-#55!,!;%!1$$4%'/(!8#/*!9)--4!
6'7$418-0-/*!'/*#!*)-!9'/62!O:&'/(!&#*1*'#/!8#/*'/:-!*#!1$$4%!8#/*!9)--4!'/!*)-!
6'7$418-6!$#7'*'#/!*#!?--$!*)-!9'/(7!4-<-4!6:&'/(!4'5*#552!,)-!1'&$41/-!'7!'/!1!
7'6-74'$!9'*)!8M-6!8#/*/!1*!*)'7!$#'/*2!W!74#9=!70##*)!&-8#<-&%!5�!*)'7!
7'6-74'$!'7!188#0$4'7)-6!15*-&!4'5*#55!;%!74#94%!/-:*&14'['/(!*)-!8#/*!9)--4!1/6!
&:66-&!$-61472
%4A+D#M(28#128#N+-62)#,-6AA/(28#,628(+(62A
\#&!*1?-#55!'/!(:7*%!#&!7*&#/(!8̋'/6!8#/6'*'#/7=!:7-!#5!1!)'()-&!*)&:7*!7-**'/(!
*)1/!*)-!0'/'0:0!&-H:'&-6!'7!&-8#00-/6-62!J)-/!*)-!$&-<1'4'/(!9'/6!'7!1*!#&!
/-1&!]B^!*#!*)-!&:/91%=!*)-!$#77';'4'*%!#5!9'/6!7)'5*7!&-7:4*'/(!'/!(:7*%!*1'49'/6!
8#0$#/-/*7!6:&'/(!&#*1*'#/!#&!4'5*#55!'/8&-17-72!O:&'/(!*)'7!8#/6'*'#/=!8#/7'6-&!*)-!
:7-!#5!*)&:7*!7-**'/(7!84#7-!*#!#&!1*!01@'0:0!*1?-#55!*)&:7*2!,)-!:7-!#5!1!)'()-&!
*1?-#55!*)&:7*!7-**'/(!&-6:8-7!*)-!&-H:'&-6!&:/91%!4-/(*)!1/6!0'/'0'[-7!*)-!
1'&$41/-!-@$#7:&-!*#!(:7*%!8#/6'*'#/7!6:&'/(!&#*1*'#/=!4'5*#55=!1/6!'/'*'14!84'0;2
HI+6;.-#"=J#>KK!
Predictions for the Future
Integrated
Beyond Classroom
More Simulator and Line
Changes in Structure
Aligned
Targeted CFIT / ALARP
Skills. Full stop.
CRM = NTS = TEM
Still important
Skill Development
Emotions “HEA”
Fitness for Duty
Performance Protection
Changes in Content
Systems Thinking
Fatigue / Health /...
Frustration / Stress Confusion
Adapting to Situations
Safety Emerges from System
Some Good Resources
http://www.aavpasymposium.org
Dr Matthew J W Thomas
E-mail: [email protected]: +61 438 808 808 Web: http://www.unisa.edu.au/safety
UniSA © 2009