![Page 1: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/1.jpg)
Neuroergonomicsfor flight safety
Dr. Vsevolod PeysakhovichISAE-SUPAERO, Toulouse, France
[email protected]@peysakhovich
![Page 2: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/2.jpg)
What can go wrong?
2
![Page 3: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/3.jpg)
A difficult landing…
3
![Page 4: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/4.jpg)
…and limited attentional resources
4
48 sec
![Page 5: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/5.jpg)
Human causes of accidents
5
1900 Today
![Page 6: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/6.jpg)
Human causes of accidents
6
1900 Today
![Page 7: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/7.jpg)
Cockpit without pilots?
7
![Page 8: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/8.jpg)
Aircraft evolution…
8
![Page 9: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/9.jpg)
Aircraft evolution improved safety
9
Air
bu
s, 2
01
9 1950 1960 1980 1990
Average human cognitive capacity
but…
![Page 10: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/10.jpg)
“Errors” are inevitable! Let’s adapt to them
“I was thinking. I was telling myselfthat what was particularly missingfor the good system performance, itwas the human error, and that it hasto urgently get involved.”
10
![Page 11: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/11.jpg)
“Errors” are inevitable! Let’s adapt to them
11
![Page 12: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/12.jpg)
Adaptive systems: closing the loop
12
System state
Control
Operator state
Feedback
![Page 13: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/13.jpg)
What is neuroergonomics?
13
Neuroscience Ergonomics
![Page 14: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/14.jpg)
Why measure something at all?
14
![Page 15: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/15.jpg)
What we can measure?
15
Ph
oto
: O
. K
on
on
enk
o/
Ro
sco
smo
s
![Page 16: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/16.jpg)
What we can measure?
16
Ph
oto
: O
. K
on
on
enk
o/
Ro
sco
smo
s
Eye tracking:• Eye movements• Pupil diameter
EEG:• Brain waves (Δ, Θ, A, B, Γ, M)• Event-related potentials (ERP)
fNIRSHaemodynamic response
Face temperature
System state
Respiratory rate
ECG:• Heart rate• Heart rate variability
![Page 17: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/17.jpg)
How can we measure?
17
EEG fNIRS Eye Tracking
P
O
R
T
A
B
I
L
I
T
Y
A
C
C
U
R
A
C
Y
![Page 18: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/18.jpg)
Aerospace: different operational constraints
18
![Page 19: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/19.jpg)
Studying phenomena from lab to real life
19
Ecologic validityExperimental control
![Page 20: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/20.jpg)
How to study inattentional deafness?
20
Oddball paradigm
Ecologic validityExperimental control
![Page 21: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/21.jpg)
How to study inattentional deafness?
21Giraudet, L., St-Louis, M. E., Scannella, S., & Causse, M. (2015). P300 event-related potential as an indicator of inattentional deafness?. PLoS one, 10(2)
To land or not to land?400 times
Ecologic validityExperimental control
![Page 22: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/22.jpg)
How to study inattentional deafness?
22
To hear or not to hear?240 times
Dehais, F., Roy, R. N., & Scannella, S. (2019). Inattentional deafness to auditory alarms. Behavioural brain research, 360, 51-59.
Ecologic validityExperimental control
![Page 23: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/23.jpg)
How to study inattentional deafness?
23Dehais, F., Rida, I., Roy, R. N., Iversen, J., Mullen, T., & Callan, D. (2019). A pBCI to Predict Attentional Error Before it Happens in Real Flight Conditions. IEEE SMC
To hear or not to hear?1000 times
Ecologic validityExperimental control
![Page 24: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/24.jpg)
How to study inattentional deafness?
24
• Real alert with context
• Once or twice
• More pilots
• 100% accuracy
Ecologic validityExperimental control
![Page 25: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/25.jpg)
From science (fiction) to real life
What can we implement today?
![Page 26: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/26.jpg)
Hint: Monitoring matters!
26
Pilot Not Flying Pilot Monitoring
![Page 27: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/27.jpg)
Eye tracking systems ancestors
27
Fitts, Jones & Milton (1950)
![Page 28: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/28.jpg)
Modern eye tracking systems
28
Wearable Remote
![Page 29: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/29.jpg)
Eye tracking in an aircraft
29
![Page 30: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/30.jpg)
Eye tracking in full flight simulator
30
![Page 31: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/31.jpg)
Eye-tracking integration
31
Gaze-BasedFlight Deck Adaptation
Gaze-BasedAircraft
Adaptation
In flight
Pilot Training &
Performance Analysis
On ground
Visual BehaviorDatabase
On-boardGaze
Recordings
Visual BehaviorChecking
System
Peysakhovich, V., Lefrançois, O., Dehais, F., & Causse, M. (2018). The neuroergonomics of aircraft cockpits: the four stages of eye-tracking integration to enhance flight safety. Safety, 4(1), 8.
![Page 32: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/32.jpg)
Eye-tracking integration
32
Gaze-BasedFlight Deck Adaptation
Gaze-BasedAircraft
Adaptation
In flight
Pilot Training &
Performance Analysis
On ground
Visual BehaviorDatabase
On-boardGaze
Recordings
Visual BehaviorChecking
System
Peysakhovich, V., Lefrançois, O., Dehais, F., & Causse, M. (2018). The neuroergonomics of aircraft cockpits: the four stages of eye-tracking integration to enhance flight safety. Safety, 4(1), 8.
![Page 33: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/33.jpg)
Eye-tracking integration
33
Gaze-BasedFlight Deck Adaptation
Gaze-BasedAircraft
Adaptation
In flight
Pilot Training &
Performance Analysis
On ground
Visual BehaviorDatabase
On-boardGaze
Recordings
Visual BehaviorChecking
System
Peysakhovich, V., Lefrançois, O., Dehais, F., & Causse, M. (2018). The neuroergonomics of aircraft cockpits: the four stages of eye-tracking integration to enhance flight safety. Safety, 4(1), 8.
![Page 34: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/34.jpg)
Human state integration
34
Human State-Based
Flight Deck Adaptation
Human State-Based
Aircraft Adaptation
In flight
Pilot Training &
Performance Analysis
On ground
Human StateDatabase
On-boardHuman State Recordings
Human StateChecking
System
Peysakhovich, V., Lefrançois, O., Dehais, F., & Causse, M. (2018). The neuroergonomics of aircraft cockpits: the four stages of eye-tracking integration to enhance flight safety. Safety, 4(1), 8.
![Page 35: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/35.jpg)
What is yet to be solved?
1. Summarize big (human) data
2. Certification(Explainable AI)
3. Ethics and privacy
4. 100% accuracy35
Human State-BasedFlight Deck Adaptation
Human State-BasedAircraft Adaptation
In flight
Pilot Training & Performance
Analysis
On ground
HumanState
Database
On-boardHuman State Recordings
Human StateChecking
System1
23 41 4
![Page 36: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/36.jpg)
Technology! But also safety regulations…
36
![Page 37: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/37.jpg)
Venues to talk more about neuroergonomics
37
▪ ETAVIMarch, 17
Toulouse, France
▪ ICCASMarch, 18-19
Toulouse, France
▪ NECSeptembre, 8-11Munich, Germany
![Page 38: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/38.jpg)
38
![Page 39: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/39.jpg)
Institut Supérieur de l’Aéronautique et de l’Espace
10, avenue Edouard Belin – BP 54032
31055 Toulouse Cedex 4 – France
T +33 5 61 33 80 80
www.isae-supaero.fr
39
¡Muchas gracias por su atención!
![Page 40: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/40.jpg)
Institut Supérieur de l’Aéronautique et de l’Espace
10, avenue Edouard Belin – BP 54032
31055 Toulouse Cedex 4 – France
T +33 5 61 33 80 80
www.isae-supaero.fr
40
¡Muchas gracias por su atención!
![Page 41: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/41.jpg)
Institut Supérieur de l’Aéronautique et de l’Espace
10, avenue Edouard Belin – BP 54032
31055 Toulouse Cedex 4 – France
T +33 5 61 33 80 80
www.isae-supaero.fr
![Page 42: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/42.jpg)
Aviation – the safest transportation mode
42
Air
bu
s, 2
01
9
![Page 43: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/43.jpg)
Are we too reliant on automation?
43
Decreasing manual flying skills Necessity to master (and to monitor)the latest changes in cockpit technology
![Page 44: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/44.jpg)
It’s all started with an idea…
44
![Page 45: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/45.jpg)
Eye tracking systems ancestors
45
Shackel (1960)Dodge & Cline (1901)
![Page 46: Neuroergonomics for flight safety - gob.mx · NEC Septembre, 8-11 Munich, Germany. 38. Institut Supérieur de l’Aéronautique et de l’Espace 10, avenue Edouard Belin –BP 54032](https://reader034.vdocuments.site/reader034/viewer/2022042101/5e7e1d40e07ed352d60daf8b/html5/thumbnails/46.jpg)
Modern eye tracking systems
46
SR
Res
earc
h,
Ey
eL
ink
100
0 P
lus
Pu
pil
La
bs,
Pu
pil
Inv
isib
le