causse thesis summary - free

66
PhD Thesis INSTITUT SUP ÉRIEUR DE l’AÉRO NAUTIQUE ET DE L’ESPACE Mickaël Causse Influence of age and reward on piloting performance: a neuroergonomics contribution to aeronautic safety Doctoral School : Aéronautique Astronautique Thesis advisor: Josette Pastor Frédéric Dehais

Upload: others

Post on 04-May-2022

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Causse Thesis summary - Free

PhD Thesis

INSTITUT SUPÉRIEUR DE l’AÉRONAUTIQUE ET DE L’ESPACE

Mickaël Causse

Influence of age and reward on piloting performance:

a neuroergonomics contribution to aeronautic safety

Doctoral School : Aéronautique Astronautique

Thesis advisor:

Josette Pastor

Frédéric Dehais

Page 2: Causse Thesis summary - Free

General abstract

Traditionally, the human-system interactions (HIS) have been studied through the modification of

the human behavior in response to system variations. Although this approach has opened the way to

great progresses in HIS framework, especially when observations led to descriptive modeling, an

important part of the interaction remains unknown. Indeed, while, for instance in military

aeronautics, the physiological constraints were taken into account (eg. G-lock), the cognitive

constraints inherent to the brain functioning were largely unknown. In this perspective, human error

becomes a symptom of the system deficiency. An innovative and original approach to cope with the

problematic of the HIS is to use the tools and concepts of the integrative neurosciences and of the

neuropsychology. These researches, based on the study of the neural substrate of cognitive

mechanisms, have made the knowledge related to the man-environment interaction grow. These

scientific results arouse human factor interest and generated a new discipline in the USA:

neuroergonomics.

In this work, we aimed at applying this approach to better understand the pilot performance,

especially its degradation. We tried to comprehend the similarity of executive functions (EFs)

impairments that appear in the diminished subject (neurological pathology, sleep deprivation, effects

of some medications etc.) and in the healthy pilot facing degraded conditions. Indeed, piloting errors

may be, in some cases, attributed to temporary perturbations of the EFs. These findings plead in

favor of the hypothesis of common cerebral mechanisms and suggest the existence of a cognitive

continuum, spanning from optimal intellectual performances to a degraded functioning, as found in

pathological subjects. Therefore, the hypothesis of the cognitive continuum leads us to focus on the

deleterious impact of aging and emotion on the pilot’s performance.

We have performed a first study aiming at understanding the cross influences of the risk and the

reward on decision making in aeronautics. A first experiment has used neuroimaging tools (fMRI) in a

neuroeconomics experimental protocol (See part 1). The behavioral results showed an increase of

risky decisions under the influence of the reward. The neuroimaging preliminary results point out

that the shift from “cold” reasoning to “hot" reasoning has resulted in a transfer from dorsolateral

prefrontal cortex (main center of the EFs) activations toward ventromedial prefrontal cortex

(strongly connected to emotional brains areas) activations. A separate psychophysiological

experiment was conducted with the same neuroeconomics experimental protocol (see part 2). The

behavioral results confirmed the increased risky decision under the financial incentive. In addition,

Page 3: Causse Thesis summary - Free

this behavioral shift was associated with an increased heart rate, sign of the emotional impact of the

reward.

We have also conducted a research to assess the subtle perturbations of the EFs during normal

aging so as to identify predictive clues of the ability to pilot in general aviation, where no age limit is

defined (see part 3). The results showed that reasoning, working memory abilities and flight

experience were predictive of the piloting performance. The most the reasoning and the working

memory abilities were efficient, the smaller the flight path deviation (FPD), amount of angular

deviation in X axes from the ideal flight path, was. In the same way, the most the pilots were

experienced, the smaller the FPD was. The adjusted r² showed that our model explained 44.51% of

the variation of the FPD. Besides, the aging effects on cognitive functioning were examined and the

predictibility of this latter, plus age and flight experience, on the relevance of a decision-making

performed during landing was assessed. The results confirmed the strong vulnerability of EFs to the

aging effects, except the reasoning abilities that were somewhat preserved in our sample of pilot. In

addition, the working memory, flight experience and trait impulsivity were predictive of poor

decision-making. The pilots who made the good decision to go-around demonstrated a better

percentage of correct answers in the 2-back task, a larger total flight experience and a lower motor

impulsiveness compared to pilots who made a poor decision.

Finally, we have operationalized our results by experimentations carried out in real flight with

oculometric means (see part 4). An eye tracker was embedded in a light aircraft and a flight scenario

was defined. In addition, two different landing were performed, a nominal one and a degraded one

(with a simulated engine failure). The study revealed the feasibility of collecting visual information

taking of pilots in real flight, and the pupil dilation indicated stress and mental load effects. The pilots

spent less time glancing at the instruments, and focused on fewer instruments in the degraded

condition. Moreover, the pupil size varied with the flight phases in the degraded condition, which

reflected the variations of stress and attention levels. These encouraging results open two tracks: the

development of new eye trackers able to overcome current technical limitations, and

neuroergonomics researches providing guidelines for new man-machine interfaces integrating both

flight and crew state vectors.

All theses researches illustrate the capacity of neuroergonomics to grow beyond the disciplines

from which it emerges, in particular human factors.

Page 4: Causse Thesis summary - Free

Part 1

Economic issues provokes hazardous landing decision-making by enhancing the activity of

"emotional" neural pathways

Submitted to ICRAT 10

Mickaël Causse

Centre Aéronautique et Spatial ISAE-SUPAERO; Inserm; Imagerie cérébrale et handicaps neurologiques UMR 825; F-31059 Toulouse, France

Université de Toulouse; UPS; Imagerie cérébrale et handicaps neurologiques UMR 825; CHU Purpan, Place du Dr Baylac, F-31059 Toulouse Cedex 9, France

[email protected]

Frédéric Dehais Centre Aéronautique et Spatial ISAE-SUPAERO; Université de Toulouse, France

[email protected]

Patrice Péran Laboratoire de Neuroimagerie, IRCCS Fondation Santa Lucia, Rome, 00149, Italie

[email protected]

Umberto Sabatini Laboratoire de Neuroimagerie, IRCCS Fondation Santa Lucia, Rome, 00149, Italie

[email protected]

Josette Pastor Inserm; Imagerie cérébrale et handicaps neurologiques UMR 825; F-31059 Toulouse, France

Université de Toulouse; UPS; Imagerie cérébrale et handicaps neurologiques UMR 825; CHU Purpan, Place du Dr Baylac, F-31059 Toulouse Cedex 9, France

[email protected]

Abstract— The analysis of aeronautical accidents highlights the fact that some airline pilots demonstrate a

trend to land whereas the approach is not well stabilized. This behavior seems to be the consequence of

various factors, including financial issues. Our hypothesis is that financial constraints modulate the brain

circuitry of emotion and reward, in particular via the interactions between two prefrontal structures: the

dorsolateral prefrontal cortex (DLPFC), main center of the executive functions (EFs), high level cognitive

abilities, and the ventromedial prefrontal cortex (VMPFC), structure linked with the limbic system, major

substratum of emotional processes. In our experiment, participants performed a simplified task of landing

in which the level of uncertainty and the financial incentive was manipulated. A preliminary behavioral

experiment (n = 12) was conducted. A second experiment using fMRI is in progress and a case study only

is reported here. The behavioral data showed that the participants made more risky decision to land in the

Page 5: Causse Thesis summary - Free

financial incentive condition in comparison to the neutral condition, where no financial incentive was

delivered. This was particularly true when the uncertainty was high. The functional neuroimaging results

showed that the reasoning performed in neutral condition resulted in enhanced activity in DLPFC. On the

contrary, under the influence of the financial incentive, VMPFC activity was increased. These results

showed the effectiveness of the financial incentive to bias decision-making toward a more risky and less

rational behavior from a safety point of view. Functional neuroimaging data showed a shift from cold to

hot reasoning in presence of the financial incentive, suggesting that pilot erroneous trend to land could be

explained by a temporary perturbation of the decision-making process due to the negative emotional

consequences associated with the go-around.

Keywords: decision making, emotion, reward, piloting performance.

I. INTRODUCTION

Approach and landing are critical flight phases. They require formalised sequences of actions

(e.g. to put and lock the gear down, to extend the flaps) and to follow an arrival procedure through

several waypoints. They also require decision-making processes based upon rational elements like

the maximum crosswind speed for a given aircraft. Uncertainty, a worsening factor since it generates

psychological stress, can be high during landing. Moreover, several psychosocial factors lead some

pilots to irrational decision-making, such as keeping landing on whereas all safety parameters are not

respected [1]. According to the legislation, such hazardous conditions (e.g. thunderstorm, heavy rain,

strong crosswind or windshear) require to go-around and to perform a new attempt to land more

securely or a diversion to another airport. A study conducted by the MIT [2] demonstrates that in

2000 cases of approaches under thunderstorm conditions, two aircrews out of three keep on landing

in spite of incompatible meteorological conditions. This phenomenon, called plan continuation error

[3], also exists in general aviation. Indeed, the BEA (the French national institute for air accident

analysis) revealed that this pilots’ trend to land (the get-home-itis syndrome) have been responsible

for more than 41.5 percent of casualties in light aircrafts [4].

II. ECONOMIC PRESSURES AND LANDING DECISION

Many experiments have addressed the difficulty for pilots to revise their flight plan and several

cognitive and psychosocial explanatory hypotheses have been put forward [5] [6] [7] [8]. Another

explanation to this trend to land in spite of bad meteorological conditions or an unstabilized approach

may resides in the impact of a large range of aversive consequences associated with the decision to

go-around. Indeed, a go-around generates a stress in the crew and the passengers, the pilot can feel it

like a failure and it may lead to difficulties to reinsert the aircraft in the traffic. Moreover, a go-around

has negative financial consequences for the company (fuel consumption in particular). An

Page 6: Causse Thesis summary - Free

organisation’s emphasis on productivity may unconsciously set up goal conflicts with safety. The

culture of the company weighs on security: if it attaches a negative connotation to a go-around, it is

an excellent candidate for landing accidents. One now-defunct airline used to pay passengers one

dollar for each minute their flight was late until a crew attempted to land through a thunderstorm and

crashed [9]. According to Orasanu [8], companies also emphasize fuel economy and getting

passengers to their destinations rather diverting the flight, perhaps inadvertently sending mixed

messages to their pilots concerning safety versus productivity. Those blurred messages create

conflicting motives, which can affect unconsciously pilots’ risk assessments and the course of action

they choose. All these emotional pressures could alter the rational reasoning by shifting decision-

making constraints from safety rules to economic optimization.

III. FROM COLD TO HOT DECISION MAKING

Neglected during the first half of the 20th century, the role of emotion on cognitive functioning has

been recently fully established in the cognitive neurosciences. According to Koenig and Sander [10],

this historical neglect of emotion is explained by the difficulty inherent to its investigation and by the

influence of a scientifically-correct Cartesianism that considered the cognitive system as the

“incarnation of reason”. Today, it is commonly admitted that experiencing an emotion can trigger

unconscious processes useful to decision-making, in particular when the uncertainty is high [11]. Many

experiments put forward evidence of a strong interaction between the limbic system, “emotional

brain”, and other structures like the prefrontal cortex, the “rational brain”. For instance, Drevet &

Raichle [12] have shown the existence of a dynamic balance between regions of the limbic system

(amygdala, posteromedial cortex, ventral anterior cingulate cortex) and regions more associated to

EFs (dorsal anterior cingulate cortex, DLPFC). Similarly, Mayberg and colleagues [13] have put in

evidence that an increased activity of limbic and paralimbic regions (subgenual cingulate, anterior

insula) was proportional to the decrease in activity of neocortical regions (right prefrontal cortex,

inferior parietal cortex) during the experience of sadness. These types of observations are supported

by a study of Mitchell [14], which demonstrated that the activity of the DLPFC was inversely

proportional to the VMPFC. A previous study of Goel and Dolan [15] have also highlighted this type of

emotional and cognitive subdivision in the prefrontal cortex (PFC) in a reasoning task. In this study,

when the reasoning task was performed without emotional induction, cold reasoning, DLPFC

activations were found. When the same task was performed with emotional induction, hot reasoning,

VMPFC activations were observed. All these studies allow to understand how emotion or stress are in

Page 7: Causse Thesis summary - Free

relation to cognitive functions and how they can modulate the cognitive performance, in particular

the EFs [16], mainly implemented within the PFC.

Our hypothesis is that plan continuation error may be, at least in part, related to a shift from cold

reasoning to hot reasoning, in result of the different negative emotional consequences associated

with the decision to go-around. Cold reasoning may be supported by EFs whereas hot reasoning may

be less rational from a safety point of view and more oriented toward company’s financial interest. In

a fully neuroergonomics approach, we propose to investigate this hypothesis by reproducing a

simplified landing task performed in an fMRI.

IV. METHODS

A. Subjects

Two separate experiments were conducted. 12 physically and psychiatrically healthy volunteers

were recruited from the local population to participate to the behavioral experiment (age: mean = 28,

SD = 3.69). 1 participant (age 28) was scanned in the fMRI. All subjects were right handed as measured

by the Edinburgh handedness inventory [17]. The experiment was approved by the local ethic

committee and an informed consent was obtained before participation.

B. From aicraft to fMRI

The task was based on a simplified reproduction of a real flight instrument, the ILS (Instrument

Landing System). The participants were instructed that they were flying a plane during the landing

phase and that like pilots, they were allowed to avoid landing if they believed that landing was unsafe

(Figure 1). Decisions were based on two elements of the ILS: the localizer and the glide path, which

provide lateral and vertical guidance to adjust the trajectory of the aircraft to the runway. This

information was given by two rhombuses, like in real life, displayed below and on the right of the PFD

(Primary flight Display). It was explained to participants that the landing was safe when both

rhombuses were close to the center of their axes, the farthest from the center the rhombuses were,

the higher was the risk of crash. For each trial, the participants indicated their responses by pressing a

button on the response pad. A first independent variable with two modalities was the degree of

uncertainty, high or low, linked with the rhombuses position. The second independent variable was

the type of incentive, neutral or financial. During the neutral condition, the incentive was only based

on a feedback that gave information on the accuracy of the responses. During the financial condition,

a financial incentive was added to the one that gave feedback on the accuracy of the responses. The

task consisted of a set of 4 runs, 2 neutral, and 2 financial in which the level of uncertainty was

manipulated according to the two modalities (high and low).

Page 8: Causse Thesis summary - Free

Figure 1. Simplified reproduction of the decision-making performed during the landing phase. In the upper part, the real environment. From left to right: the PFD within the cockpit, a zoom on the PFD and the throttle. In the bottom part, the experimental environment. From left to

right: the fMRI, the simplified PFD (with only the two rhombuses of the ILS) and the response pad that allowed to decide to accept to land or to go-around.

C. Stimuli

1) The ILS

The stimuli was based on a 480x480 pixels simplified PFD with ILS and they reproduced a landing

situation without external visibility. During the scan, they are displayed via back projection and an

angled mirror in the head coil housing. Two different levels of uncertainty, depending of the positions

of the two rhombuses, are randomly sorted within the 4 runs. In landings with low uncertainty, the

decision-making was straightforward: either the rhombuses were very far from their respective

center, requiring a go-around (likelihood of successful landing: 0%), either they were very close,

requiring a landing acceptance (likelihood of successful landing: 100%). In landings with high

uncertainty, rhombuses were borderline (not very far, not very close from the center) and the

likelihoods (unknown by the subjects) of a successful landing or a crash is 50%. The positions of the

two rhombuses were related to a score. Each axis was graduated with a 16 points scale, the most the

rhombuses were far from the center, the higher was the score and the weaker was the likelihood to

land securely (Figure 2).

Page 9: Causse Thesis summary - Free

Figure 2. Categorization of the level of uncertainty according to the rhombuses positions. A score was computed according to the position of

both rhombuses (the zoom gives information on only one rhombuse, the graduation was not displayed during the experimentation). The rhombuses position were counterbalanced to avoid laterality effects. The order of presentation of the stimuli was randomized.

2) Feedback

After each response, the participants received a feedback that informed on the response accuracy

(OK, for a successful landing or a justified go-around; NO, for an erroneous decision to land or an

unjustified go-around). During the condition with financial incentive, a second feedback gave

information about the financial consequences of the decision (Figure 3). At the end of each run, a

global feedback indicated the percentage of correct responses (safety score). Concerning the run with

financial incentive, another feedback indicated the cumulative amount of money won or loss (financial

score).

Figure 3. The various feedbacks displayed after each decision making. Without incentive, only the accuracy feedback was delivered (ok/no),

with financial incentive, the monetary consequences are also displayed after the accuracy feedback.

D. The payoff matrix

During the financial incentive condition, negative emotional consequences associated with a go-

around have been reproduced by a payoff matrix. This matrix was set up to bias responses in favor of

landing acceptance. A go-around was systematically punished by a financial penalty. The penalty was

less important (-2€) when the go-around was justified (in the case where rhombuses were very far

from their center) than when it was unjustified (-5€). This systematic punishment of the decision to

go-around reproduced the systematic negative consequences associated with this latter in real life. A

successful landing was rewarded (+5€) whereas an erroneous decision to land was punished (+5€).

The fact that the erroneous decision to go-around was less punished that the erroneous decision to

Page 10: Causse Thesis summary - Free

land may appears counterintuitive but the matrix was set up in this way for two reasons. Firstly, in real

life, the pilots know that crash and overrun are rather unlikely events whereas the negative

consequences associated with a go-around are systematic. The analysis of unstabilized approach

confirms that accidents are rather rare in spite of frequent risk taking [2]. Secondly, we could not

reproduce the low frequency of accidents in an fMRI experiment because the cerebral signal

associated with rare events could not emerge from a statistical point of view. For this reason, we were

compelled to modulate the weight of the punishment rather than the frequency (Figure 4).

Case GO GO-A

Choice

GO +5€ -2€

GO-A -5€ -2€

Figure 4. Payoff matrix biased in favor of landing acceptance. A successful landing pays 5€, an erroneous decision to land costs 2€, a justified go-around costs 2€ and an unjustified one costs 5€.

E. Experimental design for the behavioral study

We used a 2x2 factorial design crossing the financial incentive (neutral and financial) and the

uncertainty (high, 50% chance of crash, or low, 100% or 0% chance of crash). Stimulus display and

data acquisition were done with Cogent 2000 v125 running under Matlab environment (Matlab

7.2.0.232, R2006a, The MathWorks, USA). Two types of runs were presented during the experiment:

neutral and financial ones. There were three likelihoods (0%, 100%, 50%) of successfully landing (40

trials for 0%; 40 trials for 100% and 80 trials for 50%), depending on both positions of the rhombuses

displayed on the PFD. These likelihoods were unknown by the subjects. Each trial consisted in a

presentation of the stimulus (3 s) during which the participant performed his decision thanks to a

response pad, followed after a delay (5.5 s) by the feedback informing of the accuracy of the response

(2 s). During the incentive condition the financial outcome was also displayed ({+5€}, {-5€} or {–2€}.

Finally, an inter trial interval (2 s) was introduced. Response and financial feedback delivery was

contingent upon the subject’s response

F. Experimental design for the fMRI study

The fMRI design was identical to that of the behavioral study excepted that the stimulus display

duration was shorter (2.5 s) and that the delay duration (6-10 s) and the inter trial interval duration (3-

9 s) were variables for neuroimaging technical issues. The long variable delay before the feedback

allowed us to distinguish the hemodynamic signal associated with the decision taking during the

Page 11: Causse Thesis summary - Free

stimulus presentation from the sustained signal associated with the reward uncertainty during the

delay (Figure 5).

Figure 5. Experimental design. Four runs were performed by the participants (160 trials). The analysis of neuroimaging data was performed during the stimulus presentation (at choice) and during the delay. The order of presentation of the run was fully counterbalanced to avoid

order effects.

Before the experiment, participants performed two runs (neutral and financial) to become familiar

with the task and the payoff matrix. The training is identical to the behavioral task.

G. fMRI image acquisition

The experiment was conducted at the Fondazione Santa Lucia (Rome). All the data were acquired

in a single session on a 3 T Allegra scanner (Siemens Medical Solutions, Erlangen, Germany) with a

maximum gradient strength of 40 mT/m, using a standard quadrature birdcage head coil for both RF

transmission and RF reception. The fMRI data were acquired using a gradient echo-EPI, with 38 axial

slices with a voxel size of 3 × 3 × 3.75 mm3 (matrix size 64 × 64; FOV 192 × 192 mm2) in ascending

order. The acquisition time was 2.47 s / 65 ms/ slice.

H. Analysis of fMRI data

Data analysis was performed within the Statistical Parametric Mapping analytic package (SPM5,

Wellcome Department of Cognitive Neurology, London, UK). The data were sinc-interpolated in time.

All images were re-aligned to the first acquired volume to correct head motion. Image was then

spatially normalized and the transformation parameters were then applied to the functional volumes,

smoothed with a (6*6*6*8 mm) isotropic Gaussian smoothing kernel. The preliminary analysis

focused on non-specific effect of the financial incentive by collapsing reward regressor for the period

Page 12: Causse Thesis summary - Free

of the stimulus and the delay and for every level of uncertainty. Thus two regressors were used: [low /

high uncertainty, neutral], [low / high uncertainty, financial].

V. RESULTS

A. Statistical analysis

All behavioral data were analyzed with Statistica 7.1 (© StatSoft). The Kolmogorov-Smirnov

goodness-of-fit test shown that data distribution was not normal, therefore, the effects of the

financial incentive (neutral vs. financial), of the level of uncertainty (low vs. high) and their interactions

on our dependant variables, the reaction times (RT) and the percentage of landing, were examined

thanks to Friedman’s ANOVA for overall effects and Sign test for paired analyses. The same type of

analysis was also used to examine the effects of the type of stimulus (0%, 100% and 50%) on the same

dependant variables.

B. Behavioral results

1) Effect of Uncertainty on reaction times

The Sign test revealed an effect of uncertainty on RT (p < .023). Higher uncertainty generated

longer mean RT than low uncertainty (see Figure 6).

Figure 6. Reaction times (in ms) according to the two level of uncertainty (low and high).

2) Cross-effect of incentive and uncertainty on reaction times

The Friedman’s ANOVA did not revealed an overall effect of the type of incentive on the RT.

However, the Sign test revealed an effect of the financial incentive on RT (p < .023) for the stimuli

Page 13: Causse Thesis summary - Free

where the landing was obviously possible (100% vs. 100%*). During the financial condition, the

subjects RT were shorter than during the neutral condition, see Figure 7.

Figure 7. Reaction times (in ms) according to the 3 type of stimulus and for the two types of incentive. The asterisk indicates the presence of the financial incentive.

3) Cross-effect of incentive and uncertainty on decision making

In response to the asymmetric payoff matrix, subjects exhibited a significant shift in the likelihood

of accepting landings. More precisely, the Sign test showed that under uncertainty (50% vs. 50%*), the

percentage of landing acceptance increased (p = .026), from 32.093% (SD = 12.06) to 74.03% (SD =

27.85), see Figure 8.

Figure 8. Percentage of landing acceptance according to the level of uncertainty and the type of financial incentive. Concerning the high

uncertainty (50%), results showed a conservative behavior without incentive (under 50% of acceptance) and a risky behavior with incentive (beyond 50% of acceptance). The asterisk indicates the presence of the financial incentive.

Page 14: Causse Thesis summary - Free

C. Functional neuroimaging case study

The behavioral results of the subject that performed the task in the scanner were coherent with

the behavioral group. The RT decreased in the financial incentive condition. Moreover the financial

incentive generated a shift in the likelihood of accepting landings under high uncertainty, from 50% in

the neutral condition to 85% of landing acceptance in the financial condition.

We investigated which brain regions were differently involved in decision-making under monetary

incentive and in the neutral condition by performing overall contrasts that collapsed the time of

choice and the time of the delay. The cold reasoning, performed during the neutral condition was

associated with an increased activity in right DLPFC and occipital cortex. On the contrary, the hot

reasoning, performed under financial incentive was related with an increased activity in bilateral

VMPFC (Figure 9, TABLE I. ).

Figure 9. (A) Increased activation of the bilateral VMPC (BA11) during stimulus and delay for monetary incentive vs. neutral. (B)

Increased activation of the right DLFPC (BA46) during stimulus and delay for neutral vs. monetary incentive (p<0.01; cluster > 15). (C) Global patterns for monetary incentive (in green) and neutral (in red) conditions.

Page 15: Causse Thesis summary - Free

TABLE I. TALAIRACH COORDONINATES, Z-VALUE AND CLUSTER SIZES (K) OF THE ACTIVATED ANATOMICAL STRUCTURES FOR THE CONTRASTS NEUTRAL VS. FINANCIAL AND FINANCIAL VS. NEUTRAL. ALL AREAS WERE SIGNIFICANT AT P < . 01 UNCORRECTED.

Anatomical structures

(Broadman’s area)

Neutral - financial Financial - neutral

Talairach coordinates

Talairach coordinates

x y z z-value k x y z z-value k

Frontal

CPFDLD (BA46) 53 36 19 2.35 22

CPFVMBL (BA11) -12 42 -24 2.33 21

Occipital

V2 (BA18) -33 -85 -12 2.30 15

VI. CONCLUSION

We used an approach borrowed from neuroeconomics to investigate the impact of an economic

pressure, namely the cost of a go-around, on landing decision-making. This preliminary work reports

behavioral results and a case study in fMRI. The behavioral data confirmed the impact of the financial

incentive. Firstly, subjects responded with a slightly faster response time for the financial incentive

condition when the decision to land was obvious (100%), showing more precipitate responses. The

decision to land in this context is rewarded by 5€ and this reduction of the RT may be interpreted as a

search of the reward at the expense of a detailed analysis of the rhombuses positions. Secondly, the

financial incentive has biased responses toward more risky decision-making. Whereas, under

uncertainty, participants are rather conservative during the neutral condition (32.093% of landing),

they took more risky decisions under the influence of the biased payoff matrix. The payoff matrix has

associated the go-around with immediate negative consequences and participants became reluctant

to do it.

The preliminary neuroimaging results confirmed that the change in decision-making entailed by

the financial incentive was subserved by a shift from a cerebral region dedicated to reasoning (DLPFC)

to a region involved in emotion processing (VMPFC). The modulation of occipital cortex activation

gives clues on the possible top-down perturbing effect of monetary incentive on visual information

processing The behavioral data associated to these neuroimaging results are in favor of a shift from a

cold reasoning under the neutral condition to a hot reasoning in presence of the financial incentive.

According to us, this shift can be generalized to pilots and demonstrates that the erroneous trend to

Page 16: Causse Thesis summary - Free

land whereas the approach is not stabilized is the result, at least for a part, of the different aversive

negative consequences associated with the go-around decision, in particular the financial cost for the

company. This is suggesting that this phenomenon may be explained by a temporary perturbation of

decision-making process under an emotional factor. Data from fMRI sessions are currently analyzed

and include a total of 16 participants.

REFERENCES

[1] Dehais, F., Tessier, C., Christophe, L., & Reuzeau, F. (2009). The perseveration syndrome in the pilot’s activity: guidelines and cognitive countermeasures.

[2] Rhoda, D., & Pawlak, M. (1999). An assessment of thunderstorm penetrations and deviations by commercial aircraft in the terminal area. Massachusetts Institute of Technology, Lincoln Laboratory, Project Report NASA/A-2, 3.

[3] Orasanu, J., Martin, L., & Davison, J. (1998). Errors in Aviation Decision Making: Bad Decisions or Bad Luck? RECON(20020063485).

[4] BEA. (2000). Objectif : destination. Technical report, Bureau Enquête Analyse.

[5] Goh, J., & Wiegmann, D. (2002). Human factors analysis of accidents involving visual flight rules flight into adverse weather. Aviation, space, and environmental medicine, 73(8), 817.

[6] Muthard, E., & Wickens, C. (2002). Change detection after preliminary flight decisions: Linking planning errors to biases in plan monitoring.

[7] O'Hare, D., & Smitheram, T. (1995). "Pressing on" into deteriorating conditions: An application of behavioral decision theory to pilot decision making. The International Journal of Aviation Psychology, 5(4), 351-370.

[8] Orasanu, J., Ames, N., Martin, L., & Davison, J. (2001). Factors in Aviation Accidents: Decision Errors. Linking expertise and naturalistic decision making, 209.

[9] Nance, J. (1986). Blind trust: William Morrow & Co.

[10] Sander, D., & Koenig, O. (2002). No inferiority complex in the study of emotion complexity: A cognitive neuroscience computational architecture of emotion. Cognitive Science Quarterly, 2, 249-272.

[11] Damasio, A. (1995). L'erreur de Descartes Paris. Éditions Odile Jacob.

[12] Drevets, W., & Raichle, M. (1998). Reciprocal suppression of regional blood flow during emotional versus higher cognitive processes: Implications for interactions between emotion and cognition. Cognition and Emotion, 12(3), 353-385.

[13] Mayberg, H., Liotti, M., Brannan, S., McGinnis, S., Mahurin, R., Jerabek, P., et al. (1999). Reciprocal limbic-cortical function and negative mood: converging PET findings in depression and normal sadness. American Journal of Psychiatry, 156(5), 675.

[14] Mitchell, R., & Phillips, L. (2007). The psychological, neurochemical and functional neuroanatomical mediators of the effects of positive and negative mood on executive functions. Neuropsychologia, 45(4), 617-629.

[15] Goel, V., & Dolan, R. (2003). Reciprocal neural response within lateral and ventral medial prefrontal cortex during hot and cold reasoning. Neuroimage, 20(4), 2314-2321.

[16] Schoofs, D., Wolf, O., & Smeets, T. (2009). Cold pressor stress impairs performance on working memory tasks requiring executive functions in healthy young men. Behavioral Neuroscience, 123(5), 1066-1075.

[17] Oldfield, R. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 9(1), 97.

Page 17: Causse Thesis summary - Free

Part 2

Economic issues provokes hazardous landing by enhancing emotional decision-making:

evidence from psychophysiological measures

I. METHODS

A. Psychophysiological experimental design

Like in the fMRI study, we used a 2x2 factorial design crossing the type of financial incentive,

neutral or financial, and the level of uncertainty, high, 50% chance of crash, or low, 100% or 0%

chance of crash, depending on both positions of the rhombuses displayed on the PFD. These

likelihoods were unknown by the subjects. The two types of incentive were introduced in two

separate runs and the two level of uncertainty were manipulated within each run. Each trial consisted

in a presentation of the stimulus (2.5 s), during which the participant made his choice thanks to a

response pad, followed after a variable delay (6-10 s) by the feedback informing of the accuracy of the

response (2 s). During the financial condition, the financial outcome {+5€}, {-5€} or {–2€}) was

displayed (1.5 s) after the accuracy feedback (0.5 s). Finally, a variable inter trial interval (3-9 s) was

introduced. Response and financial feedback delivery was contingent upon the subject’s response.

Before the experiment, participants performed two runs (neutral and financial) to become familiar

with the task and the payoff matrix. The training is identical to the behavioral task. During the

experience, after the run with financial motivation, the participants rated how much the payoff matrix

had influenced their decisions on a 1-9 scale.

B. Data acquisition and analysis

Stimulus display and data acquisition were done with Cogent 2000 v125 running under Matlab

environment (Matlab 7.2.0.232, R2006a, The MathWorks, USA). All behavioral data were analyzed

with Statistica 7.1 (© StatSoft).

1) Behavioral data Mean response times (RT) and percentages of landing acceptance were calculated for each level of

uncertainty (low and high) and for each type of stimulus (0%, 100% and 50%). The effects of the type

of incentive (neutral vs. financial), the level of uncertainty and their interactions on our dependant

variables, RT and the percentage of landing, were examined thanks to two-way repeated measures

Page 18: Causse Thesis summary - Free

ANOVA. The same ANOVA were also used to examine the effects of the type of stimulus (0%, 100%

and 50%) on the same dependant variables. The analysis of the effect of the type of stimulus required

separate two-way repeated measures ANOVA because this independent variable was nested within

the level of uncertainty. Fisher's least significant difference post hoc test was used to examine paired

comparisons. Bravais-Pearson correlation was used to estimate the matching between self-perceived

level of influence of the financial incentive and its effective impact on the increase acceptance of

landing between the neutral incentive and the financial incentive.

2) Psychophysiological data Participants were comfortably installed and tested in a moderately lit room, in which the

illumination was held constant (background luminance: about 450 lux). The ProComp Infiniti

(©Thought Technology Ltd.) was used to record continuously the two physiological measures, the

heart rate (HR) and the skin conductance responses (SCRs). Data were digitized at a sampling rate of

256 Hz. Before the experiment, they were asked to relax during 5 minutes to collect the rest state

(baseline).

Skin conductance was recorded using the SC-Flex/Pro sensor, a pair of electrodes placed on the

distal phalanges of digits II and III of the left hand. The SCRs were measured in micro Siemens (μS) and

analyzed off-line. Responses were computed as a change in conductance from the pre-stimulus level

to the peak of the response. Following information provided by Dawson [45], the SCRs were

computed as follow (where G is the skin conductance value and Ts the stimulus onset): Gmax [ts + 3;

ts + 7] - Gmin [ts - 3; ts - 1]. The minimum level occurring within 1-3 s from stimulus presentation was

subtracted from the peak value occurring within a 3-7 s window, an absence of response was

computed as 0. The time course of phasic SCRs being 4–6 s, our long inter-stimulus interval paradigm

allowed avoiding SCRs overlapping.

The heart rate was measured using the EKG-Flex/Pro sensor. Three electrodes connected to an

extender cable were applied on the participant’s chest. We used and Uni-Gel electrodes to enhance

the quality of the signal. This latter was measured in microvolts (μV) and inter-beat-intervals were

converted to beats per minute (bpm).

Mean values were calculated for the heart rate parameter during the complete duration of three

periods of interest: the baseline and the two types of runs, neutral and financial. Two-way repeated-

measures ANOVA were performed to assess the significance of heart rate changes across these three

periods and Fisher's least significant difference post hoc test was used to examine the impact of each

task in comparison to baseline. Separate one-way repeated-measures ANOVA were performed on

Page 19: Causse Thesis summary - Free

delta values (i.e. run – baseline) to compare the significance of physiological changes across the two

types of runs. Then, the effect of the type of incentive, the level of uncertainty and their interactions

on the heart rate was examined during shorter time windows. The mean values were calculated in the

time window between the stimulus onset and 10 s post-stimulus onset. The mean values per

condition of these stimulus locked data were submitted to two-way repeated-measures ANOVA to

examine the effect of the type of incentive, of the level of uncertainty, and their interactions on the

HR. As in behavioral analysis, the effect of the type of stimulus was examined with separate two-way

repeated-measures ANOVA. Fisher's least significant difference post hoc test was used to examine

paired comparisons. In addition, similar separate analyses were conducted for SCRs to assess the

effect of the same independent variables.

II. RESULTS

A. Behavioral results

1) Effect of Uncertainty and incentive on reaction times The ANOVA revealed a main effect (for both type of incentive) of uncertainty on RT (p<.001,

F(1,15) = 45.539). High uncertainty generated longer mean RT than low uncertainty, see Figure 10.

Figure 10. Main effect of the level of uncertainty on reactions times (for both type of incentive, neutral and financial).

The ANOVA revealed a main effect (for both level of uncertainty) of the type of incentive on the RT

(p = .001, F(1,15) = 16.143). During the financial incentive conditions, RT were shorter than during the

neutral condition, see Figure 11. The analysis of the interactions between the level of uncertainty and

the type of incentive did not revealed significant results (p = .006, F(1,15) = 3.930). However, a

separate analysis examining the effect of the type of incentive according to the different type of

stimuli showed a significant interaction (p = .001, F(2,30) = 6.826), see Figure 12. The Fisher's LSD post

hoc test revealed that the decision-making was performed faster under the influence of the financial

Page 20: Causse Thesis summary - Free

incentive than the neutral incentive during the 100% (p = .001) and 50% (p = .004) stimuli. That was

not the case concerning the 0% stimuli (p = .263). These result explain the non significance of the

interaction between incentive and uncertainty and suggests that the RT reduction under the financial

incentive influence is associated to the landing decision but not with the decision to go-around (i.e.

0%).

Figure 11. . Main effect of the type of incentive on reactions times (for both level of uncertainty, low and high).

Figure 12. Reaction times according to the three types of stimuli for * the two type of incentive. There was a main effect of the type of

incentive. More precisely, during the financially motivated condition, the reaction times of the stimuli 100% and 50% were significantly shorter under the financial incentive.

2) Effect of uncertainty and incentive on decision-making The magnitude of the mean total outcome was important (39.31€, SD = 11.42), beyond the

maximum outcome of 30€ that the participants thought they could win. Its demonstrated that the

decision-making were oriented toward economic optimization. Indeed, in response to the asymmetric

payoff matrix, subjects exhibited a significant shift in the likelihood of accepting landings. We

calculated a response bias score: a negative response bias would correspond to a conservative

behavior (less landing acceptance that expected) whereas a positive response bias would correspond

to a risky behavior (more landing acceptance than expected). The ANOVA showed that there was a

Page 21: Causse Thesis summary - Free

main effect of the uncertainty (p < .001, F(1,16) = 121.165) and a main effect of the incentive (p < .001,

F(1,16) = 16.211) on the magnitude of the response bias: there was an overall increased likelihood of

making a landing acceptance when the uncertainty was high and under the financial incentive. In

addition, the ANOVA also revealed an interaction effect between these two variables (p = .021, F(1,15)

= 6.530): the financial incentive increased the likelihood to accept a landing criterion, in particular

when the uncertainty was high (Figure 13).

Figure 13. Response bias according to the level of uncertainty and the type of incentive. A positive response bias demonstrated an increase likelihood of accepting landing.

The separate ANOVA examining stimulus effect showed that there was an overall effect of the type

of incentive on the decision-making performed during the three types of stimuli (p < .001, F(1,15) =

20.242). The Fisher's LSD post hoc test revealed that the landing acceptance rate was increased with

the financial incentive in comparison to the neutral incentive during the three different types of

stimuli: 0% (p=.048), 100% (p=.008) and 50% (p=.001), see Figure 14, TABLE II. And TABLE III. ).

Page 22: Causse Thesis summary - Free

Figure 14. Percentage of landing acceptances according to the three types of stimuli and the type of financial incentive. With the financial

incentive, the landing acceptance rate is higher across the three types of stimuli.

Finally, the Bravais-Pearson correlation showed that there was no significant relationship between

the self-perceived level of influence of the financial incentive and its effective impact on the increase

acceptance of landing between the neutral incentive and the financial incentive (p = .578, r = +.17).

The conscious perception of change in their decision criterion was not consistently related to their

objective behavior.

TABLE II. MEAN REACTION TIMES (IN MS), MEAN PERCENTAGE OF LANDING ACCEPTANCE AND STANDARD DEVIATION ACCORDING TO THE THREE TYPES OF STIMULI AND THE TWO TYPES OF INCENTIVE.

RT (ms)

Low uncertainty High uncertainty

0% 100% 50%

Neutral incentive 1351.06 (±184.85) 1385.63 (±221.59) 1592.43 (±174.45)

Financial incentive 1415.03 (±231.92) 1186.56 (±160.46) 1421.34 (±158.44)

% of landing acceptance

Neutral incentive 2.66 (±7.98) 87.33 (±7.03) 63.66 (±19.12)

Financial incentive 10 (±8.94) 97.5 (±4.47) 85.9375 (±10.03)

Page 23: Causse Thesis summary - Free

TABLE III. MAIN ANOVA RESULTS (P AND F VALUES) USING THE THREE DIFFERENT PREDICTORS AND THEIR INTERACTIONS TO ASSESS THEIR EFFECTS ON TWO BEHAVIORAL MEASURES, REACTION TIMES AND PERCENTAGE OF LANDING (** ≤.01 ; *** ≤.001).

Predictors Level of

uncertainty

Stimulus type Type of

incentive

Level of

uncertainty *

Type of incentive

Stimulus type *

Type of

incentive

RT p <.001*** <.001*** .001*** .067 <.001***

% of landing p

F

/

/

<.001***

802.109

<.001**

20.242

/

/

.013**

5.079

B. Physiological results

Since informatics issue, physiological data were available for only 15 participants.

1) Skin conductance response The ANOVA did not reveal main effect neither of the uncertainty (p = .560, F(1,14) = 0.356) nor the

type of incentive (p < .451, F(1,14) = 0.603). However, the separate ANOVA showed that there was an

effect of the type of stimuli (p < .035, F(2.28) = 3.810). In particular, the Fisher's LSD post hoc test

revealed that 0% stimuli type elicited more important SCRs than 100% ones (p = .011). See Figure 17

for illustration.

2) Heart rate The ANOVA performed on the three periods of interest (baseline, run neutral and run money)

revealed that the HR was significantly different across these three periods (p < .001, F(2.28) = 11.233),

see Figure 15. The LSD post hoc test demonstrated that the mean HR was higher during the two runs

than during the baseline (p < .001 in both comparisons). In addition, the comparison performed on

delta values between the two runs showed that the heart rate was more elevated during the financial

run than the neutral one (p < .009, F(1.14) = 9.570).

Page 24: Causse Thesis summary - Free

Figure 15. Mean heart rate across three periods of interest: at rest (baseline), the neutral run and the financial run. The HR was significantly

lower during the baseline than the two runs and was more elevated during the financial run than the neutral one.

The stimulus locked analyzes did not demonstrated any effect of the uncertainty (p < .102, F(1.14)

= 3.126). However, a main effect of the type of incentive was also found (p = .030, F(1.14) = 6.023),

which is consistent with the previous results found by performing whole run comparisons. The ANOVA

did not revealed main effects of the type of stimuli. The LSD post hoc test showed that (asterisk

indicates the financial incentive) 50%* elicited a higher HR that 0% (p < .001), 100% (p < .001) and 50%

(p = .003); that 100%* elicited a higher HR than 0% (p = .005), 100% (p = .003) and 50% (p = .025) and

finally that HR was more elevated during 0%* stimuli that 0% (p = .006), 100% (p = .004) and 50% (p =

.032). The highest mean heart rate was recorded during 50%* stimuli (74.90 bpm), when the

uncertainty was high and the financial motivation was administered, see Figure 16, TABLE IV. and

TABLE V. ).

Page 25: Causse Thesis summary - Free

Figure 16. Mean heart rate across the three types of stimuli * the two types of incentive. The highest mean heart rate was recorded during

the 50%* stimuli. The asterisk indicates the financial incentive.

Figure 17. Illustration of the skin conductance measure for one subject during a whole run. The last rise corresponds to the display of the global outcome.

Page 26: Causse Thesis summary - Free

TABLE IV. MEAN STIMULUS LOCKED VALUES ACCORDING TO THE THREE TYPES OF STIMULI AND THE TWO TYPES OF INCENTIVE.

TABLE V. MAIN ANOVA RESULTS (P AND F VALUES) USING THE THREE DIFFERENT PREDICTORS AND THEIR INTERACTIONS TO ASSESS THEIR EFFECTS ON TWO BEHAVIORAL MEASURES, REACTION TIMES AND PERCENTAGE OF LANDING (* ≤.05).

Predictors Level of

uncertainty

Stimulus type Type of

incentive

Level of

uncertainty *

Type of incentive

Stimulus type *

Type of

incentive

HR p .102 .177 .030* .083 .096

SCRs p

F

.560

0.356

.035*

3.810

.451

0.603

.788

0.075

.268

1.381

III. DISCUSSION

We used an approach borrowed from neuroeconomics to investigate the impact of an economic

pressure, namely the cost of a go-around, on landing decision-making. In this experiment, both

uncertainty and type of incentive were manipulated. As expected, these variables have significantly

impacted the behavior of participants. The uncertainty generated two types of reasoning: when the

rhombuses positions were non-ambiguous, all participants reported that the decision was

straightforward; on the contrary, when they tried hard to find a rule when the rhombuses positions

were ambiguous. This latter subjective result is supported by the increase of the reaction when the

ambiguity was high. Nevertheless, these results were expected and suggest that low uncertainty

stimuli were easily categorized and high uncertainty stimuli engendered deepest reasoning.

The payoff matrix was designed to incite participants to maximize their monetary reward by

biasing their response criterion (i.e. a bias to indicate that the landing was possible). All subjects

HR

Low uncertainty High uncertainty

0% 100% 50%

Neutral incentive 72,47 (±5.14) 72.38 (±5.67) 72.84 (±6.05)

Financial incentive 74.48 (±7.38) 74.63 (±7.07) 74.90 (±6.21)

SCRs

Neutral incentive 0.62 (±0.58) 0.35 (±0.27) 0.47 (±0.43)

Financial incentive 0.42 (±0.38) 0.36 (±0.33) 0.37 (±0.03)

Page 27: Causse Thesis summary - Free

understood that the payoff matrix biased decisions toward the landing acceptance. They showed a

significant shift in the response criterion, and trend to increase their landing acceptance rate with the

financial incentive, in particular when the uncertainty was high. Whereas their behaviors were rather

conservative in the neutral condition, they made more risky decisions under the influence of the

financial incentive, to avoid the risk of a penalty in case of go-around. In addition, reaction times were

longer in the neutral condition, showing more reasoned decision making. Cold reasoning appeared to

be more analytic, more objective from a safety point of view whereas hot reasoning was associates

with a search of the reward at the expense of a detailed analysis of the rhombuses positions.

The comparison of the mean HR during the baseline with the period of each run has confirm that

this measure is sensitive to the increased activity of SNA in response to the task energy mobilization

and the investment of mental effort to deal with the task [1] [1]. Indeed, the mean HR was

significantly higher during both tasks compared with the resting state.

In addition, the heart rate measure showed that the financial incentive elicited ANS arousal. Both

stimulus locked and whole runs analyzes showed that the mean HR was higher in the financially

motivated condition. This result demonstrated that the financial incentive was linked with a

modulation of the emotional state of the participants. Although the magnitude of the change between

the two runs was small (2.21 bpm) it was nevertheless very significant and persistent on all

participants. It should be noted that the magnitude of this increase is consistent with the literature

related to emotional induction. For instance, Brosschot [3] showed that negative and positive emotion

induction elicited respectively an HR rise of respectively 2.14 and 1.06 bpm. This increase of the HR

between the neutral and the financial incentive cannot be attributable to a significant improvement of

the performances or of the difficulty, in particular in the ambiguous condition. On the contrary, in this

latter condition, the false alarms increased drastically in presence of the financial motivation. Yet,

since the work of Wright, making the link between motivational theory and cardiovascular activity,

many studies shown that cardiac activity is related to the increased difficulty of a task as long as the

success is important and the improvement of the performance is possible. Indeed, the organism

manages the investment of its resources so as not to waste them. In our experiment, there were no

clear limits beyond which a landing could be categorized as possible or not, the visual cues available

on the real PFD has been removed for this purpose. In addition, in ambiguous situation, the

improvement of the performance was not possible because the chance of landing successfully was

50/50. These considerations lead us to think that, beyond an emotional effect, the increase HR reflects

the motivation to maximize financial gains. In this perspective, the payoff matrix leads participants to

Page 28: Causse Thesis summary - Free

waste energy for a purpose that is in total discrepancy with the safety. Indeed, the increase of landing

acceptance was directly linked with increased risk taking. The fact that O% stimuli type elicited more

important SCRs than 100% stimuli suggested that the decision to go-around generated an ANS arousal

than the decision to land, in response to the overall negative context recreated around the go-around

in this experiment. It may argue that participants felt frustration when they were constrained to go-

around. However, no significant effects neither of the uncertainty not of the type of incentive was

found on SCRs. Several explanations main be put forward. Each type of stimuli was repeated a large

number of times to get reliable statistics based. The SCRs are extremely sensitive to novelty, and the

repetition may have decreased progressively the SCRs response magnitudes. Moreover, the

systematic feedback displayed after each response may have also contributed to lower the SNA

responses. For instance, Van der Veen [3] has shown that the display of the feedback during task

performance reduced cardiovascular reactivity.

Our experiment was originally designed to understand pilot trend to land despite wrong

meteorological conditions. Our assumption was that pilots frame their decision to keep on landing in

terms of potential losses (such as money spent of fuel used up). Our behavioral and physiological

results showed that the risky decision to land in pilots may be, at least for a part, explained by the

immediate negative consequences associated with the decision to go-around. An organization that

emphasizes productivity (e.g. on time arrivals or saving fuel) may unconsciously set up goal conflicts

with safety. Pilots may be willing to take a risk with safety (a possible loss) to arrive on time (a sure

benefit). It has been generally assumed that people are risk aversive in their choice. Most people will

be more likely to choose 80 dollars for certain than 85% chance of winning 100 dollars. Kahnenman &

tversky [5] demonstrated those choice criterions are reversed when the sign of outcome is reversed:

faced with a choice between a certain loss of 80 dollars and a gamble, 85% chance of losing 100

dollars, most of people will choose the gamble. Thus, people are only risk averse when considering

possible gains; when faced with the prospect of losses, people suddenly become risk seeking. Our

payoff matrix clearly associates the go-around with expected losses: the mathematical expectation

linked with this decision was either null (0€), when a go-around was necessary, or negative, (-3.5€),

when a landing was possible.

In this experiment, the trial-related design sought to break up the task into three stages: decision-

making, feedback expectancy, and feedback processing. We plan to analyze the physiological data

during these three stages.

Page 29: Causse Thesis summary - Free

REFERENCES

[1] Fairclough, S., Venables, L., & Tattersall, A. (2005). The influence of task demand and learning on the psychophysiological response. International Journal of Psychophysiology, 56(2), 171-184.

[2] Gaillard, A. (2001). Stress, workload, and fatigue as three biobehavioral states: A general overview. Stress, workload, and fatigue. Mahwah, NJ: L. Erlbaum.

[3] Brosschot, J., & Thayer, J. (2003). Heart rate response is longer after negative emotions than after positive emotions. International Journal of Psychophysiology, 50(3), 181-188.

[4] Van der Veen, F., Van der Molen, M., Crone, E., & Jennings, J. (2004). Phasic heart rate responses to performance feedback in a time production task: effects of information versus valence. Biological Psychology, 65(2), 147-161.

[5] Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47(2), 263-291.

Page 30: Causse Thesis summary - Free

Part 3

Flight experience and executive functions predict flight simulator performance and poor

decision-making in general aviation pilots

Submitted to ICRAT 10

Mickaël Causse Centre Aéronautique et Spatial ISAE-SUPAERO ; Université de Toulouse, France

Inserm ; Imagerie cérébrale et handicaps neurologiques UMR 825; F-31059 Toulouse, France Université de Toulouse ; UPS ; Imagerie cérébrale et handicaps neurologiques UMR 825; CHU Purpan, Place du Dr Baylac, F-

31059 Toulouse Cedex 9, France [email protected]

Frédéric Dehais

Centre Aéronautique et Spatial ISAE-SUPAERO ; Université de Toulouse, France [email protected]

Josette Pastor

Inserm ; Imagerie cérébrale et handicaps neurologiques UMR 825; F-31059 Toulouse, France Université de Toulouse ; UPS ; Imagerie cérébrale et handicaps neurologiques UMR 825; CHU Purpan, Place du Dr Baylac, F-

31059 Toulouse Cedex 9, France [email protected]

Abstract— Unlike professional pilots who are limited by the FAA's age rule, no age limit is defined in

general aviation (GA). Some studies revealed significant aging issues on accident rates but these results

are criticized. Our overall goal is to study how the effect of age on executive functions (EFs), high level

cognitive abilities, impacts on the flying performance in GA pilots. This study relies on four components:

EFs assessment, pilot characteristics (age, flight experience, level of impulsiveness), the navigation

performance on a flight simulator and a decision-making performed during the approach. The results

showed that contrary to age, reasoning, working memory (WM) and total flight experience were

predictive of the flight performance. The WM, the flight experience and the level of impulsiveness were

predictive of poor decision-making. These results suggest that “cognitive age” is a better mean than

“chronological age” consideration to predict the ability to pilot, in particular because of the inter-

individual variability of aging impact and the beneficial effect of the flight experience.

Keywords: piloting performance, executive functions, flight experience, decision making, normal aging.

Page 31: Causse Thesis summary - Free

I. INTRODUCTION

The population of GA pilots is getting older in the USA [1] and in European countries like France

where forty one percent of private pilots are more than fifty (BEA1, 2008). Unlike professional pilots

who are limited by the FAA's age rule, no such restriction exists for GA pilots. Moreover, contrary to

commercial aviation (CA) pilots, GA pilots have not necessarily experienced a professional training, fly

mostly on their own, without any co-pilot and very few assistance systems, have less support from the

air traffic control and are more affected by weather conditions. Not surprisingly, in GA, the accident

rate is considerably higher than in CA [2].

Several studies have revealed significant aging issues on accident rates in GA [3] [4] [5], though

these results are called into question [6] [7]. The assessment of the cognitive functioning is a key issue

in pilot’s aging as long as its decline represents a much higher risk of accidents than sudden physical

incapacitation [8]. A substantial literature focuses on the evaluation of the cognitive state of pilots but

its conclusions remain contradictory. Several reasons may explain the difficulty to draw a definitive

conclusion on the effects of aging on flight performance in GA pilots. There is a great inter-individual

variability in the deleterious effects of aging on cognition [9]; the evaluations performed in classical

human factors studies are rather nonspecific in terms of explored cognitive functions and do not

necessarily focus on the ones that are the most impacted by aging; very few researches attempt to

link, in the same population, the cognitive performances to the flight abilities; the greatest part of the

studies is interested on safety aspects like communications [10], or decision making during landing

[11]; few researches are exclusively related to the GA population; finally, another source of complexity

arises from the suspected compensative role on aging effects of the flight experience [12].

II. COGNITIVE FUNCTIONS AND PILOTING

Numerous studies have been conducted to attempt to link the cognitive functioning with the flight

performance. Different measurements of cognitive efficiency have been identified as crucial to the

piloting ability, for example: time-sharing [13], speed of processing [14], attention [15] or problem

solving [16]. Cogscreen-AE [17] is among the most widely used cognitive tests batteries in pilots aging

studies. It consists of a series of computerized cognitive tasks that evaluate a large set of cognitive

functions. This battery has been shown to be able to successfully discriminate between neurologically

impaired and cognitively intact pilots [18]. Some Cogscreen-AE variables were predictive of flight

parameter violation in Russian CA pilots [19]. Furthermore, Taylor and colleagues [20] were able to

1 French Accident Investigation Bureau.

Page 32: Causse Thesis summary - Free

predict 45% of the variance of the flight simulator performance with four Cogscreen-AE predictors

(speed/WM, visual associative memory, motor coordination and tracking) in a cohort of 100 aviators

(aged 50-69). Contrary to this latter study that involved Cogscreen-AE, a rather generalist battery in

terms of explored cognitive functions, we propose to focus specifically on EFs. Indeed, these functions

are the earliest ones to be impacted by aging [21] and represent excellent clues of aging effects on the

cognitive performance. The study of EFs has appeared recently in aeronautics, for instance, Hardy [22]

found significant age-related differences in pilots’ executive functioning (e.g. inhibition, set-shifting)

and Taylor [23] established a relationship between interference control and the ability to follow air

traffic instructions.

III. EXECUTIVE FUNCTIONS AND PILOTING

EFs underlie goal-directed behavior and adaptation to novel and complex situations [24]. They

allow the inhibition of automatic responses in favor of controlled and regulated behavior, in particular

when automatic responses are no more adapted to the environment. Three major low level EFs are

moderately correlated with one another, but clearly separable [25]: set-shifting between tasks or

mental sets (“shifting”), inhibition of dominant or prepotent responses (“inhibition”), and updating

and monitoring of WM representations (“updating”). The prefrontal cortex (PFC) plays a dominant

role in the implementation of EFs that also encompass decision-making [26] or reasoning abilities [27].

According to our hypotheses, EFs are crucial to piloting. Indeed, this activity takes place in a dynamical

and changing context where new information must be integrated and updated continuously. We

assume that flying light aircraft with no autopilot and very few assistant systems (like the TCAS2 or

weather radar) presupposes a strong involvement of the EFs for handling the flight, to monitor the

engine parameters, to plan the navigation, to maintain and update situation awareness and to

correctly adapt to traffic and environmental changes and perform accurate decision-making by

inhibiting wrong behavioral responses.

IV. EXECUTIVE FUNCTIONS AND NORMAL AGING

Functional neuroimaging brings evidence that the brain is subject to anatomical and physiological

modifications in normal aging [28]. The prefrontal lobes appear to be the earliest cerebral regions to

be affected [29] and may account for a great part to age-related cognitive changes [31]. Because the

prefrontal lobes mainly implement EFs, aging is suspected to provokes a selective alteration of these

latter, for example the reasoning [30], inhibition [31] or updating [32] abilities. However, the executive

changes vary considerably across people. The complex interactions between the cerebral structures

2 Traffic Collision Avoidance System.

Page 33: Causse Thesis summary - Free

underlying EFs [9], sociocultural factors and genetic factors [33] may explain the heterogeneity of this

decline.

In this experiment, we proposed to evaluate specifically the EFs, high level cognitive abilities that

present a strong vulnerability to aging effects [21]. More precisely, we assessed three low levels EFs

(shifting, inhibition and updating) and a more established general ability: the reasoning. The reasoning

performance reflects fluid intelligence, that support processes relevant for many kinds of abilities

(verbal, spatial, mathematical, problem solving etc.) and adaptation to novelty. It is a concept very

close to the executive functioning [34] [35]. The speed of processing was also collected because it

represents a reliable measure of general cognitive decline during aging. Finally, we have also taken

into account age and the total flight experience to assess their respective participation to the flight

performance variation. Our hypothesis is that the “chronological age” is not a sufficient criterion to

predict the piloting performance and that the “cognitive age”, evaluated by the cognitive functioning,

is a more relevant criterion.

V. IMPULSIVITY

According to Sicard [36], flight safety is dependent on the quality of the decision-making process,

which is closely related to risk taking. Impulsive individuals are more likely to make risky decisions,

choosing immediate rewards despite potential long-term negative consequences [37]. Impulsivity is a

personality characteristic described as ‘‘acting without thinking”. Impulsive individuals make risky

decisions, motivated more by immediate reward rather than by the potential long-term negative

consequences of their choices, suggesting heightened sensitivity to reward and/or reduced sensitivity

to negative outcomes [38]. Several studies have highlighted the close relationship between trait

impulsiveness and decision-making. For instance, Martin [39] shown that high impulsive people do not

present error related negativity (an event related potential obtain with an electroencephalogram)

contrary to low impulsive people during risky choice, suggesting that low impulsive individual

evaluated risky choice as a poor decision whereas high impulsive individuals are biased towards

immediate reward and are less sensitive to the negative consequences associated with their choice.

Interestingly, some executive functions, like planning and impulse control, has shown to be predictive

of the level of trait impulsiveness [40], showing that this psychological trait is sustained by

neuropsychological functions. In addition, executive functions may modulate the decision making

style. For instance, Hinson [41] showed that limits on working memory (WM) function are predictive

of a more impulsive decision-making style. In this way, Keilp [42] has shown that the Go–No Go task,

verbal fluency, executive function measures and tasks requiring decision-making against time were

Page 34: Causse Thesis summary - Free

the strongest correlate of self-rated impulsiveness. These measures remained linked with the level of

impulsiveness after adjustment for age and education. According to the authors, these findings

suggest that trait impulsiveness is effectively assessed by tasks that require decision-making and

response organization under time pressure. The piloting activity takes place in a dynamic environment

where decision-making are often performed under time pressure. Our hypothesis is that trait

impulsiveness is a psychological trait that may strongly modulates the relevance of decision-making, in

particular during approach and landing where the time pressure is important.

VI. METHODS

A. Participants

The participants were 24 private licensed pilots (age 43.3, SD = 13.6) rated for visual flight

conditions. The pilots that had no longer flown during the past two years were excluded because of

the potential impact on flight simulator performance. Inclusion criteria were male, right handed, as

evaluated by the Edinburgh handedness inventory [43], native French speakers, under or

postgraduate. Non-inclusion criteria were expertise in logics, airline pilots and sensorial deficits,

neurological, psychiatric or emotional disorders and/or being under the influence of any substance

capable of affecting the central nervous system. All subjects received complete information on the

study’s goal and experimental conditions and gave their informed consent. Given that flight

experience may moderate age related deficits in the performance of domain relevant task [12], we

attempted to homogenize the flight experience distribution across the life span of our sample.

B. Flight performance

1) Navigation The flight scenario has been setup in collaboration with flight instructors to reach a satisfying level

of difficulty and realism. To familiarize the participants with the PC-based flight simulator and

minimize learning effects in order to obtain reliable flight simulator performances, each volunteer

underwent a training session. Before the navigation, they received the instructions, a flight plan and

various technical information related to the aircraft (e.g. aircraft's crosswind limit). Basically, the

scenario implied to take off, reached a waypoint with the help of the aircraft radio navigation system

and finally, land on a given airport. The pilots were instructed that they were in charge of all the

decisions and that they could only received an informative weather report before landing. In order to

increase the subject’s workload, the pilots had to perform a mental arithmetic calculation of the

ground speed (thanks to the embedded chronometer). Moreover, a failure of the compass was

scheduled. After this failure, the pilots had to navigate thanks to the magnetic compass, which

presents the particularity to be difficult to use as it is anti-directional. The flight scenario lasted

Page 35: Causse Thesis summary - Free

approximately 45 min. The performance assessment was exclusively founded on the flight path

deviations (FPD), expressed in terms of amount of angular deviation in X axes from the ideal flight

path.

2) Crosswind landing decision After the waypoint and before reaching the runway threshold, the pilots must state if the

meteorological conditions, as provided by the automatic information system of airport arrival, were

compatible with a landing or necessitate a go-around and a diversion. In this purpose, the pilots

should assess the crosswind component using a commonly used formula. The calculation result

exceeded of 6 knots the aircraft's maximum crosswind limit, as specified in the documentation

provided to the pilots at the time of the flight preparation. The measured dependant variable was

binary: correct when the pilot decided to divert before the runway threshold, incorrect when the

pilots continued their landing beyond the runway threshold.

C. Pilots impulsiveness measurement

The level of impulsivity of the pilots was measured by the French version of the Barratt

Impulsiveness Scale [44]. This test includes 34 items wich may be scored to three first-order factors:

cognitive (quick decision, 11 items), motor (acting without thinking, 11 items) and non-planning

impulsiveness (present orientation, 12 items).

D. Neuropsychological battery

1) Target hitting This test provides a basic psychomotor reaction time [45]. The instruction is to click as fast as

possible on each target. The performance is measured by a velocity index inspired by the Fitts’ law

[46]. The index is the average ratio of the base 10 logarithm of the distance in pixels between two

targets, divided by the time in seconds to go from the first target to the second.

2) The 2-back test. The 2-back test aims at assessing working memory (WM), in particular maintenance and updating

abilities [47]. Subjects view a continuous stream of stimuli and have to determine whether the current

stimulus matches in a specific dimension (shape for our test) the stimulus 2-back in the sequence

(Figure 18). For each condition, the percentage of correct responses was collected.

Page 36: Causse Thesis summary - Free

Figure 18. The 2-back test. The participant stated if the current shape match to the 2-back shape in the sequence thanks to the response box.

3) Deductive reasoning The logical reasoning test has been used in a previous study to assess executive functioning [48].

The goal of the task is to solve syllogisms by choosing, among three suggested solutions, the one that

allows concluding logically. Syllogisms are based on a logical argument in which one proposition (the

conclusion) is inferred from a rule and another proposition (the premise). We used four existing forms

of syllogisms: modus ponendo ponens, modus tollendo tollens, setting the consequent to true and

denying the antecedent. Each participant had to solve 24 randomly displayed syllogisms. The

measurement was the percentage of correct responses.

4) The computerized Wisconsin Card Sorting test The Wisconsin Card Sorting test (WCST) [49] gives information on the subject’s abstract reasoning,

discrimination learning and shifting abilities [50]. The test version here was a computer

implementation very similar to the clinical version of the WCST [51]. The participant must sort cards

according to three different unknown categories (color, shape, number); an audio feedback indicated

whether the response is correct or not (yes/no). When the participant categorized successfully ten

cards, the target category was automatically changed. The task ended when six categories was

achieved (color, shape, number, color, shape, number) or when the deck of 128 cards was used. The

total numbers of perseverative errors (at least two unsuccessful sorting on the same category) was

derived from the individual cards’ records (Figure 19).

Page 37: Causse Thesis summary - Free

Figure 19. The Wisconsin card sorting test. The participant sorted the cards according to a specific dimension. An audio feedback informed if

the sorting was correct or no.

5) Spatial stroop Spatial Stroop tests generally assess the conflict between the meaning of a word naming a location

(e.g. “below”) and the location where the word is displayed. The ability to restrain a response

according to the localization of the word gives information on inhibition efficiency. This conflict

appears to be provoked by the simultaneous activation of both motor cortices [52]. Our test

encompasses four control conditions (Figure 20). “Stroop neutral meaning” (SNM): a motor answer is

given with the appropriate hand according to the word meaning; “Stroop neutral position” (SNP): the

response is given according to the location of a string of XXXXX, displayed at the left or the right of the

screen; “Stroop meaning incompatible/compatible” (SMI/SMC): the response is given according to the

meaning of the word, compatible or incompatible with is location at the screen. In order to get the

pure effects of inhibition, the interference score was calculated to control reading and localization

effects by:

Page 38: Causse Thesis summary - Free

Figure 20. The four conditions of the spatial stroop. On the left: SNM, the participant pressed on the left/right button according to the

meaning of the word; SNP, the participant pressed the left/right button of the response box according to the location at the screen of the pattern of XXXXX. On the right: SMC/SMI, the participant pressed the left/right button according to the meaning of the word, congruent or

incongruent with its location at the screen.

E. Pilots caracteristics

Age and total flight experience in hours were collected to assess their effects on the flight

performance. We attempted to homogenize the flight experience distribution across the life span of

our sample in order to minimize the perturbation of this parameter on the flight performance

measurement.

VII. RESULTS

A. Statistical analysis

All data were analyzed with Statistica 7.1 (© StatSoft). The relationship between age and the total

flight experience was examined thanks to Bravais-Pearson correlation. The ability of our control

variables to predict the piloting performance was tested using exhaustive regression (ER) that

searches for the best possible fit between a dependent variable and a set of potential explanatory

variables. ER searches the entire space of potential models and returns those for which all parameter

estimates are statistically significant. The goodness of fit of the models was evaluated by the adjusted

coefficient of determination r². The relationship between age and neuropsychological variables was

examined thanks to Bravais-Pearson correlation. The Bonferroni-Holm [53] correction was applied to

control the familywise error rate. The landing decision being a Boolean decision, we have performed

Page 39: Causse Thesis summary - Free

discriminant analysis to examine which variables discriminate between the pilots that had erroneously

land and the pilot that goes-around.

B. Pilots characteristics

The mean level of education of our sample was high (15.45 years, SD = 2.06) and did not

significantly correlated with age (p =. 110, r = -.32). The mean Barrat total score was 41.85 (SD = 8.89)

and did not evolve significantly with age either (p =. 129, r = -.35). This impulsiveness mean score is

remarkably low compared to those of the general male population which is of 54 (SD = 17) [44].

C. Aging and executive functions

A first objective of this study was to evaluate the effects of normal aging on executive functioning,

domain independent skills. This type of analysis on general aviation pilots is relatively uncommon and

is interesting because general aviation pilots is usually a population that presents a high level of

education, supposed to be protective from aging effects. In spite of this high level of education, in our

experiment, the pilots demonstrate a significant decrease in executive functioning in nearly every

neuropsychological measure. Consistently with literature (ex: [54] [55] [56] [57] [58] [59]), the

updating in WM performances was strongly impacted by age, as shown by the decrease of the

percentage of correct answers in the 2-back test with aging. This decline is particularly obvious from

55, where six out of seven participants had a correct response rate below 70%, which is not the case

for any other of the younger participants. This is consistent with Hardy results which showed that

despite of the observation of cognitively preserved people, the number of outliers in WM

performances terms increases significantly after 50. This 2-back test performances decline highlights

an increase difficulty to update the WM content but also in maintenance abilities.

This cognitive decline, despite a high level of education may appears counterintuitive. However,

Tucker-Drob [60] has shown that a high level of education has no neuroprotective effect. According to

the author, education increases cognitive skills in advanced age, but do not influences the rate of the

cognitive decline. In other words, the best cognitive performances bring by a high level of education

are the manifestation of the persistence of higher cognitive abilities already present in the past. It is

not surprising that linear regression were able to find a significant decline with age in our population.

The speed of processing was also reduced with age. This is also a classic effect of normal aging [61]

[62] [63] and it is considered as a factor of cognitive deterioration, and not as a symptom of this

degradation. According to Salthouse [64], the speed at which the neural connections are performed

determines the number of nervous centers that cooperate at the same time, and therefore,

modulates the intellectual performance.

Page 40: Causse Thesis summary - Free

The inhibition performances were also found to be affected by age, as demonstrated by the

increase of the interference score. This decline occurs strongly after 55, where five pilots had a higher

interference rate than any other pilots. Various studies have shown that older adults demonstrate a

greater sensitivity to interference compared to young adults [65] [66]. The lack of inhibition is

considered by some authors as a major cause of decline in cognitive performance. Finally, there was

also a significant increase of perseverative errors, linked with set-shifting performances. This result

must be considered with caution: the correlation was significant mainly in reason of four outliers of

more than 50 that made a high number of perseverative errors. The reasoning abilities seemed to be

relatively preserved, the decline of correct response during the syllogism resolution task was only near

significant (p = .066). This may be explained by the relative low mean age of our sample (43.3, SD =

13.6). Indeed, De Neys [67] has shown that the reasoning performance began to decline drastically

after 65, which is precisely the age of our oldest pilot.

D. Age and experience relationships

The mean total experience of our sample was of 1676 hours of flight (Range = 57-13000). The

Bravais-Pearson correlation revealed that there was no relationship between age and total flight

experience. However, in particular because of three aged pilots that owned a large total flight

experience (respectively 61 and 13000 hours; 61 and 5000 hours; 58 and 6700 hours), the correlation

was close to reach the significance (p = .0561, r = +.39).

E. Correlation of aging and executive functions

With the exception of the reasoning performances, all the neuropsychological variables were

significantly correlated with the age. Bravais-Pearson correlation shown that the three low level

executive functions performances ― updating in WM (p < .001, r = -.73), inhibition (p = .011, r = +57)

and set-shifting (p = .034, r = +.48) ― decreased with age. The speed of processing was also

significantly reduced with age (p < .001, r = +71), see Figure 21, Figure 22, Figure 23 and Figure 24 and

TABLE VI. That was not the case of the reasoning performance that solely showed a trend to decline

(p = .066).

Page 41: Causse Thesis summary - Free

Figure 21. Updating working memory performances as a function of age.

Figure 22. Psychomotor velocity index as a function of age.

Figure 23. Interference score as a function of age.

Figure 24. Number of perseverative errors as a function of age

TABLE VI. CORRELATIONS OF CHRONOLOGICAL AGE WITH THE FIVE NEUROPSYCHOLOGICAL INDICES OF PERFORMANCES (* ≤.05 ; *** ≤.001).

Variables Corrected p-value r

UP in WM < .000*** - .73

Speed of processing < .000*** - .71

Interference = .011* + .57

Set-shifting = .034* + .48

Reasoning = .066 - .38

F. Explanatory variables of the piloting performance

The mean FPD amplitude was 27.69 (SD = 10.38). The ER revealed that the performances of two

cognitive abilities were predictive of the FPD: the reasoning and the WM (respectively, p = .0083,

Page 42: Causse Thesis summary - Free

F(1,15) = 9.20, t = -3.03; p = .0395, F(1.15) = 5.08, t = -2.25). Moreover, the total flight experience was

also a significant explanatory variable (p = .0275, F(1,15) = 5.95, t = -2.44), see Figure 25.

The most the reasoning (see Figure 26 and Figure 27) and the WM abilities were efficient, the

smaller was the FPD. In the same way, the most the pilots were experienced, the smaller was the FPD.

The adjusted r² showed that this model explained 44.51% of the FPD.

As expected, the ER did not revealed any significant effect of age on the piloting performance (p =

.2488, F(1,15) = 5.95, t = -1.19). In the same way, the speed of processing and the two others low level

EFs, set-shifting and inhibition, were not predictive of the flight performance (respectively, p = .5603,

F(1,15) = 0.35, t = -0.59; p = .8979, F(1,15) = 0.17, t = -0.13; p = .9008, F(1,15) = 0.16, t = -0.12), see

Figure 25.

It is interesting to note that the worst piloting performance (FPD = 52.01) has been done by a

rather old pilot (62) with a very small total flight experience (90 hours) whereas two others aged pilots

(both 61) with a high experience (13000 and 5000 hours) demonstrated correct flight performances

(respectively FPD = 21.08 and 32.30).

Figure 25. Synthesis of the ER. The Pareto diagram shows the three predictive variables of the flight performance: the reasoning abilities, the updating and the total flight performance.

Page 43: Causse Thesis summary - Free

Figure 26. FPD as a function of the reasoning performances. The ER revealed that the reasoning performance predicts significantly the FPD.

Figure 27. Flight path of two pilots and their respective reasoning performances. In blue, the pilot had a small flight path deviation and a good reasoning performance (83.3% of correct answers). In pink, the pilot had a large flight path deviation, he lost himself and flew by

mistake above the Blagnac airport. His flight path deviation was very important and his reasoning performances were very low (41.6% of correct answers). Flight path are rendered with FromDady [68], the width of the line codes the altitude.

G. Discriminative variable of the crosswind landing decsion

The discriminant analysis revealed that three variables were predictive of the correct decision do

go-around: updating in WM, the flight experience and the level of motor impulsiveness. The pilots

who made the good decision to go-around demonstrated a better percentage of correct answers in

Page 44: Causse Thesis summary - Free

the 2-back task, a larger total flight experience and a lower motor impulsiveness compared to pilots

who made a poor decision (respectively p < .001, F(1,10) = 21.27, t = + 4.61 ; p = .003, F(1,10) = 14.77,

t = + 3.84 ; p = .037, F(1,10) = 5.73, t = -2.39) (see TABLE VII. The non-planning impulsiveness was

nearly significantly predictive of the decision (p = .059). Neither the age nor the reasoning

performance were predictive of the relevance of the decision making (respectively, p = .534; p = .728).

The classification matrix show that this model correctly discriminate 100% of the pilots who made a

poor decision and 91.6% of the pilot who choose to go-around. It is interesting to note that a separate

analysis showed that the level of education was negatively correlated to the total score of

impulsiveness (p = .043, r = -.456).

TABLE VII. SUMMARY OF DISCRIMINANT ANALYSIS BY EXHAUSTIVE SEARCH: NEUROPSYCHOLOGICAL INDICES OF PERFORMANCES THAT PREDICT CROSSWIND LANDING DECISION (* ≤.05 ; ** ≤.01 ; *** ≤.001).

Variables β Standard error F(1,14) t p

Age .224 .172 .417 .646 .534

TFE .925 .113 14.263 3.776 .004**

Motor Imp. -.627 .123 6.528 -2.555 .030*

Cognitive Imp. .041 .102 .042 .205 .841

Non-planning Imp. -.475 .110 4.630 -2.151 .059

Speed of processing .268 .132 .928 .963 .360

Reasoning -.144 .116 .486 -.697 .503

Up in WM 1.551 .162 20.676 4.547 .001***

Set-shifting -.379 .112 2.584 -1.607 .142

Inhibition .264 .130 1.072 1.035 .327

VIII. DISCUSSION

A. Aging and piloting performance

According to our hypotheses and other authors [6] [7], the chronological age was not a relevant

variable to predicts the piloting performance. However, although the total flight experience was not

correlated with age, it may have played a beneficial effect on some aged pilots. It is interesting to note

that the worst piloting performance has been performed by a rather old pilot with a weak experience,

whereas two others aged pilots, with a high experience, demonstrated quite good flight

performances. In spite of these observations, our results raised the limitation of using the

chronological age as a single criterion to decide if a given pilot is able to fly or not. In accordance with

such statement, Schroeder [69] have pointed out the necessity to use neuropsychological tests rather

than relying on chronological age.

B. Neuropsychological tests and piloting performance

The pilots performed a neuropsychological battery that taped three crucial low-level executive

functions [25] plus reasoning and speed of processing. Finally, as revealed by the ER, reasoning

Page 45: Causse Thesis summary - Free

performance was the variable the most predictive of the ability to pilot in our study. This result is not

surprising, the reasoning abilities were strongly involved in our scenario. The pilots ought to perform

numerous observations during the navigation to estimate their position and they had to use the radio

navigation systems to reach a waypoint. Moreover, the scheduled compass failure required pilots to

use the anti-directional magnetic compass as a backup. The utilization of this instrument is complex

and could be a source of difficulty. Although we did not assess precisely the errors associated with the

use of this instrument, it seems likely that it has participated to increase the path deviation of some

pilots. These results concerning the reasoning are in line with Wiggins and O’Hare [70] that have

highlighted the links between reasoning performance, evaluated by a syllogism resolution (duncker’s

candle problem), and piloting performance. The reasoning performances reflect fluid reasoning,

central cognitive ability linked with various types of mental activity (mental calculation, problem

solving etc.) and essential to the adaptation to novel problems. Complex and novel problems cannot

be solved directly by referring to a store of long-term knowledge but require analytic or fluid

reasoning. The complexity of our scenario with unexpected event like the compass failure appears to

have contributed to a strong involvement of reasoning abilities.

The total flight experience was also predictive of the FPD. In accordance with other studies [12],

this data has confirmed the beneficial impact of experience on flight performance. This is coherent

with Taylor’s results [5] that showed that more expert pilots demonstrated better flight summary

scores, especially in the communication and approach-to-landing. Moreover, this 3- year longitudinal

study showed that aviation expertise was associated with less declines in flight simulator performance

over time.

Finally, updating ability was also linked with the pilot’s performances. This is coherent with our

expectation. Indeed, the pilot’s activity takes place in a dynamical and changing context where new

information must be integrated and updated continuously. The updating performances are crucial in

this context. Another study of Taylor et al. [20] found that the WM and the speed of processing were

predictive of the piloting performance. We are partially in line with these results. We did not retrieve a

significant effect of the speed of processing. The mean age of our sample was relatively low (43.3, SD

= 13.6) and only seven participants of more than fifty were involved in the experiment. We may argue

that more severe aging effects on speed of processing occur later in life, the sample of Taylor was

more extreme and included participants from 50 to 69, these latter probably demonstrated more

pronounced variations of speed of processing. Moreover, the task that we used to assess the speed of

Page 46: Causse Thesis summary - Free

processing had a strong motor component that could have been less relevant to flight performance

assessment.

Our overall results suggest that “cognitive age” is a better criterion than “chronological age” to

predict the ability to fly and that reasoning and updating are good candidate to assess the cognitive

age. The design of such neuropsychological batteries of tests that could be administrated during the

pilot’s periodic physical examinations could help to detect cognitive impairment associated with

increased risk of accidents.

C. The landing decision relevance

Many studies have tried to understand the difficulty of pilots to review their flight plan [71] [72]

[73] [74], especially during the final approach [75]. The updating in WM, the flight experience and the

motor impulsiveness were predictive of the landing decision relevance. Muthard [73] had already

mentioned the great difficulties encountered by some pilots to integrate critical contextual changes

such as deteriorating weather. The fact that WM performances were linked with the erroneous

decision to continue the landing is in line with this author. Indeed, updating in WM performances may

modulate the ability to integrate the meteorology evolution during the piloting scenario course.

Moreover, the aircraft's maximum crosswind limit was not recollected in WM during the critical time

of the approach. This inability to recollect critical information and to update their situation awareness

seems to lead pilots to erroneously continue the landing. The mental calculation of the crosswind is

not to incriminate, no error of that type was observed in our pilot, this is consistent with the fact that

reasoning performance was not a predictive variable. This result provides new insight into the get-

home-itis syndrome, particularly hazardous in commercial aviation [75] and general aviation, where it

is accounting for over 41.5% of casualties [76]. Assessing the integrity of the pilot’s WM seem to be a

major challenge for aviation safety since many studies have also demonstrated that its predicts the

flight performance [71] [72] [73] [74], or the ability to perform radio communications [77][78][79].

The total flight experience was also predictive of the landing decision relevance. Indeed, that was

the least experienced pilots who were more likely to make poor decision. These results are quite

consistent with those of Wiegman [80] who has shown that the time and distance traveled into

adverse weather prior to diverting were negatively correlated with pilots' flight experience. These

findings and our results suggest that landing continuation may be attributable, at least in part, to poor

situation assessment, consequence of a lack of experience.

Finally, the level of motor impulsiveness, habitual propensity of a person to act without fully

considering the consequences of his actions, was also a relevant predictor of the erroneous decision

Page 47: Causse Thesis summary - Free

to land. To our knowledge, no study has linked the level of impulsiveness with the plan continuation

error. According to Sicard [36], increased impulsivity is deleterious to the relevance of decision-making

because of increased risk-taking that it generates. Goh [71] highlighted the fact that overconfidence is

seven times more frequently cited as the main cause of accidents in those of the VFR-IMC (Visual

Flight Rules - Into instrument meteorological conditions) category. Laboratory experiments have

shown that pilots that present a trend to fly in VFR-IMC evaluate themselves as more skilled and think

that they make more relevant decision than those who does not demonstrate a trend to fly in VFR-

IMC. O’Hare and Smitheram [81] support the hypothesis that the accidents that occur in degraded

condition type VFR-IMC are largely due to poor risk assessments.

The identification of the brain mechanisms related to pilot error requires much progress. However,

the definition of such batteries of neuropsychological tests, administered to pilots during their medical

examination, would be a significant step forward in terms of aviation safety, particularly when obvious

executive decline are observed. In the same way, psychological evaluation like the impulsiveness level

seems to be relevant. A future protocol will include a larger sample of pilot, including older ones.

ACKNOWLEDGMENT

The authors wish to thank all the pilots who participate to this experiment. The study was

supported by a “Gis Longévité” grant, by the DGA grant 0434019004707565 and the Midi-Pyrenees

Regional Council grants 03012000 and 05006110.

Page 48: Causse Thesis summary - Free

REFERENCES

[1] Hardy, D. J., & Parasuraman, R. (1997). Cognition and flight performance in older pilots. Journal of experimental psychology applied,

3, 313-348.

[2] Li, G., & Baker, S. (2007). Crash risk in general aviation. Jama, 297(14), 1596.

[3] Harkey, J. A. Y. (1996). Age-related changes in selected status variables in general aviation pilots. Transportation Research Record:

Journal of the Transportation Research Board, 1517(-1), 37-43.

[4] Kay, G. (2001). Effects of aging on aviation-related cognitive function in commercial airline pilots. Paper presented at the In

Proceedings of the 72nd Meeting of the Aerospace Medical Association, Aviat Space Environ Med 2001; 72 :241.

[5] Taylor, J., Kennedy, Q., Noda, A., & Yesavage, J. (2007). Pilot age and expertise predict flight simulator performance: A 3-year

longitudinal study. Neurology, 68(9), 648.

[6] Li, G., Baker, S., Grabowski, J., Qiang, Y., McCarthy, M., & Rebok, G. (2003). Age, flight experience, and risk of crash involvement

in a cohort of professional pilots. American journal of epidemiology, 157(10), 874-880.

[7] Li, G., Baker, S., Lamb, M., Grabowski, J., & Rebok, G. (2002). Human factors in aviation crashes involving older pilots. Aviation,

space, and environmental medicine, 73(2), 134-138.

[8] Schroeder, D., Harris, H., & Broach, D. (2000). Pilot age and performance: an annotated bibliography. Oklahoma City, OK: Federal

Aviation Administration, Civil Aeromedical Institute. Human Resources Research Division.

[9] Buckner, R. L. (2004). Memory and Executive Function in Aging and AD Multiple Factors that Cause Decline and Reserve Factors

that Compensate. Neuron, 44(1), 195-208.

[10] Morrow, D., Menard, W., Ridolfo, H., Stine-Morrow, E., Teller, T., & Bryant, D. (2003). Expertise, cognitive ability, and age effects

on pilot communication. The International Journal of Aviation Psychology, 13(4), 345-371.

[11] Wiggins, M., & O’Hare, D. (1995). Expertise in aeronautical weather-related decision making: A cross-sectional analysis of general

aviation pilots. Journal of Experimental Psychology: Applied, 1(4), 305-320.

[12] Morrow, D., Menard, W., Stine-Morrow, E., Teller, T., & Bryant, D. (2001). The influence of expertise and task factors on age

differences in pilot communication. Psychology and Aging, 16(1), 31-46.

[13] Tsang, P., & Shaner, T. (1998). Age, attention, expertise, and time-sharing performance. Psychology and Aging, 13, 323-347.

[14] Taylor, J., Yesavage, J., Morrow, D., Dolhert, N., Brooks III, J., & Poon, L. (1994). The effects of information load and speech rate on

younger and older aircraft pilots' ability to execute simulated air-traffic controller instructions. The Journal of Gerontology, 49(5), P191.

[15] Knapp, C., & Johnson, R. (1996). F-16 Class A mishaps in the US Air Force, 1975-93. Aviation, space, and environmental medicine,

67(8), 777.

[16] O'Hare, D., Wiggins, M., Batt, R., & Morrison, D. (1994). Cognitive failure analysis for aircraft accident investigation. Ergonomics,

37(11), 1855-1869.

[17] Horst, R., & Kay, G. (1991). COGSCREEN- Personal computer-based tests of cognitive function for occupational medical

certification.

[18] Kay, G. (1995). CogScreen: Aeromedical Edition. Odessa, Florida: Psychological Assessment Resources: Inc.

[19] Yakimovitch, N., Strongin, G., Go'orushenko, V., Schroeder, D., & Kay, G. (1994). Flight performance and CogScreen test battery in

Russian pilots.

[20] Taylor, J., O'Hara, R., Mumenthaler, M., & Yesavage, J. (2000). Relationship of CogScreen-AE to flight simulator performance and

pilot age. Aviation, space, and environmental medicine, 71(4), 373.

[21] Raz, N. (2000). Aging of the Brain and Its Impact on Cognitive Performance: Integration of Structural and Functional Findings. The

handbook of aging and cognition.

[22] Hardy, D., Satz, P., D Elia, L., & Uchiyama, C. (2007). Age-Related Group and Individual Differences in Aircraft Pilot Cognition. The

International Journal of Aviation Psychology, 17(1), 77-90.

[23] Taylor, J., O’Hara, R., Mumenthaler, M., Rosen, A., & Yesavage, J. (2005). Cognitive ability, expertise, and age differences in

following air-traffic control instructions. Psychology and Aging, 20(1), 117-133.

[24] Royall, D. R., Lauterbach, E. C., Cummings, J. L., Reeve, A., Rummans, T. A., Kaufer, D. I., et al. (2002). Executive Control Function

A Review of Its Promise and Challenges for Clinical Research. A Report From the Committee on Research of the American

Neuropsychiatric Association (Vol. 14, pp. 377-405): Am Neuropsych Assoc.

[25] Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive

functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49-100.

[26] Sanfey, A. G., Hastie, R., Colvin, M. K., & Grafman, J. (2003). Phineas gauged: decision-making and the human prefrontal cortex.

Neuropsychologia, 41(9), 1218-1229.

[27] Colom, R., Rubio, V., Shih, P., & Santacreu, J. (2006). Fluid intelligence, working memory and executive functioning. PSICOTHEMA-

OVIEDO-, 18(4), 816.

[28] Cabeza, R., Anderson, N. D., Locantore, J. K., & McIntosh, A. R. (2002). Aging gracefully: compensatory brain activity in high-

performing older adults. Neuroimage, 17(3), 1394-1402.

Page 49: Causse Thesis summary - Free

[29] Tisserand, D. J., & Jolles, J. (2003). Special issue on the involvement of prefrontal networks in cognitive ageing. Cortex, 39, 1107-

1128.

[30] De Neys, W., & Van Gelder, E. (2009). Logic and belief across the lifespan: the rise and fall of belief inhibition during syllogistic

reasoning. Developmental Science, 12(1), 123-130.

[31] West, R., & Baylis, G. C. (1998). Effects of increased response dominance and contextual disintegration on the Stroop interference

effect in older adults. Psychology and Aging, 13, 206-217.

[32] Van der Linden, M., Bredart, S., & Beerten, A. (1994). Age-related differences in updating working memory. British journal of

psychology(1953), 85(1), 145-152.

[33] Nagel, I. E., Chicherio, C., Li, S. C., von Oertzen, T., Sander, T., Villringer, A., et al. (2008). Human aging magnifies genetic effects on

executive functioning and working memory. Frontiers in Human Neuroscience, 2.

[34] Decker, S. L., Hill, S. K., & Dean, R. S. (2007). Evidence of construct similarity in executive functions and fluid reasoning abilities.

International Journal of Neuroscience, 117(6), 735-748.

[35] Roca, M., Parr, A., Thompson, R., Woolgar, A., Torralva, T., Antoun, N., et al. (2009). Executive function and fluid intelligence after

frontal lobe lesions. Brain.

[36] Sicard, B., Taillemite, J., Jouve, E., & Blin, O. (2003). Risk propensity in commercial and military pilots. Aviation, space, and

environmental medicine, 74(8), 879-881.

[37] Moeller, F., Barratt, E., Dougherty, D., Schmitz, J., & Swann, A. (2001). Psychiatric aspects of impulsivity. American Journal of

Psychiatry, 158(11), 1783.

[38] Ainslie, G. (1975). Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological bulletin, 82(4), 463-496.

[39] Martin, L., & Potts, G. (2009). Impulsivity in decision-making: An event-related potential investigation. Personality and Individual

Differences, 46(3), 303-308.

[40] Pietrzak, R., Sprague, A., & Snyder, P. (2008). Trait impulsiveness and executive function in healthy young adults. Journal of

Research in Personality, 42(5), 1347-1351.

[41] Hinson, J., Jameson, T., & Whitney, P. (2003). Impulsive decision making and working memory. Journal of Experimental Psychology-

Learning Memory and Cognition, 29(2), 298-306.

[42] Keilp, J., Sackeim, H., & Mann, J. (2005). Correlates of trait impulsiveness in performance measures and neuropsychological tests.

Psychiatry research, 135(3), 191-201.

[43] Oldfield, R. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 9(1), 97.

[44] Bayle, F., Bourdel, M., Caci, H., Gorwood, P., Chignon, J., Ades, J., et al. (2000). Factor analysis of french translation of the Barratt

impulsivity scale (BIS-10). Canadian journal of psychiatry. Revue canadienne de psychiatrie, 45(2), 156.

[45] Loubinoux, I., Tombari, D., Pariente, J., Gerdelat-Mas, A., Franceries, X., Cassol, E., et al. (2005). Modulation of behavior and cortical

motor activity in healthy subjects by a chronic administration of a serotonin enhancer. Neuroimage, 27(2), 299-313.

[46] Fitts, P. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of

Experimental Psychology, 47, 381–391.

[47] Chen, Y., Mitra, S., & Schlaghecken, F. (2008). Sub-processes of working memory in the N-back task: an investigation using ERPs.

Clinical Neurophysiology, 119(7), 1546-1559.

[48] Causse, M., Sénard, J.-M., & Démonet, J.-F., Pastor, J. (2009). Monitoring cognitive and emotional processes through pupil and cardiac

response during dynamic vs. logical task. Applied Psychophysiology and Biofeedback. DOI 10.1007/s10484-009-9115-0.

[49] Berg, E. A. (1948). A simple objective technique for measuring flexibility in thinking. The Journal of general psychology, 39, 15.

[50] Eling, P., Derckx, K., & Maes, R. (2008). On the historical and conceptual background of the Wisconsin Card Sorting Test. Brain and

Cognition.

[51] Heaton, R. K. (1981). A MANUAL for the Wisconsin Card Sorting Test. Odessa, Florida 33556: Psychological Assessment Resources,

Inc.

[52] DeSoto, M., Fabiani, M., Geary, D., & Gratton, G. (2001). When in doubt, do it both ways: Brain evidence of the simultaneous

activation of conflicting motor responses in a spatial Stroop task. Journal of Cognitive Neuroscience, 13(4), 523-536.

[53] Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65-70.

[54] Gregoire, J., & Van Der Linden, M. (1997). Effect of age on forward and backward digit spans. Aging, Neuropsychology, and

Cognition, 4(2), 140-149.

[55] Orsini, A., Chiacchio, L., Cinque, M., Cocchiaro, C., Schiappa, O., & Grossi, D. (1986). Effects of age, education and sex on two tests

of immediate memory: a study of normal subjects from 20 to 99 years of age. Perceptual and motor skills, 63(2 Pt 2), 727.

[56] Taylor, J., Kennedy, Q., Noda, A., & Yesavage, J. (2007). Pilot age and expertise predict flight simulator performance: A 3-year

longitudinal study. Neurology, 68(9), 648.

[57] Taylor, J., O'Hara, R., Mumenthaler, M., & Yesavage, J. (2000). Relationship of CogScreen-AE to flight simulator performance and

pilot age. Aviation, space, and environmental medicine, 71(4), 373.

Page 50: Causse Thesis summary - Free

[58] Verhaeghen, P. (1993). Marcoen, A, & Goossens, L. Facts an& fiction about memory aging: A quantitative integration of research

findings. Journal of Gerontology: Psychological Sciences, 48, 157-171.

[59] Yesavage, J., Dolhert, N., & Taylor, J. (1994). Flight simulator performance of younger and older aircraft pilots: effects of age and

alcohol. Journal of the American Geriatrics Society, 42(6), 577.

[60] Tucker-Drob, E. M., Johnson, K. E., & Jones, R. N. (2009). The cognitive reserve hypothesis: A longitudinal examination of age-

associated declines in reasoning and processing speed. Developmental psychology, 45(2), 431.

[61] Birren, J. E., & Botwinick, J. (1955). Speed of response as a function of perceptual difficulty and age. Journal of Gerontology, 10(4),

433.

[62] Houx, P. J., Jolles, J., & Vreeling, F. W. (1993). Stroop interference: aging effects assessed with the Stroop Color-Word Test.

Experimental Aging Research, 19(3), 209-224.

[63] Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition. Psychological review, 103(3), 403-427.

[64] Salthouse, T. A. (1991). A theory of cognitive aging: Elsevier.

[65] Bruyer, R., Van der Linden, M., Rectem, D., & Galvez, C. (1995). Effects of age and education on the Stroop interference. Archives de

psychologie(Genève), 63(247), 257-267.

[66] West, R., & Baylis, G. C. (1998). Effects of increased response dominance and contextual disintegration on the Stroop interference

effect in older adults. Psychology and Aging, 13, 206-217.

[67] De Neys, W., & Van Gelder, E. (2009). Logic and belief across the lifespan: the rise and fall of belief inhibition during syllogistic

reasoning. Developmental Science, 12(1), 123-130.

[68] Hurter, C., Tissoires, B., & Conversy, S. (2009). FromDaDy: spreading data across views to support iterative exploration of aircraft

trajectories. IEEE Transactions on Visualization and Computer Graphics.

[69] Schroeder, D., Harris, H., & Broach, D. (1999). Pilot age and performance: An annotated bibliography (1990-1999).

[70] Wiggins, M., & O’Hare, D. (1995). Expertise in aeronautical weather-related decision making: A cross-sectional analysis of general

aviation pilots. Journal of Experimental Psychology: Applied, 1(4), 305-320.

[71] Goh, J., & Wiegmann, D. (2001). Visual flight rules flight into instrument meteorological conditions: An empirical investigation of the

possible causes. The International Journal of Aviation Psychology, 11(4), 359-379.

[72] Goh, J., & Wiegmann, D. (2002). Human factors analysis of accidents involving visual flight rules flight into adverse weather.

Aviation, space, and environmental medicine, 73(8), 817.

[73] Muthard, E., & Wickens, C. (2003). Factors that mediate flight plan monitoring and errors in plan revision: Planning under automated

and high workload conditions.

[74] Orasanu, J., Ames, N., Martin, L., & Davison, J. (2001). Factors in Aviation Accidents: Decision Errors. Linking expertise and

naturalistic decision making, 209.

[75] Rhoda, D., & Pawlak, M. (1999). An assessment of thunderstorm penetrations and deviations by commercial aircraft in the terminal

area. Massachusetts Institute of Technology, Lincoln Laboratory, Project Report NASA/A-2, 3.

[76] BEA. (2000). Objectif : destination. Technical report, Bureau Enquête Analyse.

[77] Morrow, D., Menard, W., Stine-Morrow, E., Teller, T., & Bryant, D. (2001). The influence of expertise and task factors on age

differences in pilot communication. Psychology and Aging, 16(1), 31-46.

[78] Morrow, D., Ridolfo, H., Menard, W., Sanborn, A., Stine-Morrow, E., Magnor, C., et al. (2003). Environmental support promotes

expertise-based mitigation of age differences on pilot communication tasks. Psychology and Aging, 18(2), 268-284.

[79] Taylor, J., O’Hara, R., Mumenthaler, M., Rosen, A., & Yesavage, J. (2005). Cognitive ability, expertise, and age differences in

following air-traffic control instructions. Psychology and Aging, 20(1), 117-133.

[80] Wiegmann, D., Goh, J., & O'Hare, D. (2002). The role of situation assessment and flight experience in pilots' decisions to continue

visual flight rules flight into adverse weather. Human factors, 44(2), 189.

[81] O'Hare, D., & Smitheram, T. (1995). "Pressing on" into deteriorating conditions: An application of behavioral decision theory to pilot

decision making. The International Journal of Aviation Psychology, 5(4), 351-370.

Page 51: Causse Thesis summary - Free

Part 4

Embedded eye tracker in a real aircraft: new perspectives on pilot/aircraft interaction monitoring

Presented at ICRAT 08

Frédéric Dehais Centre Aéronautique et Spatial ISAE-SUPAERO, University of Toulouse

Toulouse, France [email protected]

Mickaël Causse Centre Aéronautique et Spatial ISAE-SUPAERO; INSERM, U825, University of Toulouse. Toulouse, F-31000 France

[email protected]

Josette Pastor INSERM, U825, Toulouse, F-31000 France;

University Paul Sabatier, University of Toulouse, Toulouse, F-31000 France. [email protected]

Abstract—Currently, online assessment of the aircrew performance focuses on behavioural data (flight

data and pilot’s actions) and the detection may intervene too late for coping with the situation

degradation. An early assessment of the pilot’s “internal state”, based on physiological data collected from

his autonomous nervous system (ANS) and predictive of his behaviour, is necessary. These data give clues

both on the cognitive activity and on the emotional states and stress. The integration of ANS devices in a

cockpit presents practical drawbacks and their use is often limited to simulators. In this preliminay study,

the pros and cons of the adaptation of a standalone eye tracker in a light aircraft are presented. In spite of

a sensitivity to light conditions and a definition of areas of interest limited to a part of the cockpit, the eye

tracker has provided interesting behavioural (fixations) and physiological (pupillometry) measures in

nominal (from take-off to landing) and degraded (provoke a simulated engine failure and plane down

toward the airfield) conditions. The pilots spent less time glancing at the instruments, and focused on less

instruments in the degraded condition. Moreover, the pupil size varied with the flight phases in the

degraded condition, which reflected the variations of stress and attention levels. These encouraging results

open two tracks: the development of new eye trackers able to overcome current technical limitations, and

neuroergonomics researches providing guidelines for new man-machine interfaces integrating both flight

and crew state vectors.

Keywords: Neuroergonomics, Eye Tracker, pilot activity, pupillometry, human factors

Page 52: Causse Thesis summary - Free

I. INTRODUCTION

As long as the human operator (i. e. the pilot) is a key agent in charge of the flight, the definition of

metrics able to predict his performance is a great challenge. Currently, online assessment of the

aircrew performance focuses on behavioural data measured from the pilot/aircraft interactions. In

particular, formal methods are developed to detect human errors thanks to aircraft/pilot behaviour

monitoring [1, 2]. A predictive approach based on particle Petri nets [3] is also proposed to anticipate

possible pilot-systems conflicts [4] that are accurate precursors of the pilot’s loss of situation

awareness [5]. However, all these methods, which rely on the operator’s actions, may intervene too

late for coping with the situation degradation. An early assessment of the pilot’s “internal state”,

predictive of his behaviour, should tackle the problem. Physiological data collected from the pilot’s

autonomous nervous system (ANS) are good candidates since they give clues both on the cognitive

activity and on emotional states and stress [6] [7]. Arousal, vigilance, emotional states, attentional

demand can be derived from heart rate and blood pressure, theta and alpha brain waves,

temperature variations, respiration, skin conductance and oculometry [8] [9].

All ANS devices though present practical drawbacks. They are sensitive to the operator’s physical

state (e.g. sweating perturbs skin conductance, fever changes temperature and heart responses ...) or

to the environment (e.g. magnetic and electric fields may create artefacts on electroencephalograph

responses), and they are too cumbrous to be easily adapted to a cockpit. For example, oculometry

suffers the following limitations:

• As eye fixations “fill up” the total time, not all fixations are relevant to assess the pilot’s visual

demand. Moreover, a fixation does not necessarily imply perception [10];

• As the pupil diameter varies in function of light intensity to maximise visual capacity,

pupillometry cannot be used to assess the level of stress of the pilot under changing light

environments ;

• Electro-oculograms and most eye trackers are cumbersome devices and may disrupt pilots’

activity since pilots have to wear an electrode close to the eye or special equipment like

helmets.

These considerations tend to restrict the utilisation of ANS devices to controlled studies in

laboratory (i.e. flight simulator) [11] [12], although they have already been used in real flight

conditions [13]. Eye tracking offers a fruitful perspective since visual perception is a key for pilot to

control the flight and oculometry may provide both behavioural and cognitive/emotional physiological

measures [7] to assess the pilot’s performance:

Page 53: Causse Thesis summary - Free

• The visual-search strategy, or the selective attention to relevant visual stimuli is an index of

information needs [14];

• The eye-scanning patterns of pilots in terms of frequency of fixations seem to be related to the

instruments’ importance. The length of fixations, however, is related to the difficulty in

obtaining/interpreting information from instruments [15];

• An increase in workload is accompanied by increased fixation times [16] [17];

• The decrease of the duration and the number of eye blinks are strongly correlated to visual

demand [18] [9];

• Low frequency “pupillary oscillations” are linked to fatigue [19];

• The pupillary response is related to mental workload [20] [21] [22];

• In many cognitive processes such as language processing, perception, memory, complex

reasoning and attention, the pupil diameter grows with the difficulty of the task [23] [24];

• Pupillary responses also provide clues on the emotional state [25] [26] and pupil size may vary

on a continuum according to emotional valence [27] or reflect the emotional activation or

arousal [28].

Our long-term goal is to develop an onboard system able to predict the pilot’s performance

through the analysis of the aircraft state vector, the pilot-aircraft interactions state vector and the

pilot’s physiological state vector. However, the integration of this latter state vector implies to assess

the feasibility and the acceptability of a non intrusive physiological device. Thus, this preliminary study

proposes to assess the usability of an on-board eye tracker in real flights and aims at assessing the

benefits of this tool for human factor concerns.

II. METHODS

A. Participants

A permit to fly was given by the European Aviation Safety Agency (number 856/2007 – EASA

PTF.A07.0232) to conduct the experimentation with the restriction that ISAE flight instructors only

were authorised to fly the airplane with the on-board eye tracker. Six ISAE flight instructors, all males,

could participate to the experiment. Their mean age was 43 years (range, 35-58). Their mean flying

experience was 5896 hours (range, 1480-13000). The six participants were qualified to fly the Aquila

AT01 aircraft (two-seated light airplane, 100 horsepower).

Page 54: Causse Thesis summary - Free

B. Scenario

The flight scenario starts at nightfall at Lasbordes airfield and ends before the beginning of the

aeronautical night3. It is divided into two consecutive sequences (cf. fig 1):

• The first sequence consists in a classical visual traffic pattern : take off (1) – cross wind leg (2) –

down wind leg (3) - base leg (4) – last turn (5) – final leg (6) - touch and go (7) . This sequence is

the nominal condition;

• The second sequence starts after the previous touch and go (7) and consists in flying back

toward Lasbordes airfield at an altitude of 2500 feet. Once over the airfield, the pilot had to

chop the throttle so as to perform an engine failure exercise (8) and then to plane down

toward the airfield (9-12). This sequence is the degraded condition.

This scenario is presented to the pilot one hour before the beginning of the experimentation.

During the briefing, it is clearly exposed to each pilot that he decides the moment of the engine failure

exercise and that he may use the throttle at any moment if flight safety is altered, see Figure 28.

Figure 28: The two flight sequences: the nominal and the degraded one. The nominal sequence ends after the landing (6), the degraded sequence starts just after, with the take off (7)

C. Oculometry

A non intrusive eye tracker Tobii x50 was used for the purpose of the experimentation. This device

has 0.5 degree of accuracy and a 50 HZ sampling rates. It also provides instant re-acquisition after

3 The aeronautical night begins in France thirty minutes after sunset.

Page 55: Causse Thesis summary - Free

extreme head motion. It had to be adapted to be easily set in the Aquila aircraft without any

modification of the airplane and without provoking any disturbance for the pilot (e.g. no visual

scanning disturbance).

The eye tracker was placed below the left part of the instrument panel in front of the pilot’s seat

(cf. Figure 29). A scene camera was centrally placed under the fix part of the canopy. Data

synchronization and processing was done via an analog/numerical video converter, a Tobii external

card and a Sony Vaio laptop. These three light devices were situated in the luggage compartment.

Figure 29: The eye tracker (ET) was fixed under the instrument panel and the data processing system was placed in the luggage compartment. This latter processed and surimposed in real time both the eye tracker data and the video data coming from the scene camera.

The technical characteristics of the x50 eye tracker and its particular location allowed to track the

pilot’s eye gaze on the left part of the instrument panel where are the primary flight beacons (i.e.

airspeed, altimeter, horizon…) As shown in the Figure 30 and in the Figure 31, it is not possible to

determine the pilot’s eye gaze out of this area (i.e. the eye gaze out of the cockpit, the eye gaze on the

beacons situated on the right part).

Page 56: Causse Thesis summary - Free

Figure 30: The blue dot represents the gaze fixation. Note that the pilot is focusing on the airspeed instrument to check the rotation speed (Vr), just before taking-off

1) Area of interests A dedicated analysis software provides in real time data such as the timestamps, the (x,y)

coordinates of the pilot’s eye gaze on the left instrument panel and the pupil diameter. Moreover it is

possible to determine the number and the duration of fixations in specific “areas of interests”. In this

sense and in order to study the pilot’s ocular behaviour, fourteen areas of interest corresponding to

the fourteen beacons of the left instrument panels have been respectively defined: (1) outside

temperature, (2) compass, (3) manifold pressure, (4) alarm, (5) airspeed, (6) horizon, (7) altimeter,

(8) tachymeter, (9) bank and turn indicator, (10) directional gyroscope, (11) vertical speed, (12) VOR,

(13) switches (14) flaps (cf. fig 4).

Figure 31: The fourteen rectangles define the different area of interests.

Page 57: Causse Thesis summary - Free

2) Pupil diameter variations As the eye gaze, the pupil size is recorded continuously. In practice, establishing mean physiological

values for a group of subjects for an entire task is meaningless because of inter-individual variability.

We use delta values (differences between the mean pupil diameter during the concerned flight

sequence and the one calculated on the whole experiment) for measuring the pupil variations.

Luminosity measurements were performed. Indeed the pupil regulates the amount of light that

enters the eye, and thus, its size variations are highly sensitive to the luminosity changes. Thanks to a

lux-meter the ambient luminosity was continuously recorded in order to identify the flight sequences

where the light remains reasonably constant. In accordance with Gupta’s work [30] on pupil size

variations in function of the ambient luminosity, we have limited the pupil variations analysis to

sequences where the visible light was inferior to 25 lux.

III. RESULTS

As the sample size was small and the data did not follow normal distributions, nonparametric

statistical methods for dependent sample were used. Overall analyses were performed with the

Friedman Anova. Wilcoxon signed rank test was used for paired-samples tests. The analyses were

performed with Statistica 7.1 (© StatSoft).

A. Behavioural results

The different fixation times, expressed in percentage of the total time spent on the 14 defined

instruments during the nominal landing sequence vs. the landing sequence with the simulated failure

(the degraded landing sequence), are presented in Figure 32.

Page 58: Causse Thesis summary - Free

Figure 32: Mean fixation durations in percentages on the areas of interest of the six pilots during the nominal and the degraded landing, from base leg until the flare.

The results showed a reduction of the number of instruments gazed during the degraded sequence

in comparison to the nominal one. During the nominal sequence, all the instruments were looked

(except for the alarm panel) whereas only 10 instruments were looked during the degraded sequence.

More precisely, compass, switches and tachymeter weren’t gazed. Moreover, there was an increase of

Page 59: Causse Thesis summary - Free

the relative fixation time on the airspeed during the degraded sequence regarding to the nominal

(77.49% vs. 58.12%).

Finally, the time spent on instruments appeared to be lower during the degraded sequence (Figure

33), showing that pilots focused more on external information.

Figure 33: Mean fixation time on the airspeed instrument and all other instruments during the nominal landing and the degraded landing (time in sec)

Below is presented the official “cruise checklist” of the Aquila AT01 (TABLE VIII. ) and the fixation

times in percentages (Figure 34) obtained during the cruise check list (generally performed at an

altitude of 2000 feet).

TABLE VIII. TABLE 1: AQUILA ATO1 OFFICIAL “CRUISE CHECKLIST” [31]

Trim setChronometer Top and estimatedAltimeter setDirectional CheckedGPS Use & Stby stateEngine instruments CheckedManifold pressure 25 inchesTachymeter 2000 RPM

Page 60: Causse Thesis summary - Free

Figure 34: Mean fixation times in percentages on the 14 zones of interest during the “cruise checklist”. Note that “airspeed” stands for “airspeed indicator”, “directional” stands for directional gyroscope, “horizon” stands for gyro horizon and “bank and turn” stands for “bank and turn indicator” (mean total duration = 7.27 sec)

The analysis shows that the tachymeter is the most looked instrument (46.71%), and then comes

the manifold pressure (29.78%) and the airspeed (14.12%).

B. Pupillary response

In spite of the fact that all the experiments were conducted at nightfall, the luminosity variation

did not allow to analyse pupil diameter variations during all the sequences (TABLE IX. )

Considering this pitfall, only the degraded sequences of four pilots (pilot ID: 1 to 4) were included

in the pupil diameter variation analysis. Indeed, mean luminosity variations during the considered

sequences were only of 5.75 lux for the four pilots.

Page 61: Causse Thesis summary - Free

TABLE IX. LUMINOSITY MEASUREMENTS FOR EACH PILOT AT THE BEGINNING AND THE END OF THE TWO FLIGHT SEQUENCES (E.G. PILOT ID 1 STARTED THE FIRST FLIGHT SEQUENCE WITH 127 LUX AND ENDED IT WITH 40 LUX ; HE THEN STARTED THE SECOND SEQUENCE WITH 20

LUX AND ENDED IT WITH 7 LUX).

The Friedman’s ANOVA showed a strong significant difference (p=0.001) concerning the delta

pupillary diameter among the four flight phases (Figure 35). However, Wilcoxon post hoc paired-

samples analysis didn’t show any difference.

Figure 35: Pupillary diameter changes (in mm) regarding the four flight phases during the second flight sequences, respectively one minute before the failure (1), the failure and the crosswind (2), the base leg (3), from the last turn to the final touch (4)

IV. DISCUSSION

The introduction of a new onboard device for human factors purposes must fulfil three

requirements: 1) the device does not disturb the pilot’s activity, 2) the device is able to work correctly

Luminosity variations (in lux)

Pilot ID first sequence second sequence

Subject 1 127-40 20-7

Subject 2 82-36 8-4

Subject 3 90-35 10-6

Subject 4 93-33 10-12

Subject 5 473-230 220-91

Subject 6 610-600 520-380

Page 62: Causse Thesis summary - Free

in real flight conditions, 3) the device improves significantly the human-machine interface, and

therefore the flight safety. The preliminary neuroergonomics work that is presented here gives some

clues about the capabilities of an eye tracker as a device onboard a small aircraft.

Observation of the six pilots showed that no perturbation was generated by the eye tracker that

remained totally unnoticed after the preliminary set up phase. However, this must be confirmed on

non expert pilots and on other types of aircrafts.

Concerning the usability of the eye tracker in real flight conditions, the results are more equivocal.

Because of light issues, we weren’t able to compare pupil dilation during the nominal sequence vs. the

degraded one in all pilots. Moreover, AOIs can be only defined at places that are constrained by the

eye tracker’s location in the plane. This can be overcome by helmet eye trackers, however with an

intrusiveness of the device in the pilots’ activity. Technological progress must thus be accomplished to

allow the generalisation of onboard eye tracking experiments.

In spite of these drawbacks, the preliminary results highlight the possibility of deriving interesting

measures of the pilot’s activity from eye tracker data.

Analysis of the areas of interest captured by the eye tracker is a way to assess its accuracy in real

conditions. In particular, analysis of critical events such as the in-flight checklist sequences is a key to

evaluate the reliability of this tool:

• The visual scanning is codified by an official procedure in the flight manual than can be used as a

model of reference;

• These sequences are very short (less than ten seconds) and flight parameters to be checked by

the pilot are vital.

Such constraints lead the pilots to relevant eye fixations during these periods and allow assuming

that the areas of interest observed are also a priori the result of a real voluntary attentional activity.

In this perspective, the analysis of the areas of interest during the “cruise checklist” (cf. Fig 5)

shows that the visual scanning of the six pilots is limited to eight flight instruments. More precisely the

pilots have focused on the tachymeter, the manifold pressure, the airspeed indicator, the altimeter,

the directional gyroscope, the compass, the bank and turn indicator and the gyro horizon. These areas

of interest are consistent with the ones defined in the official Aquila “cruise checklist”: the tachymeter

and the manifold pressure have to be set to particular values and the results of these adjustments

have to be implicitly checked on the airspeed indicator, the altimeter has to be checked as the “cruise

Page 63: Causse Thesis summary - Free

checklist” starts at an altitude of 2000 feet, and the value of the directional gyroscope has to be

checked, which is done thanks to a quick comparison with the value of the compass.

Though it is not explicitly expressed in this checklist, it is totally consistent that flight indicators

such as gyro horizon and turn and bank indicator are supervised by the pilots in order to be stabilised

perfectly on the three axes (roll, pitch and yaw) to perform an optimal checklist. One may notice that

some actions of the checklist are not detected by the eye tracker but:

• The chronometer and GPS settings were not performed as the pilots were not engaged in a

complex navigation task but stayed close to the airfield;

• The trim or the engine instruments (e.g. oil pressure indicator) are located on the right part of

the instruments panel where no eye tracking could be established due to the limitation in angle

of the Tobii x50.

The analysis of the fixation times on the areas of interest during this checklist showed that the

pilots focused particularly on the tachymeter (46.71 % fixation time on this instrument during that

checklist), the manifold pressure (29.78 %) and the airspeed indicator (14.12 %). Interviews with the

six pilots have confirmed that the engine management during this checklist requires a certain amount

of attentional demand: very accurate and careful adjustments were needed on the tachymeter,

manifold pressure and the speed. The pilots spent less time on checking instruments as the altimeter

(4.48%), the directional gyroscope (1.82 %) or the compass (1.32 %). Indeed, discussion with the pilots

revealed that the altimeter was rapidly looked to check that a 2000 feet altitude was reached (i.e. to

start the cruise checklist). They also just glanced at the directional gyro and the compass: as the pilots

were not about to perform a navigation task, the cross-checking of these two indicators were of less

importance. In this sense, these findings are consistent with research conducted on the correlation

between attentional demand and time fixations [15] [16] [17]: important information implies longer

time fixations.

The analysis and the comparison of the pilots’ areas of interest during the landing in nominal and

degraded conditions (cf. fig 6 and 7) have revealed different ocular patterns. First of all, the total

duration of fixations in nominal conditions was more than two time higher to the total duration of

fixations in degraded conditions (13.59 seconds vs. 5.53 sec). This suggests that in degraded

conditions, the pilots spent more time looking outside the cockpit to assess and adapt their trajectory

in reference with the airfield. Another major difference between the two landings relied on the fact

that the pilot’s areas of interests were less distributed in the degraded conditions than in the

Page 64: Causse Thesis summary - Free

nominal one with a particular focus on the airspeed indicator (77.49% of total fixation times in

degraded landing vs. 58.12% in nominal landing). A first consideration is to take into account the fact

that in the degraded condition, the pilots had no more interest to supervise the tachymeter and the

manifold pressure due to the engine failure. Another consideration is given by the pilots who all

agreed that in the degraded condition they had faced troubles to manage their speed as they were

surprised by the high lift-to-drag ratio of the Aquila. In this sense, this led them to focus especially on

the airspeed indicator and particularly to take a key decision: maintaining the landing or going

around.

The analysis of pupil diameter variations shows some evolutions according to the different flight

sequences. More precisely, mean delta pupil diameter was of -0.25 mm before the simulated failure,

+0.47 mm during the failure and the cross wind leg, -0.10 during the base leg and +0.12 from the last

turn to the final touch. These results are consistent with the pilots’ interviews that report a high

anxiety and cognitive demand due to the management of the aircraft during the few early minutes of

the simulated failure, a lower anxiety during the base leg because of the successful stabilization of the

aircraft, and finally another increase of anxiety and cognitive demand during the landing sequence

because of the required precision and the potential go-around in case of unsafe approach.

Furthermore, the literature classically reports a high cognitive demand and a high ANS arousal during

the landing [32], which is consistent with the increase of pupil diameter observed during the last turn

and the final touch.

During the flying activity, pilots are confronted with numerous stressors that can deplete their

performance, such as time pressure, increased anxiety, and unexpected failure. Whereas a growing

literature [33] [34] sheds light on the effect of complex flight scenarios or anxiety on pilots’

performance and physiological parameters, real flight experiments remain extremely rare. Therefore,

on-board eye tracking offers promising perspectives in term of real condition monitoring of both

pilot’s actions and physiological states, although this ecological approach shows some technical

limitations. Firstly, the analysis of the areas of interest shows the reliability of the tool and its

capability to predict behaviours. Indeed, during a takeoff, it has been possible to link the absence of

visual scanning on the flaps with an omission of a required action on them later. Moreover, the AOI

analysis allows bringing to light differential visual behaviours according to the landing condition

(nominal or degraded). Secondly, the measurement of the pupil dilation gives clues on the pilot’s

emotional state and/or cognitive workload. The pupil response seems to evolve differently during

the four flight phases. The pupil diameter appears to be higher just after the simulated engine

Page 65: Causse Thesis summary - Free

failure. This observation is coherent with the increase of mental demand and/or anxiety during this

particularly critical flight phases. In addition, the occurrence of differential patterns during the failure

phases vs. the nominal ones seems to emphasize pupil diameter results. Further work should be

conducted by night to totally get rid of the luminosity variations and to attempt to produce more

results, in particular concerning the comparison of degraded and nominal conditions.

ACKNOWLEDGMENT

The authors wish to thank all the pilots from ISAE and Fabrice Bazelot, Frank Yvars and Thierry

Louvet for the time they spent on setting up the eye tracker onboard the Aquila, Eric Absil for his great

job on the eye tracker, and Christian Colongo for his support. The eye tracker Tobii x50 has been

kindly lent to ISAE by the firm “Pilot Vision” (Alexander Seger) during the experimentation.

Page 66: Causse Thesis summary - Free

REFERENCES [1] M. Heymann, A. Degani, On formal abstraction and verification of human-machine interfaces: the discrete event case. NASA

Technical Memorandum, 2001. [2] T. Callantine, The crew activity tracking system: Leveraging flight data for aiding, training, and analysis. Proceedings of the 20th

Digital Avionics Systems Conference, Daytona Beach, FL, October, 5.C.3-1--5.C.3-12 (CD- ROM), 2001. [3] C. Lesire, C. Tessier, Particle Petri nets for aircraft procedure monitoring under uncertainty. 26th International Conference on

Application and Theory of Petri Nets and Other Models of Concurrency (ATPN), Miami, Florida, USA, 2005. [4] F. Dehais, C. Lesire, A. Goudou, C. Tessier, Toward an anticipating agent to help pilots AAAI Fall Symposium "From reactive to

anticipatory cognitive embodied systems", Arlington, VA, USA, 2005. [5] F. Dehais, C. Tessier, L. Chaudron, GHOST: experimenting conflicts countermeasures in the pilot's activity. IJCAI-03, Proccedings of

the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, 2003. [6] HD. Critchley, Electrodermal responses: what happens in the brain? Neuroscientist, 8(2):132-42, 2002. [7] E. Granholm, SR. Steinhauer, Pupillometric measures of cognitive and emotional processes. International Journal of Psychophysiology,

52(1):1-6, 2004. [8] M. J. Skinner, P. A. Simpson, “Workload issues in military tactical aircraft,” Int. J. Aviat. Psychol., vol. 12, no. 1, pp. 79–93, 2002. [9] J.A. Veltman, A.W.K. Gaillard, (1996) Physiological indices of workload in a simulated flight task, Biological Psychology ,42, 323-42,

Elsevier, 2002. [10] D. De Waard, The measurement of drivers' mental workload. PhD thesis, University of Groningen. Haren, The Netherlands: University

of Groningen, Traffic Research Centre, 1996. [11] R.J. Mumaw, N.B. Sarter, C.D. Wickens, Analysis of pilots’ monitoring and performance on an automated flight deck. Proceedings of

the 11th biennial meeting of the International Symposium on Aviation Psychology, Columbus, OH, 2001. [12] P. Kasarskis, J. Stehwien, J. Hichox, A. Aretz, C. Wickens, "Comparison of expert and novice scan behaviors during VFR flight," 11th

International Symposium on Aviation Psychology, 2001. [13] J.A. Veltman, A Comparative Study of Psychophysiological Reactions During Simulator and Real Flight. International Journal of

Aviation Psychology,Volume 12, 2002. [14] P.K. Hughes, B.L. Cole, The effect of attentional demand on eye movement behaviour when driving. In A.G. Gale, M.H. Freeman,

C.M. Haslegrave, P. Smith & S.P. Taylor (Eds.), Vision in vehicles-II (pp. 221-230). Amsterdam: North-Holland, 1988. [15] G.F. Wilson, F.T. Eggemeier, Psychophysiological assessment of workload in multi-task environments. In D.L. Damos (Ed.), Multiple-

task performance (pp. 329-360). London: Taylor & Francis, 1991. [16] R.D. O'Donnell, F.T. Eggemeier, Workload assessment methodology. In K.R. Boff, L. Kaufman & J.P. Thomas (Eds.), Handbook of

perception and human performance. Volume II, cognitive processes and performance. (pp 42/1-42/49). New York: Wiley, 1986. [17] R.W Backs, L.C. Walrath, Eye movement and pupillary response indices of mental workload during visual search of symbolic displays.

Applied Ergonomics, 23, 243-254, 1992. [18] L.O. Bauer, R. Goldstein, J.A Stern, Effects of information-processing demands on physiological response patterns. Human Factors,

29, 213-234, 1987. [19] B. Wilhelm, H. Wilhelm, Das Pupilverhalten verrät Übermüdung. Zeitschrift für Verkehrssicherheit, 41, 116-118, 1995. [20] T.I. Shamsi, X. Sam Zheng, B.P. Bailey, Task-evoked pupillary response to mental workload in human-computer interaction. CHI

2004, Vienna, Australia, 2004. [21] L.T.M. Hoeks, The pupillary response as a measure of mental processing load: with application to picture naming. PhD Thesis,

Nijmegen, The Netherlands: University of Nijmegen, 1995. [22] J. Hyönä, J. Tommola, A.-M Alaja, Pupil dilation as a measure of processing load in simultaneous interpretation and other language

tasks. The Quarterly Journal of Experimental Psychology, 48A, 598-612, 1995. [23] J.L. Andreassi, Pupillary response and behaviour. In: Mahwah, N.L.E.A.(Ed.), Psychophysiology. Human behav. Physiol.response,Vol.

10. Oxford University Press, New York, pp. 218–233, 2000. [24] J.R. Jennings, M.W. Van der Molen, Preparation for speeded action as a psychophysiological concept. Psychol. Bull. 131 (3), 434–459,

2005. [25] J. Hytintk, J. Tommola, A-M. Alaja, “Pupil Dilation as a Measure of Processing Load in Simultaneous Interpretation and Other

Language Tasks”. The Quarterly Journal of Experimental Psychology, 48A (3): 598-612, 1995. [26] M. Causse, B. Pavard, J-M. Sénard, J-F. Démonet, J. Pastor, Emotion Induction through Virtual Avatars and its Impact on Reasoning:

Evidence from Autonomous Nervous System Measurements and Cognitive Assessment. Actes de la Conférence VRIC’06, Laval, France, 18-20 Avril, 2007.

[27] E. H. Hess, “Pupillometrics”. In N. S. Greenfield and R. A. Stembach (Eds.) Handbook of Psychophysiology, pages 491- 531. Holt, Richard & Winston, New York, NY, 1972.

[28] MP. Janisse, “Pupil Size, Affect and Exposure Frequency”. 128 Social Behavior and Personality, 2 (2): 125-146, 1974. [29] R. Parasuraman, M. Rizzo, Neuroergonomics: The Brain At Work, Oxford University Press, 2007. [30] A. Gupta, J. Schwiegerling, J. Straub, Design and use of an infrared Pupilometer for real-time pupil mapping in response to incremental

illumination level. Optical society of America, 2000. [31] A. Schimederer, Aquila AT01 Flight Manual, Technical report AFM-AT01-1010-100-E, AQUILA GmbH, 2005. [32] Y-H, Lee, B-S. Liu, Inflight workload assessment: comparison of subjective and physiological measurements. Aviat Space Environ

Med; 74:1078–84, 2003. [33] C. Dussault, J-C. Jouanin, M. Philippe, C-Y. Guezennec, EEG and ECG changes during simulator operation reflect mental workload

and vigilance. Aviat Space Environ Med; 76:344 –51, 2005. [34] T. Macuda, T. Poolman, P. Keller, Workload Assessment in Flight using Dense Array Eeg Schnell. 25th Digital Avionics Systems

Conference, IEEE/AIAA Volume , Issue , Oct. 2006 Page(s):1 – 11, 2006.