amygdala damage impairs emotion recognition from scenes only when they contain facial expressions

Neuropsychologia 41 (2003) 1281–1289

Rapid publication

Amygdala damage impairs emotion recognition from scenesonly when they contain facial expressions

Ralph Adolphs∗, Daniel TranelDepartment of Neurology, Division of Cognitive Neuroscience, University Hospitals and Clinics, University of Iowa,

200 Hawkins Drive, Iowa City, IA 52242, USA

Received 22 August 2002; received in revised form 23 January 2003; accepted 6 February 2003

Abstract

Bilateral damage to the human amygdala impairs recognition of negatively valenced emotions from facial expressions, but it is unclearif this finding generalizes to richer visual stimuli that contain cues in addition to faces. We investigated this issue in 4 subjects with bilateralamygdala damage, 23 with unilateral amygdala damage, 22 brain-damaged controls and 16 normal individuals. Subjects were shown twoblocks of complex social scenes; all stimuli in the two blocks were identical, except that the first block had all facial expressions in theimage erased. While control subjects were more accurate in recognizing emotions when facial expressions were present, subjects withbilateral amygdala damage did not show the same benefit for negative emotions, often performing equivalently across the two conditions.Most striking, subjects with bilateral amygdala damage were more accurate in recognizing scenes showing anger with faces erased thanwith faces present, an effect resulting in part from highly abnormal recognition of certain angry facial expressions. All four subjects withbilateral amygdala damage were impaired in recognizing angry faces shown in isolation, and frequently mistook expressions of anger forsmiles, a mistake never made by any control subject. Bilateral amygdala damage thus disproportionately impairs recognition of certainemotions from complex visual stimuli when subjects utilize information from facial expressions.© 2003 Elsevier Science Ltd. All rights reserved.

Keywords: Amygdala; Facial expression; Fear; Emotion recognition

1. Introduction

Real-life social situations typically provide a rich set ofcues that permit viewers to evaluate their emotional sig-nificance. The general visual context, the relative locationsof people, their body postures, head postures, directions ofeye gaze, and facial expressions all provide such cues. Thisgeneral observation raises two related questions: what is therelative importance of these different cues in recognizingemotions? And are different cues processed by differentbrain structures? Affirmative answers to both questions aresupported in the case of faces: the human face is a partic-ularly salient emotional cue, and there appear to be regionsof the brain that are relatively specialized for processingfaces. Foremost among these brain regions are sectors of ex-trastriate visual cortex, notably in the fusiform and superiortemporal gyri (Allison, Puce, & McCarthy, 2000; Haxby,Hoffman, & Gobbini, 2000), and the amygdala (Adolphs,2002). Whereas fusiform and superior temporal cortices

∗ Corresponding author. Tel.:+1-319-353-8610; fax:+1-319-356-4505.E-mail address: [email protected] (R. Adolphs).

URL: http://www.medicine.uiowa.edu/adolphs.

may participate primarily in constructing detailed perceptualrepresentations of faces (respectively, of the static, structuralconfiguration, and of the dynamic changes among their fea-tures), the amygdala has been shown to be required in orderto link the perception of the face to the retrieval of knowledgeabout its emotional and social meaning (Adolphs, 2002).

Ever since the seminal reports of Brown, Shafer, Kluverand Bucy (Kluver & Bucy, 1939), the primate amygdala hasbeen implicated in the regulation of social and emotionalbehaviors. Lesions of the amygdala in monkeys impair theanimal’s ability to evaluate the social and emotional mean-ing of visual stimuli (Emery et al., 2001; Kling & Brothers,1992; Meunier, Bachevalier, Murray, Malkova, & Mishikin,1999; Rosvold, Mirsky, & Pribram, 1954; Weiskrantz, 1956;Zola-Morgan, Squire, Alvarez-Royo, & Clower, 1991).While bilateral lesions of the amygdala in humans appearto have less severe consequences than they do in monkeys,they nonetheless result in alterations in social behavior andsocial cognition (Adolphs, 1999). Particularly clear is thehuman amygdala’s role in the recognition of social cuesfrom faces, a role that is best understood in regard to therecognition of basic emotions. Both lesion and functionalimaging studies have demonstrated, respectively, impaired

0028-3932/03/$ – see front matter © 2003 Elsevier Science Ltd. All rights reserved.doi:10.1016/S0028-3932(03)00064-2

1282 R. Adolphs, D. Tranel / Neuropsychologia 41 (2003) 1281–1289

recognition of facial expressions following bilateral amyg-dala damage, and activation of the amygdala during pre-sentation of emotional facial expressions (Adolphs, Tranel,Damasio, & Damasio, 1994; Morris et al., 1996). Notably,these findings are most robust for negatively valencedemotions, in particular fear, anger, and sadness in lesionstudies, and fear in functional imaging studies (Adolphs,1999; Adolphs et al., 1999; Broks et al., 1998; Calder et al.,1996; Schmolck & Squire, 2001; Young, Hellawell, Van deWal, & Johnson, 1996), although the reasons underlyingthese differential patterns of dependency on the amygdalaare debated (seeAdolphs, 2002; Rapcsak et al., 2000). Therole of the amygdala in processing information about facesis also borne out by single-unit recordings that have foundresponses relatively selective to faces in both human (Fried,MacDonald, & Wilson, 1997) and non-human primates(Leonard, Rolls, Wilson, & Baylis, 1985).

Given the amygdala’s role in recognizing negative emo-tions from facial expressions when presented in isolation,we wondered if those findings would generalize also tomore complex visual social stimuli. Based on the data re-viewed above, we hypothesized that the amygdala would bemost important to process emotional information from faces,rather than from other visual cues, and that it would be es-pecially important in processing facial expressions of neg-atively valenced emotions. We thus predicted that damageto the amygdala would yield impaired recognition of suchemotions only when the recognition relied on informationabout facial expressions, and not when it relied on informa-tion other than facial expressions. Furthermore, we expecteda significant impairment to result from bilateral amygdaladamage, but less impairment to result from unilateral amyg-dala damage, as would be expected and as consistent withprior studies (Adolphs, Tranel, & Damasio, 2001; Adolphs,Tranel, Damasio, & Damasio, 1995; Anderson, Spencer,Fulbright, & Phelps, 2000).

To obtain a clear contrast, we investigated the recognitionof emotions from two classes of stimuli that were otherwiseidentical: complex social scenes containing faces, and thesame complex social scenes with the faces digitally erased.In all cases, scenes contained multiple visual cues, includingbody posture, hand gestures, interpersonal stances, and gen-eral context, in addition to emotional facial expressions. Bydirectly comparing performances to the scenes when theycontained facial expressions, versus when the faces had beenerased, we were able to assess the extent to which subjectswere able accurately to utilize information from the face inrecognizing the emotion signaled by the scene.

2. Methods

2.1. Participants

We compared performances given by 4 individuals withbilateral amygdala damage with 2 other brain-damaged

Table 1Background neuropsychology and demographics of participants

Gender Age Education VIQ PIQ Faces

SM F 32 12 86 95 49RH M 44 16 110 116 45SZ M 45 16 95 78 43JM M 68 12 90 95 41

L 4F/6M 37 14 101 108 44R 5F/8M 34 13 94 98 44

BDC 12F/20M 55 14 95 105 46

SM, RH, SZ and JM: subjects with bilateral amygdala damage; Land R: subjects with left or right unilateral temporal lobectomy; BDC:brain-damaged controls; education: years of education; VIQ and PIQ:verbal and performance IQ from the WAIS-R or WAIS-III; faces: scoresfrom the Benton face matching test (all normal). For L, R and BDCgroups, means are shown.

groups: 23 subjects with unilateral amygdala damage, and22 brain-damaged controls without amygdala damage. Per-formances were converted to accuracy scores on the basisof data from a reference group of 16 neurologically normalcontrols (see below). All brain-damaged participants wererecruited from the Division of Cognitive Neuroscience’s Pa-tient Registry, and had been extensively characterized bothin terms of the location and extent of their lesion (Damasio& Frank, 1992; Frank, Damasio, & Grabowski, 1997), andin terms of their neuropsychological performance acrossmultiple cognitive domains (Tranel, 1996). Demographicand background neuropsychological information is givenin Table 1. Normal controls were recruited through adver-tisement. All subjects had given informed consent to par-ticipate in these studies as approved by the Internal ReviewBoard of the University of Iowa, and as consistent with theDeclaration of Helsinki.

2.1.1. Bilateral amygdala damageThree of the subjects had bilateral damage to the amygdala

and to surrounding regions of the brain due to encephalitis(SZ, JM and RH); all three were densely amnesic and re-quired continual repetition of the instructions during the task.One of the subjects (SM) had bilateral damage selective tothe amygdala and a small portion of anterior entorhinal cor-tex, and was not amnesic. Background information and facerecognition abilities have been previously published on sub-jects SM, JM and RH (Adolphs, 1999; Adolphs & Tranel,2000; Adolphs et al., 1994, 1995, 1999; Tranel & Hyman,1990). Neuroanatomy of all four subjects is shown in axialMR scans inFig. 1, showing bilateral amygdala damage inall cases.

We were able to obtain replicate data sets on multipletesting sessions with three of the subjects (RH, JM and SZ),each separated by at least 1 month. As all three of thesesubjects were amnesic, none remembered participating inthe task previously, and all showed similar patterns of per-formances across the multiple sessions. In one session, we

R. Adolphs, D. Tranel / Neuropsychologia 41 (2003) 1281–1289 1283

Fig. 1. Neuroanatomy of four subjects with bilateral amygdala damage.Axial MR scans are shown for each patient, and bilateral amygdaladamage (dark regions) are evident in the medial temporal lobes, includingamygdala, in all cases. As per radiological convention, the left sides ofthe images correspond to the right side of the brain.

reversed the order of the blocks, showing scenes with facesfirst, to ensure that there were no order effects.

2.1.2. Unilateral amygdala damageWe tested subjects with left (N = 10) and right (N = 13)

unilateral temporal lobectomy. All subjects had a historyof epilepsy, and all had stereotyped neurosurgical damagethat included, but was not confined to, the amygdala on oneside. The extent of amygdala damage was variable, and wasscored on a scale of 1 (minimal) to 3 (complete) as describedpreviously (Adolphs et al., 2001).

2.1.3. Brain-damaged controlsWe tested 22 subjects who had no damage to either amyg-

dala, to orbitofrontal cortex, or to right frontal or right pari-etal cortex, as these regions of the brain have been suggestedto impair recognition of emotion in prior studies (Adolphs,Damasio, Tranel, Cooper, & Damasio, 2000; Borod et al.,1998; Hornak, Rolls, & Wade, 1996). All 22 subjects hadsingle, focal lesions resulting from stroke, and were of sim-ilar age, education, and IQ to the subjects with amygdaladamage.

2.1.4. Normal reference groupIn order to calculate accuracy scores, we used the distri-

bution of performances given by a reference group of 16neurologically normal subjects. Since performance scoreswere calculated in reference to this normal reference group,they cannot strictly be used to obtain independent accuracy

scores for that same group. Hence, wherever performancevalues are given in the figures, we omit the data from thenormal reference group, and we omit their data from all sta-tistical comparisons.

2.2. Stimuli and task

Sixteen stills from commercial black-and-white filmswere digitized and shown in two blocks: first with all facialexpressions erased but the rest of the image unaltered, sec-ond with facial expressions (Fig. 2). The stills were froma collection of high-quality photographs of films from the1950s owned by A.R. Damasio; stimuli were chosen fromthis collection on the basis of the reliability of responsesfrom normal subjects in a pilot study. All stimuli depictedcomplex scenes containing people, and showed a range ofemotions. We chose the stimuli with an emphasis on neg-ative emotions, particularly fear and anger, as this class ofemotions was of most interest in regard to their recognition

Fig. 2. Examples of stimuli similar to those used in the experiment:(top) scene with faces erased; (bottom) scene with faces present. Due tocopyright restrictions, this example is not one of the actual stimuli usedfrom the film stills, but representative.


following amygdala damage (cf.Section 1). The stimulivaried in terms of the particular combinations of visualcues that provided information about the emotion, but allincluded both facial expression and additional cues (con-text, body posture, head posture, hand and arm gestures,direction of gaze).

Subjects were asked to match each stimulus to one froma list of seven words: happy, surprised, afraid, angry, dis-gusted, sad, neutral, in terms of how they thought the peopledepicted in the stimulus might feel. There was no time limit.In each case, subjects were asked how the person(s) in thescene appeared to be feeling. In case there was any ambi-guity (for some of the fear and anger stimuli only), it wasclarified which person was meant (always the person mostdirectly facing the viewer and generally judged to occupycenter role in the scene).

We used the performances given by the normal refer-ence group to set the standard for accuracy. All data frombrain-damaged subjects were thus scored in relation to therelative frequencies of occurrence of responses given bythe normal reference group. Thus, if 100% of normal sub-jects called a scene “happy”, a subject would get a scoreof 1.0 for choosing the label “happy” and 0.0 for all otherchoices. On the other hand, if 50% of normal subjects calleda scene “surprise”, 40% called it “afraid”, and 10% calledit “sad”, a subject would receive a score for that scene of1.0 if choosing the label “surprise”, 0.8 if choosing the la-bel “afraid”, and 0.2 if choosing the label “sad”. In thisway, correctness was made a parametric function solelyof the distribution of responses that normal subjects gave:high scores correspond to relatively better performance, lowscores to relatively worse performance. Modal responses de-fined the emotion category, as follows: 5 stimuli for fear, 4for anger, 3 for sadness, 2 for happy, 1 for surprised, and 0for neutral.

It should be noted that a stimulus may be ambiguousbetween several emotions (for instance, it might be labeledas “afraid” by half of the subjects, and as “surprised” bythe other half), but given our scoring procedure it is quitepossible to detect an impaired performance for this stim-ulus (calling it either afraid or surprised gives a normalperformance, calling it anything else is impaired, in this ex-ample). In essence, our scoring procedure reflects degreesof impairment, in reference to the normal distribution ofresponses: the less frequently a certain response is given bynormal subjects, the more impaired it is scored as being.

To obtain a meaningful contrast of how performance ac-curacy might change when information about the face waserased, we calculated subjects’ responses to stimuli with thefaces erased and with the faces present on the basis of thesame distribution: the performance of the normal referencegroup for the stimuli with faces.

In addition to data from the above main experimental task,we also had available data regarding the abilities of the foursubjects with bilateral amygdala damage to recognize facialemotion from expressions presented in isolation. These data

had been published previously for SM, RH and JM (Adolphs,1999; Adolphs et al., 1994, 1999), and are presented herefor the first time for SZ. These data are given here for thepurposes of comparison with data from the above experi-mental task; we give results obtained from six angry facialexpressions that were a subset of a larger testing set. All sixangry face stimuli were from the “Pictures of facial affect”(Ekman & Friesen, 1976), and subjects had been asked torate each face with respect to the degree of anger shown, ona scale of 0–5. Detailed information regarding the face stim-uli and method have been published previously (Adolphset al., 1994, 1995, 1999).

3. Results

3.1. Matching emotions to scenes with faces

Across all stimuli, brain-damaged subjects performed,on average, quite comparably to normal controls, yieldingmean accuracy scores of 60–70% (chance mean accuracyis at 22%), and performing above chance on every stimulus(Fig. 3, left). Consistent with prior studies in subjects withunilateral amygdala damage (Adolphs et al., 2001), sub-jects with left temporal lobectomy performed slightly worsethan brain-damaged controls, and subjects with right tem-poral lobectomy performed slightly worse yet. As a group,subjects with bilateral amygdala damage performed com-parably to controls (Fig. 3 andTable 2), although we notethe high score obtained by subject SM and the low scoreobtained by subject JM; these two subjects were also theyoungest (32 years) and oldest (68 years), respectively. Theperformances in recognizing emotions from scenes includ-ing faces, across all stimuli or across only stimuli showingfear or sadness did not achieve statistically significant dif-ferences when comparing brain-damaged controls, subjectswith unilateral amygdala damage (L and R combined), andsubjects with bilateral amygdala damage (allP-values werenon-significant, Mann–WhitneyU-tests). However, subjectswith bilateral amygdala damage did have significantly loweraccuracy scores than brain-damaged controls in regard toscenes depicting anger (U = 18.5, P < 0.05, one-taileduncorrected Mann–WhitneyU-tests).

Table 2Accuracy scores for scenes with faces given by subjects with bilateralamygdala damage and brain-damaged controls (BDC)

SM RH SZ JM BDC Chance

Total 0.83 0.63 0.56 0.39 0.72 0.22Afraid 0.67 0.77 0.41 0.29 0.72 0.19Angry 0.93 0.45 0.43 0.17 0.76 0.27Sad 0.93 0.36 0.76 0.36 0.68 0.18Happy 1.00 1.00 1.00 0.70 1.00 0.14Surprised 0.46 0.64 0.67 0.64 0.60 0.25Neutral 1.00 0.74 0.19 1.00 0.33 0.37

Chance: chance performance. Means are shown for BDC.


Fig. 3. Recognizing emotions from scenes: (a) mean performances (bars indicate S.E.M.) across all 16 stimuli when shown with faces; (b) meanperformances for stimuli with faces erased. Bars (from left to right): brain-damaged controls (dark gray); left unilateral amygdala (left slantingstripes);right unilateral amygdala (right slanting stripes); subjects SM, RH, SZ and JM with bilateral amygdala damage (black).

3.2. Matching emotions to scenes without faces

Our primary interest in this study, however, was to com-pare performances given to the same stimuli when they con-tained faces, and when the faces were erased. All subjectgroups showed the same trend: accuracy in recognizing emo-tions was compromised when faces had been erased fromthe stimuli, as one would expect (Fig. 3, right). However, avisual inspection ofFig. 3 suggests that subjects with bilat-eral amygdala damage do not show the same magnitude of

Fig. 4. Recognizing anger from scenes: (a) for stimuli with faces; (b) for stimuli with faces erased. Legend as inFig. 3.

this effect as do the other subject groups. When plotting thedata only for scenes that signal anger, we found a surprisingeffect: whereas all other subject groups found these stimulimore difficult to recognize when faces had been erased, sub-jects with bilateral amygdala damage in fact showed supe-rior accuracy for the scenes in which faces had been erasedthan for those in which faces were present (Fig. 4).

To examine these findings in more detail, we calculatedfor each subject and each stimulus a difference score, theperformance when the face was present compared to the


Table 3Difference scores (accuracy with faces minus accuracy without faces) foreach emotion category given by the subject groups

BDC Left Right Bilateral

Total 0.16 0.13 0.13 0.06Afraid 0.25 0.27 0.25 0.13Angry 0.13 −0.03 −0.05 −0.22Sad 0.09 0.10 0.12 0.06Happy 0.18 0.10 0.08 0.13Surprised 0.11 0.11 0.16 0.25Neutral 0.00 0.07 0.26 0.18

BDC: brain-damaged controls; left: unilateral left amygdala damage; right:unilateral right amygdala damage; bilateral: bilateral amygdala damage(means are given).

performance when the face was erased. These differencescores are given for each emotion category inTable 3andsummarized inFig. 5. For fear, sadness, and especially anger,subjects with bilateral amygdala damage benefited less whenthe face was included than did any other subject group. Thehighly unusual pattern seen for anger was present to a lesserextent also in the subjects with unilateral amygdala damage.

For scenes showing positive emotions (happiness and sur-prise), all subject groups showed the largest proportion oftrials in which they performed more accurately when shownscenes with faces than without faces. For scenes showingnegative emotions (fear, anger, and sadness), subjects withbilateral amygdala damage showed precisely the oppositepattern, performing better without faces than with faces onthe majority of trials. This abnormal pattern was most ex-treme in the case of scenes showing anger, where it is shownalso to a lesser extent in subjects with unilateral amygdaladamage (Fig. 6).

Fig. 5. Performances differences when recognizing emotion from scenes with faces compared to scenes without faces, shown for all stimuli (left) andfor anger only (right). Legend as inFig. 3.

We examined the mean performances given to all neg-atively valenced stimuli, both with and without faces. Wefound a significant difference among brain-damaged groups(F(2, 46) = 5.1, P < 0.01) due to a significant pairwisedifference only between brain-damaged controls and sub-jects with bilateral amygdala damage (P < 0.01, post-hocScheffe test). This finding held up with a 3×3 group by neg-ative emotion ANOVA: there was a significant effect of neg-ative emotion category (F(2, 138) = 10.5, P < 0.0001), asignificant effect of subject group (F(2, 138) = 3.64, P <

0.05), and no significant interaction. Post-hoc Scheffe testsagain showed a significant pairwise difference only betweenbrain-damaged controls and subjects with bilateral amyg-dala damage (P < 0.05).

When examining the proportion of subjects who per-formed better on scenes with faces erased, while there wereno significant differences in proportions across the meanperformance for all stimuli (χ2(2) = 4.2, P > 0.1), therewere highly significant differences when examining perfor-mances for angry scenes (χ2(2) = 19.1, P < 0.0001). Thegroups with unilateral amygdala damage and those with bi-lateral amygdala damage showed a much higher proportionof subjects performing more accurately on angry scenes withfaces erased than did brain-damaged controls (Fisher’s exacttests:P = 0.0002 and 0.0023, respectively).

3.3. Recognizing anger

When we examined the responses given to individualanger scenes, we found that subjects with amygdala damageshow such an unusual pattern in large part because they givehighly incorrect scores (obtaining an accuracy score of 0)for certain scenes containing angry facial expressions, but


Fig. 6. Proportion of trials in which subjects were more accurate when shown scenes without faces than scenes with faces, for positive emotions (happyand surprise; left), negative emotions (fear, anger, and sadness; middle), and anger (right). Proportions were calculated on the basis of the total numberof trials (subjects with bilateral amygdala who participated in multiple replications of the task thus contributed multiple data points for each stimulus).We here also show data from the normal controls (white bars), since we can calculate whether or not they performed better or worse, even though wecannot strictly use their absolute accuracy scores.

provide the correct answer (obtaining an accuracy score of1) when the faces have been erased. In several cases, subjectswith bilateral amygdala damage erroneously labeled sceneswith angry faces as “happy”, and three out of the four sub-jects made this exact mistake on several occasions, whereasno subject in any of the other groups ever made this mis-take. In several cases, the particular anger stimuli that weremistaken for happy stimuli showed facial expressions withan open mouth and bared teeth.

We were able to compare the data from the present taskwith those obtained from recognition of angry facial ex-pressions shown in isolation (from the “Pictures of facialaffect”). These data, previously published for SM, JM andRH (Adolphs, 1999; Adolphs et al., 1994, 1999), and newfor SZ, show that all four subjects are impaired in their abil-ity to recognize anger from facial expressions. When askedto rate the intensity of anger shown in angry faces, andcompared to a group of 17 normal controls (Adolphs et al.,1999), the 4 subjects with bilateral amygdala damage gaveintensity ratings that were below the normal mean intensityrating. TheirZ-scores were as follows: SM, from−0.8 to−6.0 (three experiments); RH,−4.3; SZ,−3.9; JM,−5.5.

3.4. Effects of presentation order and of extent ofamygdala damage

There were no order effects in the subjects with bilat-eral amygdala damage, an issue we could investigate in thethree subjects who were so densely amnesic that they didnot remember having seen the stimuli on a prior occasion.Across three replications of the experiment, two of whichwere in the same order as given to other subjects (sceneswithout faces shown first) and one of which was in the re-

verse order (scenes with faces shown first), there were nosignificant differences in accuracy scores for scenes show-ing negative emotions (allP-values were non-significant;uncorrected Mann–WhitneyU-tests between pairs of testingsessions).

We also investigated if there might be a correlation be-tween the extent of unilateral amygdala damage in the uni-lateral amygdala subjects, and their difference score in rec-ognizing anger scenes. We found no significant correlation(Spearman’sρ = 0.25, non-significant).

4. Discussion

The present findings provide additional detail to the roleof the amygdala in recognizing emotions from visual stimulidepicting people. We can make the following conclusions:

(1) Bilateral amygdala damage impairs recognition of neg-ative emotions from facial expressions. This finding hasbeen shown in prior publications already.

(2) Bilateral amygdala damage does not in general impairrecognition of emotions from complex static visual stim-uli, provided those stimuli contain cues in addition tofacial expressions (Figs. 3 and 4). This finding is es-pecially notable in regard to fear, whose recognition isoften impaired following bilateral amygdala damage.

(3) Whereas the inclusion of facial expressions improvedrecognition of negative emotions for all other subjectgroups, subjects with bilateral amygdala damage derivedmuch less benefit from the inclusion of facial expres-sions (Fig. 5). Most surprising of all was the finding thatall four subjects with bilateral amygdala damage actu-ally performed better in recognizing anger from scenes


that had facial information erased than from scenes con-taining facial expressions, a pattern most notable forsubjects RH and SZ (Fig. 5). Two possible interpreta-tions of this pattern of impairment would be that: (a) theamygdala is relatively specialized for processing certainnegative facial expressions, and that hence damage tothe amygdala results in a disproportionate impairment inrecognizing scenes containing such facial expressions;or (b) that the amygdala might play a broader role inprocessing faces, and that certain negative facial expres-sions are more difficult to process and hence stand outfollowing amygdala damage (Rapcsak et al., 2000).

Together with prior studies demonstrating that bilateralamygdala damage results in impaired recognition of nega-tive emotions from facial expressions when shown in iso-lation, the present pattern of results argues that impairedrecognition of these emotions following bilateral amygdaladamage is specific to facial expressions, and does not extendto recognizing emotions from other social and contextualcues present in complex scenes. In fact, the data indicatethat when facial expressions of anger are present, they in-troduce cues that subjects with bilateral amygdala mistakefor incorrect emotions. Specifically, it appeared that angerexpressions showing an open mouth with bared teeth weremistaken for a smile and consequently misclassified as hap-piness. Future studies could explore this issue further byselectively manipulating the features of facial expressions.

4.1. Limitations of the study

It is worth noting that our stimuli, like most others thathave investigated related topics, are impoverished comparedto the kinds of stimuli we experience in real life. Asidefrom the presence of cues in other modalities and muchmore contextual information, real-life emotional scenes andfacial expressions are events that transpire in time, whereasour stimuli are static snapshots. It would be important infuture studies to extend the present investigations, as wellas many others, to dynamic visual stimuli. Such visual cueswould be expected to engage different processing streams inextrastriate visual cortices (dorsal rather than ventral), andmight be expected to rely less on the amygdala than staticvisual stimuli.

It is somewhat surprising that subject SM performedquite well on both tasks on average, although her perfor-mance for recognizing scenes with faces was below that ofbrain-damaged controls for fear and surprise (Table 2). Herhigh performance scores may have resulted in ceiling ef-fects that precluded a large difference score (Fig. 5). Whileher data are consistent with our interpretation above, theyalso raise the possibility that damage to medial temporallobe structures in addition to the amygdala may augmentthe pattern of impairments we report. Future studies inpatients with damage to extra-amygdalar medial temporalstructures could address this issue.

We addressed possible differences in background neu-ropsychology by providing detailed neuropsychologicalevaluation of all brain-damaged subjects (Table 1). As isevident from the table, subjects with bilateral amygdaladamage did not differ systematically on any particular mea-sure from control subjects: they did not have generallylower IQ, were not generally older, and all had normal basicvisuoperceptual function. The only notable difference wasthat three-fourths of the subjects with bilateral amygdaladamage were densely amnesic. Were any of these factorsto lead to nonspecific impairments, they would still not ac-count for the particular pattern of findings we observed: forcontrols, performances were worse when asked to recognizescenes in which the faces had been erased, as one wouldexpect, whereas subjects with bilateral amygdala damageperformed better when faces were erased than when faceswere present.

A final limitation of course concerns the relatively smallnumber of stimuli that we used in our study. This is es-pecially the case for positive emotions: since our primaryhypothesis was aimed at recognition of negative emotions,we biased our stimulus set to include more negative emo-tions than positive emotions. A more detailed investigationwith larger numbers of stimuli could further explore the pat-tern of impairments across different individual emotions. Fi-nally, the complex question concerning all the different cuespresent in visual scenes could be explored with additionalstimuli, and perhaps systematic manipulation of specific setsof cues.

4.2. Is the amygdala specialized for processing faces?

Our findings are consistent with a disproportionate role forthe amygdala in recognizing emotions from facial expres-sions, rather than from other social visual cues. Of course,there may be social visual cues whose recognition relies onthe amygdala, but that may not have been included in ourstimuli; it may also be that the amygdala does play a role inmaking social judgments about social visual cues other thansimple judgments about basic emotions. And it seems likelythat the amygdala participates in complex judgments aboutvisual stimuli that are not social (e.g.Adolphs & Tranel,1999). It would thus be premature to conclude that the humanamygdala’s role in processing visual stimuli is restricted tofacial expressions. Nonetheless, at least within the domainof recognizing negative emotions from static visual stimuli,our data suggest that the amygdala’s role is disproportion-ately specialized for recognizing such information from fa-cial expressions.

Acknowledgements

We thank Antonio Damasio for providing the originalstimuli and encouraging this study, and Jeremy Nath, MattKarafin, and Barbara Majewska for help in testing subjects.


Supported by National Institute of Neurological Diseasesand Stroke Program Project Grant P01 19632 and the JamesS. McDonnell Foundation.

References

Adolphs, R. (1999). The human amygdala and emotion.TheNeuroscientist, 5, 125–137.

Adolphs, R. (2002). Recognizing emotion from facial expressions:Psychological and neurological mechanisms.Behavioral and CognitiveNeuroscience Reviews, 1, 21–61.

Adolphs, R., Damasio, H., Tranel, D., Cooper, G., & Damasio, A. R.(2000). A role for somatosensory cortices in the visual recognitionof emotions as revealed by three-dimensional lesion mapping.TheJournal of Neuroscience, 20, 2683–2690.

Adolphs, R., & Tranel, D. (1999). Preferences for visual stimuli followingamygdala damage.Journal of Cognitive Neuroscience, 11, 610–616.

Adolphs, R., & Tranel, D. (2000). Emotion recognition and the humanamygdala. In J. P. Aggleton (Ed.),The amygdala. A functional analysis(pp. 587–630). New York: Oxford University Press.

Adolphs, R., Tranel, D., & Damasio, H. (2001). Emotion recognition fromfaces and prosody following temporal lobectomy.Neuropsychology,15, 396–404.

Adolphs, R., Tranel, D., Damasio, H., & Damasio, A. R. (1994). Impairedrecognition of emotion in facial expressions following bilateral damageto the human amygdala.Nature, 372, 669–672.

Adolphs, R., Tranel, D., Damasio, H., & Damasio, A. R. (1995). Fear andthe human amygdala.The Journal of Neuroscience, 15, 5879–5892.

Adolphs, R., Tranel, D., Hamann, S., Young, A., Calder, A., & Anderson,A. et al., (1999). Recognition of facial emotion in nine subjects withbilateral amygdala damage.Neuropsychologia, 37, 1111–1117.

Allison, T., Puce, A., & McCarthy, G. (2000). Social perception fromvisual cues: Role of the STS region.Trends in Cognitive Science, 4,267–278.

Anderson, A. K., Spencer, D. D., Fulbright, R. K., & Phelps, E. A. (2000).Contribution of the anteromedial temporal lobes to the evaluation offacial emotion.Neuropsychology, 14, 526–536.

Borod, J. C., Obler, L. K., Erhan, H. M., Grunwald, I. S., Cicero, B. A.,& Welkowitz, J. et al., (1998). Right hemisphere emotional perception:Evidence across multiple channels.Neuropsychology, 12, 446–458.

Broks, P., Young, A. W., Maratos, E. J., Coffey, P. J., Calder, A. J., &Isaac, C. et al., (1998). Face processing impairments after encephalitis:Amygdala damage and recognition of fear.Neuropsychologia, 36, 59–70.

Calder, A. J., Young, A. W., Rowland, D., Perrett, D. I., Hodges, J.R., & Etcoff, N. L. (1996). Facial emotion recognition after bilateralamygdala damage: Differentially severe impairment of fear.CognitiveNeuropsychology, 13, 699–745.

Damasio, H., & Frank, R. (1992). Three-dimensional in vivo mapping ofbrain lesions in humans.Archives of Neurology, 49, 137–143.

Ekman, P., & Friesen, W. (1976).Pictures of facial affect. Palo Alto, CA:Consulting Psychologists Press.

Emery, N. J., Capitanio, J. P., Mason, W. A., Machado, C. J., Mendoza,S. P., & Amaral, D. G. (2001). The effects of bilateral lesions of theamygdala on dyadic social interactions in rhesus monkeys.BehavioralNeuroscience, 115, 515–544.

Frank, R. J., Damasio, H., & Grabowski, T. J. (1997). Brainvox:An interactive, multi-modal visualization and analysis system forneuroanatomical imaging.NeuroImage, 5, 13–30.

Fried, I., MacDonald, K. A., & Wilson, C. L. (1997). Single neuronactivity in human hippocampus and amygdala during recognition offaces and objects.Neuron, 18, 753–765.

Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The distributedhuman neural system for face perception.Trends in Cognitive Science,4, 223–233.

Hornak, J., Rolls, E. T., & Wade, D. (1996). Face and voice expressionidentification in patients with emotional and behavioral changesfollowing ventral frontal lobe damage.Neuropsychologia, 34, 247–261.

Kling, A. S., & Brothers, L. A. (1992). The amygdala and social behavior.In J. P. Aggleton (Ed.),The amygdala: Neurobiological aspects ofemotion, memory, and mental dysfunction. New York: Wiley–Liss.

Kluver, H., & Bucy, P. C. (1939). Preliminary analysis of functions ofthe temporal lobes in monkeys.Archives of Neural Psychiatry, 42,979–997.

Leonard, C. M., Rolls, E. T., Wilson, F. A. W., & Baylis, G. C. (1985).Neurons in the amygdala of the monkey with responses selective forfaces.Behavioral Brain Research, 15, 159–176.

Meunier, M., Bachevalier, J., Murray, E. A., Malkova, L., & Mishikin, M.(1999). Effects of aspiration versus neurotoxic lesions of the amygdalaon emotional responses in monkeys.European Journal of Neuroscience,11, 4403–4418.

Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, D., Young, A. W.,& Calder, A. J. et al., (1996). A differential neural response in thehuman amygdala to fearful and happy facial expressions.Nature, 383,812–815.

Rapcsak, S. Z., Galper, S. R., Comer, J. F., Reminger, S. L., Nielsen,L., Kaszniak, A. W., Verfaellie, M., Laguna, J. F., Labiner, D. M.,& Cohen, R. A. (2000). Fear recognition deficits after focal braindamage.Neurology, 54, 575–581.

Rosvold, H. E., Mirsky, A. F., & Pribram, K. (1954). Influence ofamygdalectomy on social behavior in monkeys.Journal of ComparativePhysiology and Psychology, 47, 173–178.

Schmolck, H., & Squire, L. R. (2001). Impaired perception of facialemotions following bilateral damage to the anterior temporal lobe.Neuropsychology, 15, 30–38.

Tranel, D. (1996). The Iowa–Benton School of NeuropsychologicalAssessment. In I. Grant, & K. M. Adams (Eds.),Neuropsychologicalassessment of neuropsychiatric disorders (pp. 81–101). New York:Oxford University Press.

Tranel, D., & Hyman, B. T. (1990). Neuropsychological correlatesof bilateral amygdala damage.Archives of Neurology, 47, 349–355.

Weiskrantz, L. (1956). Behavioral changes associated with ablation of theamygdaloid complex in monkeys.Journal of Comparative Physiologyand Psychology, 49, 381–391.

Young, A. W., Hellawell, D. J., Van de Wal, C., & Johnson, M. (1996).Facial expression processing after amygdalotomy.Neuropsychologia,34, 31–39.

Zola-Morgan, S., Squire, L. R., Alvarez-Royo, P., & Clower, R. P.(1991). Independence of memory functions and emotional behavior:Separate contributions of the hippocampal formation and the amygdala.Hippocampus, 1, 207–220.

amygdala damage impairs emotion recognition from scenes only when they contain facial expressions

Documents