investigating the cognitive neuroscience of social behavior

Neuropsychologia 41 (2003) 119–126

Investigating the cognitive neuroscience of social behavior

Ralph Adolphs∗Department of Neurology, University of Iowa College of Medicine, 200 Hawkins Drive, Iowa City, IA 52242, USA

Abstract

Social cognitive neuroscience is a fledgling discipline that has already accrued an impressive body of data, but important questionsremain regarding the theoretical constructs and methodological approaches that it utilizes. An overview of the papers in this special issuepoints to several key issues facing the field. We need a theoretical vocabulary that bridges three domains: our intuitive “folk” conceptionsof other people, the explanations offered by social psychology, and the explanations offered by cognitive neuroscience. And we need amethod that can extract common patterns across multiple studies, to complement strict hypothesis testing of individual studies. Theseissues can be addressed, in part, by giving theory and experiment equal time, and by fostering an interdisciplinary approach that includesneuroscience, psychology, philosophy, anthropology and allied disciplines.© 2002 Elsevier Science Ltd. All rights reserved.

Keywords: Folk psychology; Social cognition; Data-driven methods; Reduction; Neuroscience; Philosophy of science

1. Introduction

This introduction serves three purposes: to provide a briefoverview of the individual papers, to outline some of thefoundational issues that they raise, and to speculate on waysin which these issues might be addressed in the future. Itis not my intent to provide a review of the neurobiologyof social cognition here, for which the reader is referred todetailed extant reviews[1–5,8,14,15].

2. Overview of the studies in the special issue

The papers in this issue represent a sampling of currentwork in human and nonhuman primates. The four leadingpapers utilize functional imaging methods, the first twoPET (Decety and Chaminade; Wicker et al.) and the secondtwo fMRI (O’Doherty et al.; Pelphrey et al.), to investigateaspects of what social psychologists have called “personperception”[9,13]. The stimuli are pictures of people, andthe studies investigate the neural correlates of making cer-tain social judgments about the people that are viewed: howsuch pictures trigger sympathy, social directedness, attrac-tion, and attention. One strength shared by all the studiesis the attempt to construct specific stimulus categories; oneweakness they share is that none of the studies actuallyexamine the participants’ social behavior (at best, subjects

∗ Tel.: +1-319-353-8610; fax:+1-319-356-4505.E-mail address: [email protected] (R. Adolphs).

in the experiments provide ratings of the stimuli, presumedto correlate with how they would behave towards them ifencountered in real-life).

Despite the constraints imposed by using functionalimaging, and despite the varied approaches and stimuliused, the studies converge on a set of neural structuresthat are presumed to mediate our perception and interpre-tation of the social meaning of other people. These includehigher-order visual cortices in the temporal lobe, amygdalaand orbitofrontal cortex, and additional cortical regions,such as the left prefrontal and right parietal cortices. It istempting to regard these three sets of regions as implement-ing (respectively) three serial processing steps: an initialperceptual representation of the stimuli, a subsequent as-sociation of this perceptual representation with emotionalresponse and motivation, and a final central representationof both our emotional responses to, and our dispositionsand preparations to behave towards, the stimuli. Together,these varied processes generate not only social behavior, butalso social knowledge[2]. This picture is tempting, becauseit is plausible to assign these functions to the structureslisted; but it is certainly a vast oversimplification, for sev-eral reasons. One reason is that all of the structures listedare connected also via feedback, making it likely that theyparticipate at several points in time rather than a single one.Another reason is that there is good evidence for multiple,parallel routes of information processing. While a picture ofhierarchical processing certainly provides some insight, it isimportant to emphasize that the generation of socially rele-vant knowledge, just like the generation of social behavior,

0028-3932/02/$ – see front matter © 2002 Elsevier Science Ltd. All rights reserved.PII: S0028-3932(02)00142-2

120 R. Adolphs / Neuropsychologia 41 (2003) 119–126

will result from multiple, bidirectional tracks of informationprocessing.

No less difficult than a detailed neurobiological descrip-tion is a categorization of the processes under conceptsthat attempt to link classes of social stimuli to classes ofsocial behaviors. All of the studies use particular, socialconstructs in describing their experiments, terms like sym-pathy, attractiveness, and so on. In doing so, they also raisea thorny question: are there mental processes and neuralstructures “specialized” for social information, or do the so-cial constructs reduce to collections of simpler, non-socialprocesses?

The next two papers go some way towards addressingthis question, using a technique that provides a key elementmissing from functional imaging: fine-grained temporal res-olution (Smith et al.; Pizzagalli et al.). Using event-relatedpotentials (ERPs), these studies examine the neural cor-relates of processing negatively- and positively-valencedstimuli. These stimuli are not essentially social, but theyshare in common with social stimuli the fact that they evokeemotional responses in the viewer. This raises anotherthorny question: does all social cognition involve emotion?To some extent, the question is merely semantic, for exam-ple, we should by all rights consider language an aspect ofsocial cognition, but traditionally the field has been alliedmore closely with those aspects of cognition that draw onmotivation and emotion than those that do not.

The two ERP studies examine the timing of neural eventsin certain regions of the brain, and find reliable discrimi-nation between stimuli on the basis of their rewarding orpunishing contingencies. Several of the regions involvedoverlap with those identified in the previous four stud-ies, and some of the responses have a very short latency.This may reflect the rapid, coarse categorization in termsof valence to which social stimuli would also be subject.The findings provide some more detail to the picture wesketched above: one might imagine a rapid, perhaps entirelyfeed-forward, sweep of information processing that detectscertain highly salient features in stimuli, and that is ableto extract some very superordinate category information,such as “good/bad”, “approach/withdraw”. Subsequent pro-cessing, presumably relying in part on feedback and thegeneration of detailed information via stored associations,could then fill in this coarse categorization and providemore detailed descriptions of the social meaning of thestimuli. It is clear from the contributions discussed thus farthat the spatial resolution afforded by fMRI and the tem-poral resolution afforded by ERP studies complement oneanother, and a future aim should be their integration in thesame experiment, a mathematically challenging procedurethat has already seen some success.

Lesion studies provide data that neither functional imag-ing nor electrophysiology can: they indicate which struc-tures arenecessary for mediating certain behaviors. Thenext series of papers (Calder et al.; Phelps et al.; Stone et al.)provide a range of approaches that use some variant of the

lesion method, studies of normal aging, of a spared perfor-mance that is contrary to some functional imaging data, andof a complex impairment in inferring the mental states ofothers. It is reassuring that some of the structures outlinedin the functional studies, notably the amygdala, reappearin the lesion studies, increasing our confidence that theyparticipate in a substantive way in the social processes un-der investigation. At the same time, as the paper by Phelpset al. shows, there are cases where structures activated infunctional imaging studies fail to give rise to impairmentswhen lesioned, presumably reflecting the fact that they nor-mally participate in, but are not essential to, the processesexamined.

The study by Stone et al. revisits a question raised ear-lier, in relation to an ability that has been dubbed “theoryof mind”, our propensity to assign internal mental states toother people on the basis of, and to aid prediction of, theirbehavior. This ability may well fragment into a collectionof different strategies and processes that may share little incommon except their goal. On the other hand, “theory ofmind” processes may all require a small set of specific struc-tures that are especially important. One could even imaginea dedicated system, of a central executive sort, that wouldcoordinate these disparate abilities and decide how to deploythem in a particular social situation. These latter possibili-ties could yield neural structures that are especially impor-tant for theory of mind abilities, even though theory of mindis not a functionally homogeneous category of processes.

The next pair of papers shows a pattern of strengths andweaknesses in many ways the converse of the first severalpapers: they provide detailed behavioral assessment of so-cial behaviors, but they do not attempt to investigate theneural substrates as such (deWaal et al.; deVeer et al.). Thedifferent strengths of the studies reflect to a large extent thedifferences between studies in human and in nonhuman pri-mates: real-life human social behavior is difficult to assess,and functional imaging or invasive studies are difficult toperform in nonhuman primates. The studies in nonhumanprimates are critical if we want to understand the precursorsout of which many of our social behaviors may have arisen.Developmental studies in humans (not represented here) areimportant for the same reason. Such studies also shed lighton the question of specialization and domain-specificity: isthere evidence for processes specific to social behavior, oris social behavior assembled out of collections of processesthat are less specialized?

The final contribution (Amaral et al.) summarizes an am-bitious set of studies that aim to combine behavioral, lesion,and developmental investigations of the amygdala in mon-keys. David Amaral and Jocelyn Bachevalier are two investi-gators attempting this approach, with some startling successand a tremendous potential for developing animal models ofcertain human diseases, notably autism. While it is rare tofind such an integrated set of approaches in a single labora-tory, there is an ever growing number of collaborations thataim to foster a similar diversity of methods.

R. Adolphs / Neuropsychologia 41 (2003) 119–126 121

The upshot from all the papers is clear: social cognitiveneuroscience must be an intensely collaborative venture ifit is to succeed. It must include data from ethologists, neu-robiologists, anthropologists, and psychologists, and, if theyare to understand one another, investigators from these dif-ferent backgrounds must pursue a common language withinwhich to communicate their ideas and findings. To comple-ment the empirical papers that comprise this special issue,I will briefly outline what I see as some of the most criticalissues the field currently faces, and suggest ways in whichI think they could be pursued.

3. Foundational issues

Two key issues any new discipline faces concern itstheory and method. It is clear, on the one hand, that ourbehavior is in many respects specialized for guiding our in-teractions with others, that social behavior is more complexthan other aspects of behavior, and that our social behavioris at least quantitatively (and perhaps qualitatively) vastlydifferent from the social behavior of other species. On theother hand, it is equally apparent that there is phylogeneticcontinuity in both brain and behavior, and that social cog-nition depends on many of the same mental processes, andhence presumably on many of the same brain structures,as do non-social aspects of cognition. Do we need an ad-ditional vocabulary to explain the cognitive processes thatguide social behavior? Or can social behavior be linkedto neurobiology using existing, domain-general constructs,such as attention, memory, and so forth? I take up thesetheoretical issues in the following section.

No less daunting a problem than its theoretical frameworkare the methods of social cognitive neuroscience. In part, thedifficulty can be traced to the large bias in studying humansrather than other social species. There are good examples ofneurobiological studies of social behaviors in songbirds, involes, in sheep, and many other nonhuman species; but theuse of the label “cognitive” has typically led researchers toemphasize primates, especially humans, at the expense ofother species. The emphasis on research in humans in turnhas led to an emphasis on methods suitable for studying thatspecies, notably functional imaging, complemented by ERPsand studies of neurological and psychiatric populations. Ofthese, functional imaging has become the most popular andin many respects the most seductive tool, with a resultingglut of functional imaging studies in the face of a dearth oftheories for understanding the data they generate.

The nature of the topic studied, together with the natureof the methods used, have led to a difficulty in producingresults that show the same rigorous hypothesis testing de-manded of many other branches of science. How do wego about operationalizing an experiment to investigate, say,empathy or attractiveness? How can we possibly control forall confounding factors in such experiments? The quick an-swer is that we cannot provide precise definitions of many

of the constructs we use, and that it is impossible to elim-inate all confounds. But this does not invalidate the studyof social cognition; it suggests rather that our approach tostudying social behavior should have somewhat differentmethodological criteria from our approaches to studyingother aspects of biology.

A recent review of functional imaging databases has citedthe chemist Lee Hood in foreseeing a “discovery science”[18], in which strict hypothesis testing by individual exper-iments is replaced with a more open approach driven bythe accumulation of data. The approach acknowledges thatindividual studies, taken in isolation, may not permit deci-sive conclusions, but emphasizes that accrual of data frommultiple studies can lead to new insights. Meta-analyses,data-mining, and the tools of bioinformatics are making suchan approach possible. It may thus be that an individual func-tional imaging study that investigates a particular constructin social cognition is limited by ineliminable confounds, butthat the consistent activation within certain structures seenacross many studies nonetheless may permit us to establishthe reliability of the brain-cognition relationship. As the neu-robiological study of social behavior matures, it may definea new approach to the topic that also acknowledges the useto which sheer data can be put in order to forge a theoreticalframework in an as yet immature discipline.

4. Ten questions for the cognitive neuroscience ofsocial behavior

Considering the above issues, one might begin to providea sketch of how the neuroscientific study of social cognitionwill appear sometime in the future. Many of these issues, ofcourse, apply to cognitive neuroscience in general, but theyarguably are more acute for the cognitive neuroscience ofsocial behavior. The first five questions raised below con-cern issues of method, and the last five, issues of theory.The methodological issues pick up where we left off in thelast section and ask how we should best develop and useavailable techniques to increase the reliability of data, andto best integrate data and theory. The theoretical issues con-sist, in large part, not in providing answers to initial ques-tions, but rather in replacing the questions with differentones whose pursuit more closely reflects what interests us.Both methodological and theoretical questions circle backto a single overarching query: what do we really want so-cial cognitive neuroscience to provide? Insofar as there area variety of answers to that question, there will be a varietyof approaches to all the others.

4.1. Five questions of method

1. How can we measure social behavior? Providing anethogram of human social behavior is certainly feasiblein part, although the enormous variance and complexityof human social behavior would preclude any exhaustive


description. Detailed descriptions of nonhuman primatesocial behaviors are carried out regularly, although theyare very labor intensive. We already have some char-acterizations of statistical patterns in human behavior,personality traits describe exactly that. But is that levelof classifying behavior going to be useful for cogni-tive neuroscience investigations? No doubt, we can findsome correlations between personality traits and neuralactivity in certain structures (e.g.[6]), just as we havefound correlations between certain stimulus dimensionsand neural activity (see question 2 below). The prob-lem is not that no such systematic associations betweenneural function and categories of behavior or of stimuliwill be found: quite the opposite, the problem is thatwe will find too many. Such multiple correlations wouldpresumably indicate an association driven by the pres-ence of a factor (or set of factors) that all the correlatingmeasures share in common, but it is critical to note thatnone of these factors need be ones we have at hand now,or that we could construct a priori. One could thus imag-ine, with sufficient data, an extraction of those factorsthat correlate best with brain function across multiplestudies. All the approach requires is that we be openminded about revising the categories of social behavioras initially conceived. In principle, what classificationsof behavior we start with is not critical, for providedwe gather enough data on a variety of different behav-iors, we should be in a position to isolate from all thedata those neural systems whose activity shows regularcovariances with factors that best predict the behavior.

2. An analogous question arises in regard to the stimuli: howshould they be categorized? Two approaches have beenused: characterize them solely in terms of their physicalproperties, or in terms of a priori specified categories thatcome from some theory. The latter route includes cate-gories that come from various psychological theories, orcategories that attempt to be “ecologically valid” in somesense. As with the classification of behaviors, it turns outto be of secondary importance where we start, providedthat we are willing to revise our original stimulus cate-gories. The extreme form of an atheoretical approach issomething like reverse correlation: a very large number ofrandom stimuli are used in order to extract stimulus-braincovariances in an unbiased manner. A similar approachcan be used to derive stimulus features associated withbehaviors[10]. But such an approach does not requirethat we start with a completely blank slate and use no apriori stimulus categories whatsoever; instead, it is per-fectly feasible (in fact, preferable) to start with somepre-defined stimulus categories and yet use those initialcategories to derive other, different categories (e.g.[17]).It may thus be possible to “bootstrap” the categories ofbehavior and of stimuli by starting with whatever cate-gories we now have available, and by using methods thatpermit the extraction of new, refined categories that arelinked more tightly to neural function.

3. How can we best use data to guide theory? This questionnaturally arises from a consideration of the precedingones. What are these methods that will permit us to formnew categories of stimuli and behavior when applied tothe data? There is no shortage of them, and they all fallunder the heading of exploratory data analysis, as con-trasted with the traditional hypothesis-testing approach.Factor analysis, multidimensional scaling, clustering, andother graphical techniques are used regularly to achieveprecisely the goal we have in mind here. There is anotherimportant consideration, however: in order for these ex-ploratory methods to work best, they should be brought tobear on data from multiple studies, which means that weshould emphasize the availability of the information thatthey require. The information they require is more thanjust the statistically significant results typically reportedin papers. Reporting only results that meet some signifi-cance threshold (e.g.P < 0.05) is a bane of the literaturein all of behavioral science that has been bemoaned bystatisticians for some time[7,11], although their moanshave largely fallen on deaf ears. What we need are re-ports of effect sizes, confidence intervals, or raw data, inaddition to merely statistically thresholded findings thatomit such information. Thus, our primary question howto use the data to guide the theory can be modified to askhow we should best report the data so that it can be thusused. We already have methods that permit the discoveryof new categories from the data; what we need are theappropriately reported data to use in the first place.

4. The previous question brings up a larger concern: what isthe most appropriate way to interpret the data, even oncewe have a mature theory available? Critiques of the stan-dard null-hypothesis-significance-testing approach havebeen quick to point out a common fallacy in using thisapproach: it only tells us the probability of observingthe data given that the null hypothesis is true (a processof deduction), when we actually want to know the con-verse, the probability that the null hypothesis could betrue given the data observed. In actual fact, the null hy-pothesis is almost always false, since there is almost al-ways some effect (some difference between experimentalconditions, or between experiment and control), howeverminuscule it may be. The problem is of general concern,as has been amply voiced by statisticians for some time,but I think it might be particularly acute for the investi-gation of social behavior. This is because, unlike perhapsin chemistry, engineering, or physics, studies of socialbehavior typically involve many interacting factors overwhich we have rather poor control. We are always inves-tigating highly interconnected, complex causal networks.It is therefore, nearly certain that we will observe someeffect, if only our methods are sufficiently sensitive todetect it. Given that, strictly speaking, the null hypothesisis almost certainly false, the familiar functional imagingpictures of statistically thresholded contrasts are not re-ally what we want to know. If the study had a sufficiently


large sample size, nearly every voxel in the brain couldbe made to meet an arbitrarily small statistical threshold.In addition to knowing that there was a significant dif-ference between conditions at a voxel, we would like toknow where the largest magnitude of this effect is seenin the brain, and we would want to compare effect sizesacross studies. That is, we want to know, quantitatively,when we experimentally vary the stimuli, or when weobserve variance in the behavior, where and when in thebrain do we see the largest corresponding variance inbrain function? Such data will be most useful in givingrise to a causal theory that relates stimuli, behavior, andbrain function in a quantitative fashion.

The standard fashion of depicting data solely in termsof their statistical significance is misleading in otherways as well. For one, it gives us a false sense of lo-calization of function in the brain. A lesion study mayshow that damage to only a specific structure yieldsa statistically significant impairment, and a functionalimaging study may show significant activation belowsome P-value only at a particular location within thebrain. From this, we are tempted to conclude erroneouslythat the process in question has been localized to thatparticular structure in the brain. Quite the opposite istrue, if we had a very large sample of lesion subjects,or if we included very many subjects in our functionalimaging study, we would find that nearly every locationin the brain shows a significant effect. Does the brainwork by mass action after all? Far from it, we are justusing the wrong tool to interpret our results.

5. How can we best establish the reliability and general-izability of our results? If single studies cannot be usedto generate the kinds of causal theories we desire, whatcan? The answer is of course independent replication,but it is worth considering what “independent” meanshere. The best situation would be not to replicate afinding using exactly the same methods, but using adifferent approach that provides convergent findings. Inthis way we can be reassured that the results are not dueto some idiosyncratic feature of the particular methodor design. The goal should thus be to combine datafrom a variety of different approaches: lesion studies,functional imaging studies, studies in humans, studiesin animals, studies at cellular level, studies at systemslevel. Some such convergences are evident in the lit-erature already, and indeed in the contributions to thisissue.

Designing tasks and stimuli that could be appliedacross studies using very different techniques is chal-lenging, as is the allocation of equal funding resourcesto all the different techniques. But an appreciation ofthe importance of having a broad armamentarium ofmethods available is necessary if we want to go fromdata to theory. The first step in the process is simply toget investigators working with different approaches totalk to one another and there are now many examples of

such collaborative projects underway, especially in thedomain of social cognition.

What we do all the time is to derive hypotheses fromdata, even though no one has so far been able to justifysuch an approach (statistical tests are typically deductionsfrom theory to data). But the fact is that such abductionor induction is what we want to do, and we do it, more orless successfully, all the time. To reiterate the substanceof questions 1–5, what we need to focus on are ways ofdescribing, depicting, publishing, and discussing the datathat facilitate such an approach rather than hinder it. Theimpression one gets from considering the above ques-tions is that we already have in place many of the toolsthat we need in order to explore data for new regularitiesbetween stimuli, brain, and behavior. We just have tolearn how to apply them better and more consistently.An emphasis should be placed on educating researchersabout the different methods available, about the mean-ing of null-hypothesis significance testing, and aboutthe forms of data presentation that are most useful toprovide in research publications and databases. More de-bate is needed, and more consensus needs to be reached,on how best to implement the tools we already haveavailable.

4.2. Five questions of theory

Using the tools we have appropriately, and developingnew tools for data analysis, are certainly important issues.But no less important is work on the theoretical framework,raising five further questions I turn to next.

6. How theoretical should we be? One might begin by con-sidering the extent to which a consideration of theoret-ical issues is useful at all. While all data are necessarilyinterpreted in light of some theory, this may be a veryunarticulated, implicit theory. Of course, it is perfectlypossible to provide results that are of benefit to humanhealth, that permit a better understanding of the relationbetween brain and behavior, and that are interesting, inthe absence of explicit theoretical advances. It becomesless clear that we can do so repeatedly, unless we arewilling to be content with what will become an unman-ageably long list of datapoints that cannot be related toan overarching theoretical context. While emphasis ondata or on theory can to some extent be personal prefer-ences, none of us can afford to be without theory in thelong run, because theory is required if we want to situatethe data within a context so that they can have a broaderinfluence. So the question is not whether we should betheoretical or atheoretical; the question is whether thetime for explicit theorizing is right now, or whether itis premature to theorize until more data have been col-lected. Even preliminary theories can help us to organizethe extant data, and can guide future experiments. Onthe other hand, those with a penchant for constructing


detailed theories might be wise to wait a little, for thelonger they wait, the more likely they are to get it right.

7. Should the investigation of social cognition rely on aproprietary vocabulary, or should it attempt to ask itsquestions, and interpret its findings, in an existing vocab-ulary? If we acknowledge that, sooner or later, we willneed theory, the question arises what vocabulary such atheory should draw upon. Initially, we will probably haveto rely largely on what we have already got, which are thevocabularies of social psychology on the one hand, and ofcognitive psychology on the other, as the papers here do.But this leads to a curious dichotomy, because the formertypically assumes domain-specificity in a way in whichthe latter does not. Terms, such as dominance, attrac-tiveness, sympathy, trustworthiness, and so on all applyexclusively to social stimuli, whereas terms such as mem-ory and attention apply equally to social and non-socialtopics. What we really need is neither a proprietary vo-cabulary that would insulate the study of the social, norto subsume it into domain-general concepts. What weneed is a set of concepts that at the same time elucidatewhat is special about social cognition, while makingexplicit how social processes are instances of, are con-stituted by, or connect with, domain-general processes.

8. This brings us to one of the key questions in the field,which I already raised earlier: Are social cognitive pro-cesses reducible to non-social cognitive processes? Ananswer to this question of course depends on what onemeans by reduction. But regardless of how one con-strues that term, the picture is clear enough: are thereprocesses, and are there neural structures, that are insome way designed, specialized, and best understoodas subserving the perception of socially relevant stim-uli and the guidance of social behavior? Or is all suchspecialization merely an artifact that arises solely out ofthe particular computational demands made by a classof social stimuli? Readers will be familiar with a con-crete example, the debate over whether or not regions ofthe fusiform gyrus are specialized for processing faces,or instead implement domain-general processes thathave particular computational features (such as expertsubordinate-level categorization of perceptually ambigu-ous stimuli) [12,16]. Of course, it is certainly possibleto narrow down the options, for example by finding (orfailing to find) suitable dissociations. Are all patientswho are impaired in recognizing faces also impairedin recognizing other classes of visual stimuli? Do onlyfaces activate certain sectors of the fusiform gyrus?

While investigating these questions has indeed nar-rowed down the options (faces are not just like any otherclass of visual stimuli; but they may not be completelyunique either), it has still left us with the core question:are our brains in some way designed to process specificclasses of stimuli, like faces, and, if so, what are theseclasses? With regard to present purposes: are there brainsystems adapted for social behavior, or are all brain

systems more flexible in function than that, and merelymake available a collection of abilities which social aswell as non-social behavior can draw upon?

The question requires approaches from an evolution-ary or developmental viewpoint, but even so, it is farfrom clear that it could be completely answered, evenin principle, because it seems that any evidence ob-tained ultimately could be interpreted as consistent witheither view. Thus, it may be that our worry about theprecedence of domain-specific or domain-general pro-cessing is misplaced, and we might instead consider thealternative question, “how are social cognitive processesrelated to non-social cognitive processes”, without as-suming any precedence of one over the other in the waythat the term “reduction” implies. Rather than reducingthe social to the non-social, or vice versa, we shouldbe content with relating the two as instances of a largercategory that includes both.

The papers in this issue also provide a good exampleof the problem: the function of the amygdala and or-bitofrontal cortex, two structures whose role in process-ing the rewarding or punishing values of stimuli has beenwell established in animal experiments. Human studiesalso demonstrate that these two structures participate ina wide variety of emotion and motivation-related infor-mation processing, even when the stimuli are in no waysocial. But they also are critical for social behavior, andthere is some evidence that they may be more importantfor regulating social behavior than non-social behavior.How do we interpret these findings? One story would bethat these structures evolved as part of a neural systemfor processing reward/punishment, and for linking suchprocessing to motivated behavior, and that social behav-ior is merely one particular instance of such processing.Another story would turn this description completelyaround, and propose that these structures evolved specif-ically to regulate social behavior, because of the partic-ular demands on information processing that it makes,and that those same structures can then also be co-optedfor non-social behaviors. As with the debate about facesand the fusiform gyrus, it is not clear that we can decidebetween these two possibilities on the basis of the exper-imental evidence alone. Another ingredient in favoringone explanation over the other has to do not with thedata per se, but with our goals in what we wish to under-stand: if we want to investigate how social stimuli areprocessed, it makes sense to focus on these structures ascomponents of a system for processing socially relevantinformation; if we are interested in more abstract compu-tational abilities that constitute such processing, it makessense to focus on the general computational abilities thatthese structures implement; both descriptions are valid.

9. This brings up an issue that is worth considering. Howwill we be able to understand a future social cognitiveneuroscience? Suppose, well in the future, that we haveaccrued a vast amount of data concerning the neural bases


of social cognition. Our neurobiological explanation ofthe behavior in terms of brain processes, whatever its finalform may be, is likely to be quite different from the intu-itive, “folk-psychological” explanations of other peoplewe currently have. What do we want a neurobiologicalexplanation of social behavior to provide? It is not clearthat a future vocabulary that is maximally predictive oreconomical with respect to the neurobiological data willat the same time be easy to understand, because the ex-planatory constructs it employs may not translate to anyof our intuitive concepts regarding the behavior of people.

The issue contrasts with our account of the physi-cal world, because we are not that invested in the “folkphysics” our pre-scientific explanations relied upon. Weregularly encounter explanations from expert physiciststhat go well beyond what we can imagine or understand,but we accept that such accounts provide in some sensebetter descriptions behind the physics as they appear tous. Not so with our observations of other people: we neverhave just the appearances to begin with; all of the psycho-logical properties we attribute to them are theoretical inthe first place, in the sense that they go beyond what wemerely observe. The reason that we can understand whatsocial psychologists say is that they use concepts thatmake intuitive sense: we can relate them (with some workand training) to our pre-scientific, “folk” account of otherpeople. But for the same reason we may be unable to un-derstand the picture of social cognition that a future neu-roscience might provide. There is no reason to think thatit would map onto the theoretical primitives that derivefrom our folk psychology. I think there can be no doubtthat cognitive neuroscience can provide data showing thatbrain processes constitute the mechanisms by which weinteract with others, but there remains serious doubt inwhat way this will count as an explanation. If those datarequire a theory that is entirely alien, we may come tothink that we have simply changed the subject matter.

The problem I am dwelling on here is not that ourfolk psychology would be eliminated, because I do notthink that it could be. And, if we believe in conceptualplasticity, we may very well have a more scientific futurefolk psychology, something like what social psychologyand psychoanalysis have already in part provided us.We may have something like this, on the one hand; buton the other hand we may also have a neuroscientifictheory whose terms do not pick out the same patternsof social behaviors or social stimuli, because those cate-gories fragment into disjunctive sets of neural structuresand processes once we go inside the skull. It may be thatany possible future “folk” psychology will have to bein some sense continuous with our present folk psychol-ogy, because the primitives that it provides turn out to beindispensable for our understanding of other people. Ifa future social cognitive neuroscience is not continuouswith folk psychology in that sense, we may not be ableto adopt it as way of making sense of other people’s

behavior (except perhaps as an abstract exercise wherewe literally do not know the meaning of the constructswe manipulate). What we will need is a vocabulary thataccommodates and bridges three distinct ways of under-standing other people: our folk conceptions of others, theexplanations offered by social psychology, and those thatwill be offered by cognitive neuroscience. This is a tallorder, and the only hope of filling it is to accord all thesethree domains equal importance, rather than trying toreduce any of them to one or a combination of the others.

10. How integrative should social cognitive neuroscience be?Just like question 5 motivated us to use methods froma variety of different disciplines, we should consider thevariety of different theories different disciplines can of-fer. Given the worry raised in the previous point, we needto consider fields in addition to neuroscience in orderto provide guidance, while at the same time consideringhow much input to use from fields outside neuroscience.

What investigations should count as “social cognitiveneuroscience”? While the initial urge may be to pleaseeveryone by being all-inclusive, there is something to besaid for demarcating the boundaries of a field. For in-stance, the field should not encompass all investigationsof social behavior, just as it does not encompass all inves-tigations of cognition. So far, the emphasis has been tofocus on those aspects of social cognition that are closelyallied with emotion, and to exclude other domains thatare clearly social, but that already specify a huge and ma-ture field of their own, such as the study of language. Wecan only wait to see if the accruing data will demarcatea sufficiently distinct domain to justify a separate term,or whether all future social cognitive neuroscientists willsimply come to think of themselves as cognitive neurosci-entists who happen to be working on a particular topic. Inlarge part, the way in which the previous questions playout will influence the relative autonomy of the field. If weend up needing terms like sympathy, attractiveness, dom-inance, trustworthiness in our vocabulary to account forthe patterns of data, then it seems clear that social cogni-tive neuroscience will be seen as a distinct field with somesovereignty over the questions that it asks. If we end upaccounting for the data entirely in terms of processes thatare in no way specific to social behavior, if the vocabularyof motivated behavior in general suffices, for instance,then it is less clear that the neurobiological study of so-cial cognition can endure as a distinct field of its own.

The above ten questions force us to think about whatexactly it is that we wish to know from social cognitiveneuroscience, and what exactly it is that we should do in or-der to get to what we wish to know. I have, of course, onlyscratched the surface here; each question could be expandedinto a paper in its own right. We should ask ourselves whatkind of theory it is we want to have, and at the same time weshould ask how appropriate are the methods to achieve sucha theory. One bad tendency is to presume that deductive


methods (e.g. null-hypothesis significance testing), or re-ductive theories, are in some way superior. They are notreally superior or inferior, they often are simply besides thepoint of what really interests us. The study of social behav-ior is indeed in some sense a “soft” science, but this does notmean that conceptual relativism reigns as far as theories areconcerned, nor that everything is correlated with everythingelse as far as data are concerned. Yes, we should be openand creative in constructing our theoretical framework, andthere are indeed a multitude of patterns in the data. Whatwe need to do is to describe them accurately, quantitatively,and consistently in a manner that everyone can understand,while always questioning what they mean. At the rate thedata are pouring in, the sooner we begin to think about whatprecisely it is that we wish to understand, the sooner we canbegin to forge the analytical toolkit and theoretical frame-work that a future social cognitive neuroscience will require.

Acknowledgements

This paper benefited from critiques by Anthony Atkin-son, David Milner, Ian Gold, Max Coltheart, and MorrisMoscovitch.

References

[1] Adolphs R. Social cognition and the human brain. Trends in CognitiveSciences 1999;3:469–79.

[2] Adolphs R. Recognizing emotion from facial expressions: psycho-logical and neurological mechanisms. Behavioral and CognitiveNeuroscience Reviews 2002;1:21–61.

[3] Allison T, Puce A, McCarthy G. Social perception from visual cues:role of the STS region. Trends in Cognitive Science 2000;4:267–78.

[4] Brothers L. The social brain: a project for integrating primatebehavior and neurophysiology in a new domain. Concepts inNeuroscience 1990;1:27–51.

[5] Cacioppo JT, Berntson GG, Adolphs R, Carter CS, Davidson RJ,McClintock MK, et al., editors. Foundations in social neuroscience.Cambridge (MA): MIT Press; 2001.

[6] Canli T, Sivers H, Whitfield SL, Gotlib IH, Gabrieli JDE. Amygdalaresponses to happy faces as a function of extraversion. Science2002;296:2191.

[7] Cohen J. The earth is round (P < 0.05). American Psychologist1994;49:997–1003.

[8] Emery NJ. The eyes have it: the neuroethology, function andevolution of social gaze. Neuroscience and Biobehavioral Reviews2000;24:581–604.

[9] Gilbert DT. Ordinary personology. In: Gilbert DT, Fiske ST, LindzeyG, editors. The handbook of social psychology. 4th ed. Boston:McGraw-Hill; 1998. p. 89–150.

[10] Gosselin F, Schyns PG. Bubbles: a technique to reveal the use ofinformation in recognition. Vision Research 2001;41:2261–71.

[11] Harris RJ, editor. Special section on the use of significance testing.Psychological science, vol. 8. 1997. p. 1–20.

[12] Kanwisher N. Domain specificity in face perception. NatureNeuroscience 2000;3:759–63.

[13] Macrae CN, Bodenhausen GV. Social cognition: thinking cate-gorically about others. Annual Review of Psychology 2000;51:93–120.

[14] Ochsner K, Lieberman MD. The emergence of social cognitiveneuroscience. American Psychologist 2001;56:717–34.

[15] Panksepp J. Affective neuroscience. New York: Oxford UniversityPress, 1998.

[16] Tarr MJ, Gauthier I. FFA: a flexible fusiform area for subordinate-level visual processing automatized by expertise. Nature Neuro-science 2000;3:764–9.

[17] Theunissen FE, Sen K, Doupe A. Spectral-temporal receptive fieldsof nonlinear auditory neurons obtained using natural sounds. TheJournal of Neuroscience 2000;20:2315–31.

[18] Van Horn JD, Gazzaniga MS. Databasing fMRI studies—towards adiscovery science of brain function. Nature Reviews Neuroscience2002;3:314–8.

investigating the cognitive neuroscience of social behavior

Documents