Full Terms amp Conditions of access and use can be found athttpwwwtandfonlinecomactionjournalInformationjournalCode=pcgn20

Download by [University of South Carolina ] Date 19 February 2017 At 1622

Cognitive Neuropsychology

ISSN 0264-3294 (Print) 1464-0627 (Online) Journal homepage httpwwwtandfonlinecomloipcgn20

Toward a brain-based componential semanticrepresentation

Jeffrey R Binder Lisa L Conant Colin J Humphries Leonardo FernandinoStephen B Simons Mario Aguilar amp Rutvik H Desai

To cite this article Jeffrey R Binder Lisa L Conant Colin J Humphries LeonardoFernandino Stephen B Simons Mario Aguilar amp Rutvik H Desai (2016) Toward a brain-basedcomponential semantic representation Cognitive Neuropsychology 333-4 130-174 DOI1010800264329420161147426

To link to this article httpdxdoiorg1010800264329420161147426

View supplementary material

Published online 16 Jun 2016

Submit your article to this journal

Article views 596

View related articles

View Crossmark data

Citing articles 4 View citing articles

Toward a brain-based componential semantic representationJeffrey R Bindera Lisa L Conanta Colin J Humphriesa Leonardo Fernandinoa Stephen B SimonsbMario Aguilarb and Rutvik H Desaic

aDepartment of Neurology Medical College of Wisconsin Milwaukee WI USA bTeledyne Scientific LLC Durham NC USA cDepartment ofPsychology University of South Carolina Columbia SC USA

ABSTRACTComponential theories of lexical semantics assume that concepts can be represented by sets offeatures or attributes that are in some sense primitive or basic components of meaning Thebinary features used in classical category and prototype theories are problematic in that thesefeatures are themselves complex concepts leaving open the question of what constitutes aprimitive feature The present availability of brain imaging tools has enhanced interest in howconcepts are represented in brains and accumulating evidence supports the claim that theserepresentations are at least partly ldquoembodiedrdquo in the perception action and other modal neuralsystems through which concepts are experienced In this study we explore the possibility ofdevising a componential model of semantic representation based entirely on such functionaldivisions in the human brain We propose a basic set of approximately 65 experiential attributesbased on neurobiological considerations comprising sensory motor spatial temporal affectivesocial and cognitive experiences We provide normative data on the salience of each attributefor a large set of English nouns verbs and adjectives and show how these attribute vectorsdistinguish a priori conceptual categories and capture semantic similarity Robust quantitativedifferences between concrete object categories were observed across a large number ofattribute dimensions A within- versus between-category similarity metric showed much greaterseparation between categories than representations derived from distributional (latent semantic)analysis of text Cluster analyses were used to explore the similarity structure in the dataindependent of a priori labels revealing several novel category distinctions We discuss howsuch a representation might deal with various longstanding problems in semantic theory suchas feature selection and weighting representation of abstract concepts effects of context onsemantic retrieval and conceptual combination In contrast to componential models based onverbal features the proposed representation systematically relates semantic content to large-scale brain networks and biologically plausible accounts of concept acquisition

ARTICLE HISTORYReceived 14 June 2015Revised 24 November 2015Accepted 21 January 2016

KEYWORDSSemantics conceptrepresentation embodiedcognition cognitiveneuroscience

Introduction

Theories of semantic representation generally beginwith analyses of conceptual content This analyticalor componential approach assumes that conceptscan ultimately be represented by sets of features orattributes that are in some sense primitive or basiccomponents of meaning The present availability ofbrain imaging tools has enhanced interest in how con-cepts are represented in human neural systems andaccumulating evidence supports the claim that theserepresentations are at least partly ldquoembodiedrdquo in theperception action and other modal neural systemsthrough which concepts are acquired (Binder ampDesai 2011 Kiefer amp Pulvermuumlller 2012 MeteyardRodriguez Cuadrado Bahrami amp Vigliocco 2012) In

this paper we explore the possibility of devising a suf-ficiently explanatory componential model of semanticrepresentation based entirely on such functional div-isions in the human brain

An essential starting point is to define what types ofentities are the components of such a representationand how these differ from those of more standardsemantic theories In classical category theory con-cepts are defined by the presence or absence ofbinary features that are necessary and sufficient forcategory identification (Aristotle 1995350 BCE) Theconcept of bird for example is composed of such fea-tures as wings feathers beak flying egg-laying and soon An important extension of classical theory was therealization that most categories have somewhat fuzzy

copy 2016 Informa UK Limited trading as Taylor amp Francis Group

CONTACT Jeffrey R Binder jbindermcweduSupplemental data for this article can be accessed at httpdxdoiorg1010800264329420161147426

COGNITIVE NEUROPSYCHOLOGY 2016VOL 33 NOS 3ndash4 130ndash174httpdxdoiorg1010800264329420161147426

boundaries due to varying degrees of prototypicalityamong members (Rosch Mervis Gray Johnson ampBoyes-Braem 1976) Thus a penguin could still beidentified (though more slowly) as a bird despite thefact that it does not fly through a process of jointlyweighing all of its features Extensive work usingfeature generation tasks has documented whatpeople think of as the features of entities and howthese features rank in importance for a givenconcept (Cree amp McRae 2003 Garrard LambonRalph Hodges amp Patterson 2001 McRae de Sa ampSeidenberg 1997 Vinson Vigliocco Cappa amp Siri2003) Such data provide a powerful means of asses-sing degrees of similarity between concepts and ofgrouping concepts into hierarchical categories onthe basis of this similarity structure

A limitation that applies to these standardapproaches however is that the features theyemploy to define conceptual content are typicallyalso complex concepts Physical features like wingsfeathers and beaks are components in the sensethat they are parts of a larger entity but they are nomore primitive than the larger entities they defineLike the larger entities these parts are concepts thatmust be learned through perceptual and verbalexperience and each has its own set of defining fea-tures which in turn have their own defining featuresand so on The resulting combinatorial explosion pre-sents a bracing problem for constraining feature-based theories Across even the closed set of knownphysical objects the possible set of physical featuresis extremely large and the same is true for motionand action features such as flying swimmingrunning jumping eating singing barking howlingand so on The inability of standard verbal feature-based theories to locate a set of atomic featuresfrom which all others are derived places hard limitson the explanatory value of such approaches inseveral areas

One of these limitations and the one of centralconcern here is the lack of any known relationshipbetween verbal features and neurobiological mechan-isms There are no neural systems specifically dedi-cated to representing feathers wings beaks orflight nor is it plausible that such dedicated neuralsystems exist for every conceivable feature Even if itturns out that the brain representation of featuresand other complex concepts includes neurons orlocal neuron ensembles dedicated to a single

concept (Barlow 1972 Bowers 2009) the mere exist-ence of such ldquograndmother cellsrdquo in itself provides noaccount of the means by which external or internalstimuli lead to their activation Without such amechanistic account the claim that each possibleconcept is represented in the brain by a cell orgroup of cells adds nothing to our understanding ofconcept representations beyond a mere listing ofthe concepts What we seek instead is an understand-ing of why concepts are organized in the brain in theway that they actually are organized What facts aboutthe brain determine whether a concept is representedby one group of cells rather than another group Aclosely related problem for which feature-based con-ceptual representations are similarly limited in provid-ing an answer is the question of how features andother concepts come to be learned by the brain Theidea that concepts are represented in the brain byabstract ldquoconcept cellsrdquo also fails to address thedeeper question of how concepts can be understoodwithout a ldquogroundingrdquo in sensoryndashmotor experiencethat enables reference to the external environmentand the phenomenological qualia of consciousthought (Harnad 1990)

The limitations of traditional feature-based seman-tic theories for understanding concept representationin the brain are however a direct reflection of whatthese theories aim to achieve which is to describethe things that exist their similarity structure andtheir groupings into categories The question of howsuch information is organized in the brain is outsidethe domain of these aims so it should not be surpris-ing that these theories have the limitations they haveIn contrast embodiment theories of knowledge rep-resentation provide a fairly straightforward analysisof conceptual content in terms of sensory motoraffective and other experiential phenomena andtheir corresponding modality-specific neural represen-tations In the theory outlined here these neurobiolo-gically defined ldquoexperiential attributesrdquo provide a setof primitive features for the analysis of conceptualcontent while simultaneously grounding concepts inexperience and providing a mechanistic account ofconcept acquisition

Defining conceptual features in terms of neural pro-cesses represents a radical departure from traditionalverbal feature analysis Prior work along theselines includes the Conceptual Semantics program ofJackendoff (Jackendoff 1990) which conceives of

COGNITIVE NEUROPSYCHOLOGY 131

semantic content in terms of experiential primitivesemphasizing in particular the role of space timeevent and causality phenomena in understandingpropositional content Other related early workincludes the sensoryndashfunctional theory proposed byWarrington and colleagues to account for category-related object processing impairments in neurologicalpatients (Warrington amp Shallice 1984) In this theorythe relevant semantic content of physical entities isessentially reduced to two experiential attributes pro-cessed in two distinct brain networks Subsequentwork has generally recognized the limitations of asimple sensoryndashfunctional dichotomy (Allport 1985Warrington amp McCarthy 1987) leading to investi-gation of an ever-expanding list of sensory motorand affective attributes of concepts and the neuralcorrelates of these attributes

Studies of modality-specific contributions to con-ceptual knowledge have usually focused on a singleattribute or small set of attributes Lynott andConnell collected normative ratings for a sample of423 concrete adjectives that describe object proper-ties (Lynott amp Connell 2009) and 400 nouns (Lynottamp Connell 2013) on strength of association witheach of the five primary sensory modalities Partici-pants were asked to rate how strongly they experi-enced a particular concept by hearing tastingfeeling through touch smelling and seeing Gainottiet al (Gainotti Ciaraffa Silveri amp Marra 2009 GainottiSpinelli Scaricamazza amp Marra 2013) expanded onthis set by dividing the visual domain into threedimensions (shape colour and motion) and addingmanipulation experience as a motor dimensionRatings were obtained on 49 animal plant and arte-fact concepts revealing highly significant differencesbetween categories in the perceived relevance ofthese sensoryndashmotor dimensions to each conceptUsing a similar set of dimensions Hoffman andLambon Ralph (Hoffman amp Lambon Ralph 2013)obtained strength of association ratings for a set of160 object nouns Participants were asked ldquoHowmuch do you associate this item with a particular(colour visual form observed motion sound tactilesensation taste smell action)rdquo The resulting attri-bute ratings predicted lexicalndashsemantic processingspeed more accurately than verbal feature-based rep-resentations (McRae Cree Seidenberg amp McNorgan2005) and they supported a novel conceptual distinc-tion between mechanical devices (eg vehicles) and

other non-living objects in that the former hadstrong sound and motion characteristics that madethem more similar to animals

Although prior work in this area has focused over-whelmingly on concrete object and action conceptsmore recent studies explore dimensions of experiencethat may be more important in the representation ofabstract concepts Several authors have emphasizedthe role of affective and social experiences in abstractconcept acquisition and embodiment (Borghi FluminiCimatti Marocco amp Scorolli 2011 Kousta ViglioccoVinson Andrews amp Del Campo 2011 ViglioccoMeteyard Andrews amp Kousta 2009 Wiemer-Hastingsamp Xu 2005 Zdrazilova amp Pexman 2013) Crutch et al(Crutch Williams Ridgway amp Borgenicht 2012)obtained ratings for 200 abstract and 200 concretenouns on the relatedness of each word to conceptsof time space quantity emotion polarity (positiveor negative) social interaction morality andthought in addition to the more physical dimensionsof sensation and action These representations suc-cessfully predicted performance by a severelyaphasic patient on an abstract noun comprehensiontask in which semantic similarity between target andfoil responses was manipulated whereas similaritymetrics (latent semantic analysis cosine similarity)based on patterns of word usage (Landauer ampDumais 1997) were not predictive (Crutch TrocheReilly amp Ridgway 2013) Troche et al (TrocheCrutch amp Reilly 2014) provide further analyses ofthese representations identifying three latent factors(labelled perceptual salience affective associationand magnitude) that accounted for 81 of the var-iance Abstract nouns were rated higher than concretenouns on the dimensions of emotion polarity socialinteraction morality and space

Building on this prior work our aim in the presentstudy was to develop a more comprehensive concep-tual representation based on known modalities ofneural information processing This more comprehen-sive representation would ideally capture aspects ofexperience that are central to the acquisition ofevent concepts as well as object concepts andabstract as well as concrete concepts The resultingrepresentation described in the following sectioncontains entries corresponding to specializedsensory and motor processes affective processessystems for processing spatial temporal and causalinformation social cognition processes and abstract

132 J R BINDER ET AL

cognitive operations We hope this approach stimu-lates further empirical and theoretical efforts towarda comprehensive brain-based semantic theory

Neural components of experience

The following section briefly describes the proposedcomponents of a brain-based semantic represen-tation which are listed in Tables 1 and 2 Each ofthese components is defined by an extensive bodyof physiological evidence that can only be hinted athere Examples are provided of how each componentapplies to a range of concepts Two fundamental prin-ciples guided selection of the components First weassume that all aspects of mental experience can con-tribute to concept acquisition and therefore conceptcomposition These ldquoaspects of mental experiencerdquoinclude not only sensory perceptions but also

affective responses to entities and situations experi-ence with performing motor actions perception ofspatial and temporal phenomena perception of caus-ality experience with social phenomena and experi-ence with internal cognitive phenomena and drivestates Second we focus on experiential phenomena

Table 1 Sensory and motor components organized by domainDomain Component Description (with reference to nouns)

Vision Vision something that you can easily seeVision Bright visually light or brightVision Dark visually darkVision Colour having a characteristic or defining colourVision Pattern having a characteristic or defining visual texture

or surface patternVision Large large in sizeVision Small small in sizeVision Motion showing a lot of visually observable movementVision Biomotion showing movement like that of a living thingVision Fast showing visible movement that is fastVision Slow showing visible movement that is slowVision Shape having a characteristic or defining visual shape or

formVision Complexity visually complexVision Face having a human or human-like faceVision Body having human or human-like body partsSomatic Touch something that you could easily recognize by

touchSomatic Temperature hot or cold to the touchSomatic Texture having a smooth or rough texture to the touchSomatic Weight light or heavy in weightSomatic Pain associated with pain or physical discomfortAudition Audition something that you can easily hearAudition Loud making a loud soundAudition Low having a low-pitched soundAudition High having a high-pitched soundAudition Sound having a characteristic or recognizable sound or

soundsAudition Music making a musical soundAudition Speech someone or something that talksGustation Taste having a characteristic or defining tasteOlfaction Smell having a characteristic or defining smell or smellsMotor Head associated with actions using the face mouth or

tongueMotor Upper limb associated with actions using the arm hand or

fingersMotor Lower limb associated with actions using the leg or footMotor Practice a physical object YOU have personal experience

using

Table 2 Spatial temporal causal social emotion drive andattention componentsDomain Name Description (with reference to nouns)

Spatial Landmark having a fixed location as on a mapSpatial Path showing changes in location along a

particular direction or pathSpatial Scene bringing to mind a particular setting or

physical locationSpatial Near often physically near to you (within easy

reach) in everyday lifeSpatial Toward associated with movement toward or into youSpatial Away associated with movement away from or out

of youSpatial Number associated with a specific number or amountTemporal Time an event or occurrence that occurs at a typical

or predictable timeTemporal Duration an event that has a predictable duration

whether short or longTemporal Long an event that lasts for a long period of timeTemporal Short an event that lasts for a short period of timeCausal Caused caused by some clear preceding event action

or situationCausal Consequential likely to have consequences (cause other

things to happen)Social Social an activity or event that involves an

interaction between peopleSocial Human having human or human-like intentions

plans or goalsSocial Communication a thing or action that people use to

communicateSocial Self related to your own view of yourself a part of

YOUR self-imageCognition Cognition a form of mental activity or a function of the

mindEmotion Benefit someone or something that could help or

benefit you or othersEmotion Harm someone or something that could cause harm

to you or othersEmotion Pleasant someone or something that you find pleasantEmotion Unpleasant someone or something that you find

unpleasantEmotion Happy someone or something that makes you feel

happyEmotion Sad someone or something that makes you feel

sadEmotion Angry someone or something that makes you feel

angryEmotion Disgusted someone or something that makes you feel

disgustedEmotion Fearful someone or something that makes you feel

afraidEmotion Surprised someone or something that makes you feel

surprisedDrive Drive someone or something that motivates you to

do somethingDrive Needs someone or something that would be hard

for you to live withoutAttention Attention someone or something that grabs your

attentionAttention Arousal someone or something that makes you feel

alert activated excited or keyed up ineither a positive or negative way

COGNITIVE NEUROPSYCHOLOGY 133

for which there are likely to be corresponding dis-tinguishable neural processors drawing on evidencefrom animal physiology brain imaging and neurologi-cal studies In particular we focus on macroscopicneural systems that can be distinguished with invivo human imaging methods We do not require orpropose that these neural processors are self-con-tained ldquomodulesrdquo or that they are highly localized inthe brain only that they process information that pri-marily represents a particular aspect of experience

A detailed listing of the queries used to elicit associ-ation ratings for each component is available for down-load (see link prior to the References section)

Visual components

Visual and other sensory components are listed inTable 1 Components of visual experience for whichthere are likely to be corresponding distinguishableneural processors include luminance size colourvisual texture visual shape visual motion and biologi-cal motion Within the visual shape perceptionnetwork are distinguishable networks that primarilyprocess faces human body parts and three-dimen-sional spaces

Luminance is a basic property of vision but is alsorelevant to such concepts as sun light gleam shineink night dark black and so on that are defined bybrightness or darkness Luminance is an example ofa scalar quantitymdashthat is one that represents a con-tinuous one-dimensional variable Linguistic represen-tation of scalars tends to focus on ends of thecontinuum (eg brightdark hotcold heavylightloudsoft etc) which raises a general question ofwhether or to what extent opposite ends of a scalar(eg bright and dark) are represented in distinguish-able neural processors Are there non-identicalneural networks that encode high and low luminanceThe answer depends to some degree on the spatialscale in question It is known that some neuronsrespond somewhat selectively to stimuli with highluminance while others respond more to stimuli withlow luminance (ie ON and OFF cells in retina andV1) resulting in two networks that are distinguishableat a microscopic scale If these and other luminance-tuned neurons intermingle within a single networkhowever the high- and low-luminance networksmight be indistinguishable at least at the macroscopicscale of in vivo imaging

In contrast the scalar quantity visual size is linked tolarge-scale retinotopic organization throughout thevisual cortical system and has been proposed as amajor determinant of medial-to-lateral organizationeven at the highest levels of the ventral visualstream (Hasson Levy Behrmann Hendler amp Malach2002) At these higher levels medial-to-lateral organiz-ation appears not to depend on the actual retinalextent of a given object exemplar which varies withdistance from the observer but rather on a ldquotypicalrdquoor generalized perceptual representation Images ofbuildings for example produce stronger activationin medial regions than do images of insects and theopposite pattern holds for lateral regions even whenthe images are the same physical size (Konkle ampOliva 2012) By extension the general theory that con-cepts are represented partly in perceptual systemswould predict similar medialndashlateral activation differ-ences in the ventral visual stream for concepts repre-senting objects that are reliably large or reliably small

There is now strong evidence not only for a fairlyspecialized colour perception network in the humanbrain (Bartels amp Zeki 2000 Beauchamp HaxbyJennings amp DeYoe 1999) but also for activation inor near this processor by concepts with colourcontent (Hsu Frankland amp Thompson-Schill 2012Hsu Kraemer Oliver Schlichting amp Thompson-Schill2011 Kellenbach Brett amp Patterson 2001 MartinHaxby Lalonde Wiggs amp Ungerleider 1995Simmons et al 2007) The question of whether thereare distinguishable neural processors activated by par-ticular colour concepts (eg red vs green vs blue) hasnot been studied but this appears unlikely Colour-sensitive neurons in the monkey inferior temporalcortex form a network of millimetre-sized ldquoglobsrdquowithin which there is evidence for spatial clusteringof neurons by colour preference (Conway amp Tsao2009) However the spatial scale of these clusters ison the order of 50ndash100 microm beyond the spatial resol-ution of current in vivo imaging methods More impor-tantly it is not clear that the spatial organization ofcolour concept representations should necessarilyreflect this perceptual organization Division of thevisible light frequency spectrum into colour conceptsvaries considerably across cultures (Berlin amp Kay1969) whereas the spatial organization of colour-sen-sitive neurons presumably does not As with the attri-bute of luminance we have assumed for these reasonsthat a single neural processor encodes the colour

134 J R BINDER ET AL

content of concepts regardless of the particular colourin question A concept like lemon that is reliably associ-ated with a particular colour is expected to activatethis neural processor to approximately the samedegree as a concept like coffee that is reliably associ-ated with a different colour

Visual motion perception involves a relativelyspecialized neural processor known as MT or V5 (Alb-right Desimone amp Gross 1984 Zeki 1974) Responsesin MT vary with motion velocity and degree of motioncoherence (Lingnau Ashida Wall amp Smith 2009 ReesFriston amp Koch 2000 Tootell et al 1995) Motion vel-ocity is relevant for our purposes as it is strongly rep-resented at a conceptual level For examplemovements may be relatively fast or slow in velocityand this scalar quantity is central to action conceptslike run dash zoom creep ooze walk and so on aswell as entity concepts like bullet jet hare snail tor-toise sloth and so on Movements may also have aspecific pattern or quality as illustrated for exampleby the large number of verbs that express manner ofmotion (turn bounce roll float spin twist etc) Weare not aware of any evidence for large-scale spatialorganization in the cortex according to type ormanner of motion therefore the proposed semanticrepresentation is designed to capture strength ofassociation with visually observable characteristicmotion in general regardless of specific type Theexception to this rule is the perception of biologicalmotion which has been linked with a neural processorthat is clearly distinct from MT located in the posteriorsuperior temporal sulcus (STS Grossman amp Blake2002 Pelphrey Morris Michelich Allison amp McCarthy2005) Biological motion contains higher order com-plexity that is characteristic of face and body partactions Perception of biological motion plays acentral role in understanding face and body gesturesand thus the affective and intentional state of otheragents (ie theory of mind Allison Puce amp McCarthy2000) Biological motion is thus one of several keyattributes distinguishing intentional agents (boy girlwoman man etc) from inanimate entities

Visual shape perception involves an extensiveregion of extrastriate cortex including a large lateralextrastriate area known as the lateral occipitalcomplex (Malach et al 1995) and an adjacent areaon the ventral occipitalndashtemporal surface known asVOT (Grill-Spector amp Malach 2004) In human func-tional magnetic resonance imaging (fMRI) studies

these areas respond more strongly to images ofobjects than to images in which shape contours ofthe objects have been disrupted (ldquoscrambledrdquoimages) similar to response patterns observed inprimate ventral temporal neurons (Tanaka SaitoFukada amp Moriya 1991) Shape information is gener-ally the most important attribute of concrete objectconcepts and distinguishes concrete concepts froma range of abstract (eg affective social and cogni-tive) entities and event concepts Variation in the sal-ience of visual shape also occurs within the domainof concrete entities Shape is typically not a definingfeature of substance nouns (metal plastic water) butit has relevance for some substance-like mass nouns(butter coffee rice) Concrete objects also vary inshape complexity Animals for example tend tohave more complex shapes than plants or tools (Snod-grass amp Vanderwart 1980) We hypothesize that bothshape salience and shape complexity determine thedegree of activation in shape processing regionsduring concept retrieval

More focal areas within or adjacent to these shapeprocessing regions show preferential responses tohuman faces (Kanwisher McDermott amp Chun 1997Schwarzlose Baker amp Kanwisher 2005) and bodyparts (Downing Jiang Shuman amp Kanwisher 2001)A specialized mechanism for face perception is con-sistent with the central role of face recognition insocial interactions Visual perception of body partsprobably plays a crucial role in understandingobserved actions and mapping these onto internalrepresentations of the body schema during actionlearning These processors would be differentially acti-vated by concepts pertaining to faces (nose mouthportrait) and body parts (hand leg finger) As a firstapproximation we propose that both of these neuralprocessors are activated to some degree by all con-cepts referring to human beings which by necessityinclude a face and other human body parts Activationof the face processor may be minimal in the case ofconcepts that represent general human types androles (boy man nurse etc) as these concepts do notinclude information about any specific face whereasconcepts referencing specific individuals (brotherfather Einstein) might include specific facial featureinformation and therefore activate the face processorto a greater degree

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior

COGNITIVE NEUROPSYCHOLOGY 135

parahippocampal region (Epstein amp Kanwisher 1998Epstein Parker amp Feiler 2007) Although this processoris typically studied using visual stimuli and is oftenconsidered a high-level visual area it more likely inte-grates visual proprioceptive and perhaps auditoryinputs all of which provide correlated informationabout three-dimensional space Further discussion ofthis component of experience is provided in thesection below on Spatial Components (Table 2)

Somatosensory components

Somatosensory networks of the cortex represent bodylocation (somatotopic representation) body positionand joint force (proprioception) pain surface charac-teristics of objects (texture and temperature) andobject shape (Friebel Eickhoff amp Lotze 2011Hendry Hsiao amp Bushnell 1999 Lamm Decety ampSinger 2011 Longo Azanon amp Haggard 2010McGlone amp Reilly 2010 Medina amp Coslett 2010Serino amp Haggard 2010) Somatotopy is an inherentlymulti-modal phenomenon because the body locationtouched by a particular object is typically the samebody location as that used during exploratory ormanipulative motor actions involving the object Con-ceptual and linguistic representations are more likelyto encode the action as the salient characteristic ofthe object rather than the tactile experience (we sayldquoI threw the ballrdquo to describe the event not ldquothe balltouched my handrdquo) therefore somatotopic knowledgewas encoded in the proposed representation usingaction-based rather than somatosensory attributes(see Motor Components)

Temperature tactile texture and pain wereincluded as components of the representation aswas a component referring to the salience of objectweight Weight is perceived mainly via proprioceptiverepresentations and is a salient attribute of objectswith which we interact Temperature and weight aretwo more examples of scalar entities that may ormay not have a macroscopic topography in thebrain (Hendry et al 1999 Mazzola Faillenot BarralMauguiere amp Peyron 2012) That is although thereis evidence that these types of somatosensory infor-mation are distinguishable from each other no evi-dence has yet been presented for a scalartopography that separates for example hot fromcold representations Tactile texture can refer to avariety of physical properties we operationalized this

concept as a continuum from smooth to rough andthus the texture component is another scalar entityFor all three of these scalars we elected to representthe entire continuum as a single component Thatis the temperature component is intended tocapture the salience of temperature to a conceptregardless of whether the associated temperature ishot or cold

The published literature on conceptual processingof somatosensory information has focused almostexclusively on impaired knowledge of body parts inneurological patients (Coslett Saffran amp Schwoebel2002 Kemmerer amp Tranel 2008 Laiacona AllamanoLorenzi amp Capitani 2006 Schwoebel amp Coslett2005) though no definitive localization has emergedfrom these studies A number of functional imagingstudies suggest an overlap between networksinvolved in real pain perception and those involvedin empathic pain perception (perception of pain inothers) and perception of ldquosocial painrdquo (Eisenberger2012 Lamm et al 2011) suggesting that retrieval ofpain concepts involves a simulation process To ourknowledge no studies have yet examined the concep-tual representation of temperature tactile texture orweight

Finally object shape is an attribute learned partlythrough haptic experiences (Miqueacutee et al 2008) butthe degree to which haptic shape is learned indepen-dently from visual shape is unclear Some objects areseen but not felt (ie very large and very distantobjects) but the converse is rarely true at least insighted individuals It was therefore decided that thevisual shape query would capture knowledge aboutshape in both modalities and that a separate queryregarding extent of personal manipulation experience(see Motor Components) would capture the impor-tance of haptic shape relative to visual shape

Auditory components

The auditory cortex includes low-level areas tuned tofrequency (tonotopy) amplitude and spatial location(Schreiner amp Winer 2007) as well as higher levelsshowing specialization for auditory ldquoobjectsrdquo such asenvironmental sounds (Leaver amp Rauschecker 2010)and speech sounds (Belin Zatorre Lafaille Ahad ampPike 2000 Liebenthal Binder Spitzer Possing ampMedler 2005) Corresponding auditory attributes ofthe proposed conceptual representation capture the

136 J R BINDER ET AL

degree to which a concept is associated with a low-pitched high-pitched loud (ie high amplitude)recognizable (ie having a characteristic auditoryform) or speech-like sound The attribute ldquolow inamplituderdquo was not included because it was con-sidered to be a salient attribute of relatively fewconceptsmdashthat is those that refer specifically to alow-amplitude variant (eg whisper) Concepts thatfocus on the absence of sound (eg silence quiet)are probably more accurately encoded as a nullvalue on the ldquoloudrdquo attribute as fMRI studies haveshown auditory regions where activity increases withsound intensity but not the converse (Roumlhl amp Uppen-kamp 2012)

A few studies of sound-related knowledge(Goldberg Perfetti amp Schneider 2006 Kellenbachet al 2001 Kiefer Sim Herrnberger Grothe ampHoenig 2008) reported activation in or near the pos-terior superior temporal sulcus (STS) an auditoryassociation region implicated in environmentalsound recognition (Leaver amp Rauschecker 2010 J WLewis et al 2004) Trumpp et al (Trumpp KlieseHoenig Haarmeier amp Kiefer 2013) described apatient with a circumscribed lesion centred on theleft posterior STS who displayed a specific deficit(slower response times and higher error rate) invisual recognition of sound-related words All ofthese studies examined general environmentalsound knowledge nothing is yet known regardingthe conceptual representation of specific types ofauditory attributes like frequency or amplitude

Localization of sounds in space is an importantaspect of auditory perception yet auditory perceptionof space has little or no representation in languageindependent of (multimodal) spatial concepts thereare no concepts with components like ldquomakessounds from the leftrdquo because spatial relationshipsbetween sound sources and listeners are almostnever fixed or central to meaning Like shape attri-butes of concepts spatial attributes and spatial con-cepts are learned through strongly correlatedmultimodal experiences involving visual auditorytactile and motor processes In the proposed semanticrepresentation model spatial attributes of conceptsare represented by modality-independent spatialcomponents (see Spatial Components)

A final salient component of human auditoryexperience is the domain of music Music perceptionwhich includes processing of melody harmony and

rhythm elements in sound appears to engage rela-tively specialized right lateralized networks some ofwhich may also process nonverbal (prosodic)elements in speech (Zatorre Belin amp Penhune2002) Many words refer explicitly to musical phenom-ena (sing song melody harmony rhythm etc) or toentities (instruments performers performances) thatproduce musical sounds Therefore the model pro-posed here includes a component to capture theexperience of processing music

Gustatory and olfactory components

Gustatory and olfactory experiences are each rep-resented by a single attribute that captures the rel-evance of each type of experience to the conceptThese sensory systems do not show clear large-scaleorganization along any dimension and neuronswithin each system often respond to a broad spec-trum of items Both gustatory and olfactory conceptshave been linked with specific early sensory andhigher associative neural processors (Goldberg et al2006 Gonzaacutelez et al 2006)

Motor components

The motor components of the proposed conceptualrepresentation (Table 1) capture the degree to whicha concept is associated with actions involving particu-lar body parts The neural representation of motoraction verbs has been extensively studied withmany studies showing action-specific activation of ahigh-level network that includes the supramarginalgyrus and posterior middle temporal gyrus (BinderDesai Conant amp Graves 2009 Desai Binder Conantamp Seidenberg 2009) There is also evidence for soma-totopic representation of action verb representation inprimary sensorimotor areas (Hauk Johnsrude ampPulvermuller 2004) though this claim is somewhatcontroversial (Postle McMahon Ashton Meredith ampde Zubicaray 2008) Object concepts associated withparticular actions also activate the action network(Binder et al 2009 Boronat et al 2005 Martin et al1995) and there is evidence for somatotopicallyspecific activation depending on which body part istypically used in the object interaction (CarotaMoseley amp Pulvermuumlller 2012) Following precedentin the neuroimaging literature (Carota et al 2012Hauk et al 2004 Tettamanti et al 2005) a three-

COGNITIVE NEUROPSYCHOLOGY 137

way partition into head-related (face mouth ortongue) upper limb (arm hand or fingers) andlower limb (leg or foot) actions was used to assesssomatotopic organization of action concepts A refer-ence to eye actions was felt to be confusingbecause all visual experiences involve the eyes inthis sense any visible object could be construed asldquoassociated with action involving the eyesrdquo

Finally the extent to which action attributes arerepresented in action networks may be partly deter-mined by personal experience (Calvo-Merino GlaserGrezes Passingham amp Haggard 2005 JaumlnckeKoeneke Hoppe Rominger amp Haumlnggi 2009) Mostpeople would rate ldquopianordquo as strongly associatedwith upper limb actions but most people also haveno personal experience with these specific actionsBased on prior research it is likely that the action rep-resentation of the concept ldquopianordquo is more extensivein the case of pianists Therefore a separate com-ponent (called ldquopracticerdquo) was created to capture thelevel of personal experience the rater has with per-forming the actions associated with the concept

Spatial components

Spatial and other more abstract components are listedin Table 2 Neural systems that encode aspects ofspatial experience have been investigated at manylevels (Andersen Snyder Bradley amp Xing 1997Burgess 2008 Kemmerer 2006 Kranjec CardilloSchmidt Lehet amp Chatterjee 2012 Woods et al2014) As mentioned above the acquisition of basicspatial concepts and knowledge of spatial attributesof concepts generally involves correlated multimodalinputs The most obvious spatial content in languageis the set of relations expressed by prepositionalphrases which have been a major focus of study in lin-guistics and semantic theory (Landau amp Jackendoff1993 Talmy 1983) Lesion correlation and neuroima-ging studies have implicated various parietal regionsparticularly the left inferior parietal lobe in the proces-sing of spatial prepositions and depicted locativerelations (Amorapanth Widick amp Chatterjee 2010Noordzij Neggers Ramsey amp Postma 2008 Tranel ampKemmerer 2004 Wu Waller amp Chatterjee 2007) Pre-positional phrases appear to express most if not all ofthe explicit relational spatial content that occurs inEnglish a semantic component targeting this type ofcontent therefore appears unnecessary (ie the set

of words with high values on this component wouldbe identical to the set of spatial prepositions) Thespatial components of the proposed representationmodel focus instead on other types of spatial infor-mation found in nouns verbs and adjectives

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior parahip-pocampal region (Epstein amp Kanwisher 1998 Epsteinet al 2007) The perception of three-dimensionalspace is based largely on visual input but also involvescorrelated proprioceptive information wheneverthree-dimensional space is experienced via move-ment of the body through space Spatial localizationin the auditory modality also probably contributes tothe experience of three-dimensional space Some con-cepts refer directly to interior spaces such as kitchenfoyer bathroom classroom and so on and seem toinclude information about general three-dimensionalsize and layout as do concepts about types of build-ings (library school hospital etc) More generallysuch concepts include the knowledge that they canbe physically entered navigated and exited whichmakes possible a range of conceptual combinations(entered the kitchen ran through the hall went out ofthe library etc) that are not possible for objectslacking large interior spaces (entered the chair) Thesalient property of concepts that determineswhether they activate a three-dimensional represen-tation of space is the degree to which they evoke aparticular ldquoscenerdquomdashthat is an array of objects in athree-dimensional space either by direct referenceor by association Space names (kitchen etc) andbuilding names (library etc) refer directly to three-dimensional scenes Many other object conceptsevoke mental representations of scenes by virtue ofthematic relations such as the evocation of kitchenby oven An object concept like chair would probablyfail to evoke such a representation as chairs are notlinked thematically with any particular scene Thusthere is a graded degree of mental representation ofthree-dimensional space evoked by different con-cepts which we hypothesize results in a correspond-ing variation in activation of the three-dimensionalspace neural processor

Large-scale (topographical) spatial information iscritical for navigation in the environment Entitiesthat are large and have a fixed location such as geo-logical formations and buildings can serve as land-marks within a mental map of the environment We

138 J R BINDER ET AL

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

semantic content in terms of experiential primitivesemphasizing in particular the role of space timeevent and causality phenomena in understandingpropositional content Other related early workincludes the sensoryndashfunctional theory proposed byWarrington and colleagues to account for category-related object processing impairments in neurologicalpatients (Warrington amp Shallice 1984) In this theorythe relevant semantic content of physical entities isessentially reduced to two experiential attributes pro-cessed in two distinct brain networks Subsequentwork has generally recognized the limitations of asimple sensoryndashfunctional dichotomy (Allport 1985Warrington amp McCarthy 1987) leading to investi-gation of an ever-expanding list of sensory motorand affective attributes of concepts and the neuralcorrelates of these attributes

Studies of modality-specific contributions to con-ceptual knowledge have usually focused on a singleattribute or small set of attributes Lynott andConnell collected normative ratings for a sample of423 concrete adjectives that describe object proper-ties (Lynott amp Connell 2009) and 400 nouns (Lynottamp Connell 2013) on strength of association witheach of the five primary sensory modalities Partici-pants were asked to rate how strongly they experi-enced a particular concept by hearing tastingfeeling through touch smelling and seeing Gainottiet al (Gainotti Ciaraffa Silveri amp Marra 2009 GainottiSpinelli Scaricamazza amp Marra 2013) expanded onthis set by dividing the visual domain into threedimensions (shape colour and motion) and addingmanipulation experience as a motor dimensionRatings were obtained on 49 animal plant and arte-fact concepts revealing highly significant differencesbetween categories in the perceived relevance ofthese sensoryndashmotor dimensions to each conceptUsing a similar set of dimensions Hoffman andLambon Ralph (Hoffman amp Lambon Ralph 2013)obtained strength of association ratings for a set of160 object nouns Participants were asked ldquoHowmuch do you associate this item with a particular(colour visual form observed motion sound tactilesensation taste smell action)rdquo The resulting attri-bute ratings predicted lexicalndashsemantic processingspeed more accurately than verbal feature-based rep-resentations (McRae Cree Seidenberg amp McNorgan2005) and they supported a novel conceptual distinc-tion between mechanical devices (eg vehicles) and

other non-living objects in that the former hadstrong sound and motion characteristics that madethem more similar to animals

Although prior work in this area has focused over-whelmingly on concrete object and action conceptsmore recent studies explore dimensions of experiencethat may be more important in the representation ofabstract concepts Several authors have emphasizedthe role of affective and social experiences in abstractconcept acquisition and embodiment (Borghi FluminiCimatti Marocco amp Scorolli 2011 Kousta ViglioccoVinson Andrews amp Del Campo 2011 ViglioccoMeteyard Andrews amp Kousta 2009 Wiemer-Hastingsamp Xu 2005 Zdrazilova amp Pexman 2013) Crutch et al(Crutch Williams Ridgway amp Borgenicht 2012)obtained ratings for 200 abstract and 200 concretenouns on the relatedness of each word to conceptsof time space quantity emotion polarity (positiveor negative) social interaction morality andthought in addition to the more physical dimensionsof sensation and action These representations suc-cessfully predicted performance by a severelyaphasic patient on an abstract noun comprehensiontask in which semantic similarity between target andfoil responses was manipulated whereas similaritymetrics (latent semantic analysis cosine similarity)based on patterns of word usage (Landauer ampDumais 1997) were not predictive (Crutch TrocheReilly amp Ridgway 2013) Troche et al (TrocheCrutch amp Reilly 2014) provide further analyses ofthese representations identifying three latent factors(labelled perceptual salience affective associationand magnitude) that accounted for 81 of the var-iance Abstract nouns were rated higher than concretenouns on the dimensions of emotion polarity socialinteraction morality and space

Building on this prior work our aim in the presentstudy was to develop a more comprehensive concep-tual representation based on known modalities ofneural information processing This more comprehen-sive representation would ideally capture aspects ofexperience that are central to the acquisition ofevent concepts as well as object concepts andabstract as well as concrete concepts The resultingrepresentation described in the following sectioncontains entries corresponding to specializedsensory and motor processes affective processessystems for processing spatial temporal and causalinformation social cognition processes and abstract

132 J R BINDER ET AL

cognitive operations We hope this approach stimu-lates further empirical and theoretical efforts towarda comprehensive brain-based semantic theory

Neural components of experience

The following section briefly describes the proposedcomponents of a brain-based semantic represen-tation which are listed in Tables 1 and 2 Each ofthese components is defined by an extensive bodyof physiological evidence that can only be hinted athere Examples are provided of how each componentapplies to a range of concepts Two fundamental prin-ciples guided selection of the components First weassume that all aspects of mental experience can con-tribute to concept acquisition and therefore conceptcomposition These ldquoaspects of mental experiencerdquoinclude not only sensory perceptions but also

affective responses to entities and situations experi-ence with performing motor actions perception ofspatial and temporal phenomena perception of caus-ality experience with social phenomena and experi-ence with internal cognitive phenomena and drivestates Second we focus on experiential phenomena

Table 1 Sensory and motor components organized by domainDomain Component Description (with reference to nouns)

Vision Vision something that you can easily seeVision Bright visually light or brightVision Dark visually darkVision Colour having a characteristic or defining colourVision Pattern having a characteristic or defining visual texture

or surface patternVision Large large in sizeVision Small small in sizeVision Motion showing a lot of visually observable movementVision Biomotion showing movement like that of a living thingVision Fast showing visible movement that is fastVision Slow showing visible movement that is slowVision Shape having a characteristic or defining visual shape or

formVision Complexity visually complexVision Face having a human or human-like faceVision Body having human or human-like body partsSomatic Touch something that you could easily recognize by

touchSomatic Temperature hot or cold to the touchSomatic Texture having a smooth or rough texture to the touchSomatic Weight light or heavy in weightSomatic Pain associated with pain or physical discomfortAudition Audition something that you can easily hearAudition Loud making a loud soundAudition Low having a low-pitched soundAudition High having a high-pitched soundAudition Sound having a characteristic or recognizable sound or

soundsAudition Music making a musical soundAudition Speech someone or something that talksGustation Taste having a characteristic or defining tasteOlfaction Smell having a characteristic or defining smell or smellsMotor Head associated with actions using the face mouth or

tongueMotor Upper limb associated with actions using the arm hand or

fingersMotor Lower limb associated with actions using the leg or footMotor Practice a physical object YOU have personal experience

using

Table 2 Spatial temporal causal social emotion drive andattention componentsDomain Name Description (with reference to nouns)

Spatial Landmark having a fixed location as on a mapSpatial Path showing changes in location along a

particular direction or pathSpatial Scene bringing to mind a particular setting or

physical locationSpatial Near often physically near to you (within easy

reach) in everyday lifeSpatial Toward associated with movement toward or into youSpatial Away associated with movement away from or out

of youSpatial Number associated with a specific number or amountTemporal Time an event or occurrence that occurs at a typical

or predictable timeTemporal Duration an event that has a predictable duration

whether short or longTemporal Long an event that lasts for a long period of timeTemporal Short an event that lasts for a short period of timeCausal Caused caused by some clear preceding event action

or situationCausal Consequential likely to have consequences (cause other

things to happen)Social Social an activity or event that involves an

interaction between peopleSocial Human having human or human-like intentions

plans or goalsSocial Communication a thing or action that people use to

communicateSocial Self related to your own view of yourself a part of

YOUR self-imageCognition Cognition a form of mental activity or a function of the

mindEmotion Benefit someone or something that could help or

benefit you or othersEmotion Harm someone or something that could cause harm

to you or othersEmotion Pleasant someone or something that you find pleasantEmotion Unpleasant someone or something that you find

unpleasantEmotion Happy someone or something that makes you feel

happyEmotion Sad someone or something that makes you feel

sadEmotion Angry someone or something that makes you feel

angryEmotion Disgusted someone or something that makes you feel

disgustedEmotion Fearful someone or something that makes you feel

afraidEmotion Surprised someone or something that makes you feel

surprisedDrive Drive someone or something that motivates you to

do somethingDrive Needs someone or something that would be hard

for you to live withoutAttention Attention someone or something that grabs your

attentionAttention Arousal someone or something that makes you feel

alert activated excited or keyed up ineither a positive or negative way

COGNITIVE NEUROPSYCHOLOGY 133

for which there are likely to be corresponding dis-tinguishable neural processors drawing on evidencefrom animal physiology brain imaging and neurologi-cal studies In particular we focus on macroscopicneural systems that can be distinguished with invivo human imaging methods We do not require orpropose that these neural processors are self-con-tained ldquomodulesrdquo or that they are highly localized inthe brain only that they process information that pri-marily represents a particular aspect of experience

A detailed listing of the queries used to elicit associ-ation ratings for each component is available for down-load (see link prior to the References section)

Visual components

Visual and other sensory components are listed inTable 1 Components of visual experience for whichthere are likely to be corresponding distinguishableneural processors include luminance size colourvisual texture visual shape visual motion and biologi-cal motion Within the visual shape perceptionnetwork are distinguishable networks that primarilyprocess faces human body parts and three-dimen-sional spaces

Luminance is a basic property of vision but is alsorelevant to such concepts as sun light gleam shineink night dark black and so on that are defined bybrightness or darkness Luminance is an example ofa scalar quantitymdashthat is one that represents a con-tinuous one-dimensional variable Linguistic represen-tation of scalars tends to focus on ends of thecontinuum (eg brightdark hotcold heavylightloudsoft etc) which raises a general question ofwhether or to what extent opposite ends of a scalar(eg bright and dark) are represented in distinguish-able neural processors Are there non-identicalneural networks that encode high and low luminanceThe answer depends to some degree on the spatialscale in question It is known that some neuronsrespond somewhat selectively to stimuli with highluminance while others respond more to stimuli withlow luminance (ie ON and OFF cells in retina andV1) resulting in two networks that are distinguishableat a microscopic scale If these and other luminance-tuned neurons intermingle within a single networkhowever the high- and low-luminance networksmight be indistinguishable at least at the macroscopicscale of in vivo imaging

In contrast the scalar quantity visual size is linked tolarge-scale retinotopic organization throughout thevisual cortical system and has been proposed as amajor determinant of medial-to-lateral organizationeven at the highest levels of the ventral visualstream (Hasson Levy Behrmann Hendler amp Malach2002) At these higher levels medial-to-lateral organiz-ation appears not to depend on the actual retinalextent of a given object exemplar which varies withdistance from the observer but rather on a ldquotypicalrdquoor generalized perceptual representation Images ofbuildings for example produce stronger activationin medial regions than do images of insects and theopposite pattern holds for lateral regions even whenthe images are the same physical size (Konkle ampOliva 2012) By extension the general theory that con-cepts are represented partly in perceptual systemswould predict similar medialndashlateral activation differ-ences in the ventral visual stream for concepts repre-senting objects that are reliably large or reliably small

There is now strong evidence not only for a fairlyspecialized colour perception network in the humanbrain (Bartels amp Zeki 2000 Beauchamp HaxbyJennings amp DeYoe 1999) but also for activation inor near this processor by concepts with colourcontent (Hsu Frankland amp Thompson-Schill 2012Hsu Kraemer Oliver Schlichting amp Thompson-Schill2011 Kellenbach Brett amp Patterson 2001 MartinHaxby Lalonde Wiggs amp Ungerleider 1995Simmons et al 2007) The question of whether thereare distinguishable neural processors activated by par-ticular colour concepts (eg red vs green vs blue) hasnot been studied but this appears unlikely Colour-sensitive neurons in the monkey inferior temporalcortex form a network of millimetre-sized ldquoglobsrdquowithin which there is evidence for spatial clusteringof neurons by colour preference (Conway amp Tsao2009) However the spatial scale of these clusters ison the order of 50ndash100 microm beyond the spatial resol-ution of current in vivo imaging methods More impor-tantly it is not clear that the spatial organization ofcolour concept representations should necessarilyreflect this perceptual organization Division of thevisible light frequency spectrum into colour conceptsvaries considerably across cultures (Berlin amp Kay1969) whereas the spatial organization of colour-sen-sitive neurons presumably does not As with the attri-bute of luminance we have assumed for these reasonsthat a single neural processor encodes the colour

134 J R BINDER ET AL

content of concepts regardless of the particular colourin question A concept like lemon that is reliably associ-ated with a particular colour is expected to activatethis neural processor to approximately the samedegree as a concept like coffee that is reliably associ-ated with a different colour

Visual motion perception involves a relativelyspecialized neural processor known as MT or V5 (Alb-right Desimone amp Gross 1984 Zeki 1974) Responsesin MT vary with motion velocity and degree of motioncoherence (Lingnau Ashida Wall amp Smith 2009 ReesFriston amp Koch 2000 Tootell et al 1995) Motion vel-ocity is relevant for our purposes as it is strongly rep-resented at a conceptual level For examplemovements may be relatively fast or slow in velocityand this scalar quantity is central to action conceptslike run dash zoom creep ooze walk and so on aswell as entity concepts like bullet jet hare snail tor-toise sloth and so on Movements may also have aspecific pattern or quality as illustrated for exampleby the large number of verbs that express manner ofmotion (turn bounce roll float spin twist etc) Weare not aware of any evidence for large-scale spatialorganization in the cortex according to type ormanner of motion therefore the proposed semanticrepresentation is designed to capture strength ofassociation with visually observable characteristicmotion in general regardless of specific type Theexception to this rule is the perception of biologicalmotion which has been linked with a neural processorthat is clearly distinct from MT located in the posteriorsuperior temporal sulcus (STS Grossman amp Blake2002 Pelphrey Morris Michelich Allison amp McCarthy2005) Biological motion contains higher order com-plexity that is characteristic of face and body partactions Perception of biological motion plays acentral role in understanding face and body gesturesand thus the affective and intentional state of otheragents (ie theory of mind Allison Puce amp McCarthy2000) Biological motion is thus one of several keyattributes distinguishing intentional agents (boy girlwoman man etc) from inanimate entities

Visual shape perception involves an extensiveregion of extrastriate cortex including a large lateralextrastriate area known as the lateral occipitalcomplex (Malach et al 1995) and an adjacent areaon the ventral occipitalndashtemporal surface known asVOT (Grill-Spector amp Malach 2004) In human func-tional magnetic resonance imaging (fMRI) studies

these areas respond more strongly to images ofobjects than to images in which shape contours ofthe objects have been disrupted (ldquoscrambledrdquoimages) similar to response patterns observed inprimate ventral temporal neurons (Tanaka SaitoFukada amp Moriya 1991) Shape information is gener-ally the most important attribute of concrete objectconcepts and distinguishes concrete concepts froma range of abstract (eg affective social and cogni-tive) entities and event concepts Variation in the sal-ience of visual shape also occurs within the domainof concrete entities Shape is typically not a definingfeature of substance nouns (metal plastic water) butit has relevance for some substance-like mass nouns(butter coffee rice) Concrete objects also vary inshape complexity Animals for example tend tohave more complex shapes than plants or tools (Snod-grass amp Vanderwart 1980) We hypothesize that bothshape salience and shape complexity determine thedegree of activation in shape processing regionsduring concept retrieval

More focal areas within or adjacent to these shapeprocessing regions show preferential responses tohuman faces (Kanwisher McDermott amp Chun 1997Schwarzlose Baker amp Kanwisher 2005) and bodyparts (Downing Jiang Shuman amp Kanwisher 2001)A specialized mechanism for face perception is con-sistent with the central role of face recognition insocial interactions Visual perception of body partsprobably plays a crucial role in understandingobserved actions and mapping these onto internalrepresentations of the body schema during actionlearning These processors would be differentially acti-vated by concepts pertaining to faces (nose mouthportrait) and body parts (hand leg finger) As a firstapproximation we propose that both of these neuralprocessors are activated to some degree by all con-cepts referring to human beings which by necessityinclude a face and other human body parts Activationof the face processor may be minimal in the case ofconcepts that represent general human types androles (boy man nurse etc) as these concepts do notinclude information about any specific face whereasconcepts referencing specific individuals (brotherfather Einstein) might include specific facial featureinformation and therefore activate the face processorto a greater degree

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior

COGNITIVE NEUROPSYCHOLOGY 135

parahippocampal region (Epstein amp Kanwisher 1998Epstein Parker amp Feiler 2007) Although this processoris typically studied using visual stimuli and is oftenconsidered a high-level visual area it more likely inte-grates visual proprioceptive and perhaps auditoryinputs all of which provide correlated informationabout three-dimensional space Further discussion ofthis component of experience is provided in thesection below on Spatial Components (Table 2)

Somatosensory components

Somatosensory networks of the cortex represent bodylocation (somatotopic representation) body positionand joint force (proprioception) pain surface charac-teristics of objects (texture and temperature) andobject shape (Friebel Eickhoff amp Lotze 2011Hendry Hsiao amp Bushnell 1999 Lamm Decety ampSinger 2011 Longo Azanon amp Haggard 2010McGlone amp Reilly 2010 Medina amp Coslett 2010Serino amp Haggard 2010) Somatotopy is an inherentlymulti-modal phenomenon because the body locationtouched by a particular object is typically the samebody location as that used during exploratory ormanipulative motor actions involving the object Con-ceptual and linguistic representations are more likelyto encode the action as the salient characteristic ofthe object rather than the tactile experience (we sayldquoI threw the ballrdquo to describe the event not ldquothe balltouched my handrdquo) therefore somatotopic knowledgewas encoded in the proposed representation usingaction-based rather than somatosensory attributes(see Motor Components)

Temperature tactile texture and pain wereincluded as components of the representation aswas a component referring to the salience of objectweight Weight is perceived mainly via proprioceptiverepresentations and is a salient attribute of objectswith which we interact Temperature and weight aretwo more examples of scalar entities that may ormay not have a macroscopic topography in thebrain (Hendry et al 1999 Mazzola Faillenot BarralMauguiere amp Peyron 2012) That is although thereis evidence that these types of somatosensory infor-mation are distinguishable from each other no evi-dence has yet been presented for a scalartopography that separates for example hot fromcold representations Tactile texture can refer to avariety of physical properties we operationalized this

concept as a continuum from smooth to rough andthus the texture component is another scalar entityFor all three of these scalars we elected to representthe entire continuum as a single component Thatis the temperature component is intended tocapture the salience of temperature to a conceptregardless of whether the associated temperature ishot or cold

The published literature on conceptual processingof somatosensory information has focused almostexclusively on impaired knowledge of body parts inneurological patients (Coslett Saffran amp Schwoebel2002 Kemmerer amp Tranel 2008 Laiacona AllamanoLorenzi amp Capitani 2006 Schwoebel amp Coslett2005) though no definitive localization has emergedfrom these studies A number of functional imagingstudies suggest an overlap between networksinvolved in real pain perception and those involvedin empathic pain perception (perception of pain inothers) and perception of ldquosocial painrdquo (Eisenberger2012 Lamm et al 2011) suggesting that retrieval ofpain concepts involves a simulation process To ourknowledge no studies have yet examined the concep-tual representation of temperature tactile texture orweight

Finally object shape is an attribute learned partlythrough haptic experiences (Miqueacutee et al 2008) butthe degree to which haptic shape is learned indepen-dently from visual shape is unclear Some objects areseen but not felt (ie very large and very distantobjects) but the converse is rarely true at least insighted individuals It was therefore decided that thevisual shape query would capture knowledge aboutshape in both modalities and that a separate queryregarding extent of personal manipulation experience(see Motor Components) would capture the impor-tance of haptic shape relative to visual shape

Auditory components

The auditory cortex includes low-level areas tuned tofrequency (tonotopy) amplitude and spatial location(Schreiner amp Winer 2007) as well as higher levelsshowing specialization for auditory ldquoobjectsrdquo such asenvironmental sounds (Leaver amp Rauschecker 2010)and speech sounds (Belin Zatorre Lafaille Ahad ampPike 2000 Liebenthal Binder Spitzer Possing ampMedler 2005) Corresponding auditory attributes ofthe proposed conceptual representation capture the

136 J R BINDER ET AL

degree to which a concept is associated with a low-pitched high-pitched loud (ie high amplitude)recognizable (ie having a characteristic auditoryform) or speech-like sound The attribute ldquolow inamplituderdquo was not included because it was con-sidered to be a salient attribute of relatively fewconceptsmdashthat is those that refer specifically to alow-amplitude variant (eg whisper) Concepts thatfocus on the absence of sound (eg silence quiet)are probably more accurately encoded as a nullvalue on the ldquoloudrdquo attribute as fMRI studies haveshown auditory regions where activity increases withsound intensity but not the converse (Roumlhl amp Uppen-kamp 2012)

A few studies of sound-related knowledge(Goldberg Perfetti amp Schneider 2006 Kellenbachet al 2001 Kiefer Sim Herrnberger Grothe ampHoenig 2008) reported activation in or near the pos-terior superior temporal sulcus (STS) an auditoryassociation region implicated in environmentalsound recognition (Leaver amp Rauschecker 2010 J WLewis et al 2004) Trumpp et al (Trumpp KlieseHoenig Haarmeier amp Kiefer 2013) described apatient with a circumscribed lesion centred on theleft posterior STS who displayed a specific deficit(slower response times and higher error rate) invisual recognition of sound-related words All ofthese studies examined general environmentalsound knowledge nothing is yet known regardingthe conceptual representation of specific types ofauditory attributes like frequency or amplitude

Localization of sounds in space is an importantaspect of auditory perception yet auditory perceptionof space has little or no representation in languageindependent of (multimodal) spatial concepts thereare no concepts with components like ldquomakessounds from the leftrdquo because spatial relationshipsbetween sound sources and listeners are almostnever fixed or central to meaning Like shape attri-butes of concepts spatial attributes and spatial con-cepts are learned through strongly correlatedmultimodal experiences involving visual auditorytactile and motor processes In the proposed semanticrepresentation model spatial attributes of conceptsare represented by modality-independent spatialcomponents (see Spatial Components)

A final salient component of human auditoryexperience is the domain of music Music perceptionwhich includes processing of melody harmony and

rhythm elements in sound appears to engage rela-tively specialized right lateralized networks some ofwhich may also process nonverbal (prosodic)elements in speech (Zatorre Belin amp Penhune2002) Many words refer explicitly to musical phenom-ena (sing song melody harmony rhythm etc) or toentities (instruments performers performances) thatproduce musical sounds Therefore the model pro-posed here includes a component to capture theexperience of processing music

Gustatory and olfactory components

Gustatory and olfactory experiences are each rep-resented by a single attribute that captures the rel-evance of each type of experience to the conceptThese sensory systems do not show clear large-scaleorganization along any dimension and neuronswithin each system often respond to a broad spec-trum of items Both gustatory and olfactory conceptshave been linked with specific early sensory andhigher associative neural processors (Goldberg et al2006 Gonzaacutelez et al 2006)

Motor components

The motor components of the proposed conceptualrepresentation (Table 1) capture the degree to whicha concept is associated with actions involving particu-lar body parts The neural representation of motoraction verbs has been extensively studied withmany studies showing action-specific activation of ahigh-level network that includes the supramarginalgyrus and posterior middle temporal gyrus (BinderDesai Conant amp Graves 2009 Desai Binder Conantamp Seidenberg 2009) There is also evidence for soma-totopic representation of action verb representation inprimary sensorimotor areas (Hauk Johnsrude ampPulvermuller 2004) though this claim is somewhatcontroversial (Postle McMahon Ashton Meredith ampde Zubicaray 2008) Object concepts associated withparticular actions also activate the action network(Binder et al 2009 Boronat et al 2005 Martin et al1995) and there is evidence for somatotopicallyspecific activation depending on which body part istypically used in the object interaction (CarotaMoseley amp Pulvermuumlller 2012) Following precedentin the neuroimaging literature (Carota et al 2012Hauk et al 2004 Tettamanti et al 2005) a three-

COGNITIVE NEUROPSYCHOLOGY 137

way partition into head-related (face mouth ortongue) upper limb (arm hand or fingers) andlower limb (leg or foot) actions was used to assesssomatotopic organization of action concepts A refer-ence to eye actions was felt to be confusingbecause all visual experiences involve the eyes inthis sense any visible object could be construed asldquoassociated with action involving the eyesrdquo

Finally the extent to which action attributes arerepresented in action networks may be partly deter-mined by personal experience (Calvo-Merino GlaserGrezes Passingham amp Haggard 2005 JaumlnckeKoeneke Hoppe Rominger amp Haumlnggi 2009) Mostpeople would rate ldquopianordquo as strongly associatedwith upper limb actions but most people also haveno personal experience with these specific actionsBased on prior research it is likely that the action rep-resentation of the concept ldquopianordquo is more extensivein the case of pianists Therefore a separate com-ponent (called ldquopracticerdquo) was created to capture thelevel of personal experience the rater has with per-forming the actions associated with the concept

Spatial components

Spatial and other more abstract components are listedin Table 2 Neural systems that encode aspects ofspatial experience have been investigated at manylevels (Andersen Snyder Bradley amp Xing 1997Burgess 2008 Kemmerer 2006 Kranjec CardilloSchmidt Lehet amp Chatterjee 2012 Woods et al2014) As mentioned above the acquisition of basicspatial concepts and knowledge of spatial attributesof concepts generally involves correlated multimodalinputs The most obvious spatial content in languageis the set of relations expressed by prepositionalphrases which have been a major focus of study in lin-guistics and semantic theory (Landau amp Jackendoff1993 Talmy 1983) Lesion correlation and neuroima-ging studies have implicated various parietal regionsparticularly the left inferior parietal lobe in the proces-sing of spatial prepositions and depicted locativerelations (Amorapanth Widick amp Chatterjee 2010Noordzij Neggers Ramsey amp Postma 2008 Tranel ampKemmerer 2004 Wu Waller amp Chatterjee 2007) Pre-positional phrases appear to express most if not all ofthe explicit relational spatial content that occurs inEnglish a semantic component targeting this type ofcontent therefore appears unnecessary (ie the set

of words with high values on this component wouldbe identical to the set of spatial prepositions) Thespatial components of the proposed representationmodel focus instead on other types of spatial infor-mation found in nouns verbs and adjectives

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior parahip-pocampal region (Epstein amp Kanwisher 1998 Epsteinet al 2007) The perception of three-dimensionalspace is based largely on visual input but also involvescorrelated proprioceptive information wheneverthree-dimensional space is experienced via move-ment of the body through space Spatial localizationin the auditory modality also probably contributes tothe experience of three-dimensional space Some con-cepts refer directly to interior spaces such as kitchenfoyer bathroom classroom and so on and seem toinclude information about general three-dimensionalsize and layout as do concepts about types of build-ings (library school hospital etc) More generallysuch concepts include the knowledge that they canbe physically entered navigated and exited whichmakes possible a range of conceptual combinations(entered the kitchen ran through the hall went out ofthe library etc) that are not possible for objectslacking large interior spaces (entered the chair) Thesalient property of concepts that determineswhether they activate a three-dimensional represen-tation of space is the degree to which they evoke aparticular ldquoscenerdquomdashthat is an array of objects in athree-dimensional space either by direct referenceor by association Space names (kitchen etc) andbuilding names (library etc) refer directly to three-dimensional scenes Many other object conceptsevoke mental representations of scenes by virtue ofthematic relations such as the evocation of kitchenby oven An object concept like chair would probablyfail to evoke such a representation as chairs are notlinked thematically with any particular scene Thusthere is a graded degree of mental representation ofthree-dimensional space evoked by different con-cepts which we hypothesize results in a correspond-ing variation in activation of the three-dimensionalspace neural processor

Large-scale (topographical) spatial information iscritical for navigation in the environment Entitiesthat are large and have a fixed location such as geo-logical formations and buildings can serve as land-marks within a mental map of the environment We

138 J R BINDER ET AL

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

cognitive operations We hope this approach stimu-lates further empirical and theoretical efforts towarda comprehensive brain-based semantic theory

Neural components of experience

The following section briefly describes the proposedcomponents of a brain-based semantic represen-tation which are listed in Tables 1 and 2 Each ofthese components is defined by an extensive bodyof physiological evidence that can only be hinted athere Examples are provided of how each componentapplies to a range of concepts Two fundamental prin-ciples guided selection of the components First weassume that all aspects of mental experience can con-tribute to concept acquisition and therefore conceptcomposition These ldquoaspects of mental experiencerdquoinclude not only sensory perceptions but also

affective responses to entities and situations experi-ence with performing motor actions perception ofspatial and temporal phenomena perception of caus-ality experience with social phenomena and experi-ence with internal cognitive phenomena and drivestates Second we focus on experiential phenomena

Table 1 Sensory and motor components organized by domainDomain Component Description (with reference to nouns)

Vision Vision something that you can easily seeVision Bright visually light or brightVision Dark visually darkVision Colour having a characteristic or defining colourVision Pattern having a characteristic or defining visual texture

or surface patternVision Large large in sizeVision Small small in sizeVision Motion showing a lot of visually observable movementVision Biomotion showing movement like that of a living thingVision Fast showing visible movement that is fastVision Slow showing visible movement that is slowVision Shape having a characteristic or defining visual shape or

formVision Complexity visually complexVision Face having a human or human-like faceVision Body having human or human-like body partsSomatic Touch something that you could easily recognize by

touchSomatic Temperature hot or cold to the touchSomatic Texture having a smooth or rough texture to the touchSomatic Weight light or heavy in weightSomatic Pain associated with pain or physical discomfortAudition Audition something that you can easily hearAudition Loud making a loud soundAudition Low having a low-pitched soundAudition High having a high-pitched soundAudition Sound having a characteristic or recognizable sound or

soundsAudition Music making a musical soundAudition Speech someone or something that talksGustation Taste having a characteristic or defining tasteOlfaction Smell having a characteristic or defining smell or smellsMotor Head associated with actions using the face mouth or

tongueMotor Upper limb associated with actions using the arm hand or

fingersMotor Lower limb associated with actions using the leg or footMotor Practice a physical object YOU have personal experience

using

Table 2 Spatial temporal causal social emotion drive andattention componentsDomain Name Description (with reference to nouns)

Spatial Landmark having a fixed location as on a mapSpatial Path showing changes in location along a

particular direction or pathSpatial Scene bringing to mind a particular setting or

physical locationSpatial Near often physically near to you (within easy

reach) in everyday lifeSpatial Toward associated with movement toward or into youSpatial Away associated with movement away from or out

of youSpatial Number associated with a specific number or amountTemporal Time an event or occurrence that occurs at a typical

or predictable timeTemporal Duration an event that has a predictable duration

whether short or longTemporal Long an event that lasts for a long period of timeTemporal Short an event that lasts for a short period of timeCausal Caused caused by some clear preceding event action

or situationCausal Consequential likely to have consequences (cause other

things to happen)Social Social an activity or event that involves an

interaction between peopleSocial Human having human or human-like intentions

plans or goalsSocial Communication a thing or action that people use to

communicateSocial Self related to your own view of yourself a part of

YOUR self-imageCognition Cognition a form of mental activity or a function of the

mindEmotion Benefit someone or something that could help or

benefit you or othersEmotion Harm someone or something that could cause harm

to you or othersEmotion Pleasant someone or something that you find pleasantEmotion Unpleasant someone or something that you find

unpleasantEmotion Happy someone or something that makes you feel

happyEmotion Sad someone or something that makes you feel

sadEmotion Angry someone or something that makes you feel

angryEmotion Disgusted someone or something that makes you feel

disgustedEmotion Fearful someone or something that makes you feel

afraidEmotion Surprised someone or something that makes you feel

surprisedDrive Drive someone or something that motivates you to

do somethingDrive Needs someone or something that would be hard

for you to live withoutAttention Attention someone or something that grabs your

attentionAttention Arousal someone or something that makes you feel

alert activated excited or keyed up ineither a positive or negative way

COGNITIVE NEUROPSYCHOLOGY 133

for which there are likely to be corresponding dis-tinguishable neural processors drawing on evidencefrom animal physiology brain imaging and neurologi-cal studies In particular we focus on macroscopicneural systems that can be distinguished with invivo human imaging methods We do not require orpropose that these neural processors are self-con-tained ldquomodulesrdquo or that they are highly localized inthe brain only that they process information that pri-marily represents a particular aspect of experience

A detailed listing of the queries used to elicit associ-ation ratings for each component is available for down-load (see link prior to the References section)

Visual components

Visual and other sensory components are listed inTable 1 Components of visual experience for whichthere are likely to be corresponding distinguishableneural processors include luminance size colourvisual texture visual shape visual motion and biologi-cal motion Within the visual shape perceptionnetwork are distinguishable networks that primarilyprocess faces human body parts and three-dimen-sional spaces

Luminance is a basic property of vision but is alsorelevant to such concepts as sun light gleam shineink night dark black and so on that are defined bybrightness or darkness Luminance is an example ofa scalar quantitymdashthat is one that represents a con-tinuous one-dimensional variable Linguistic represen-tation of scalars tends to focus on ends of thecontinuum (eg brightdark hotcold heavylightloudsoft etc) which raises a general question ofwhether or to what extent opposite ends of a scalar(eg bright and dark) are represented in distinguish-able neural processors Are there non-identicalneural networks that encode high and low luminanceThe answer depends to some degree on the spatialscale in question It is known that some neuronsrespond somewhat selectively to stimuli with highluminance while others respond more to stimuli withlow luminance (ie ON and OFF cells in retina andV1) resulting in two networks that are distinguishableat a microscopic scale If these and other luminance-tuned neurons intermingle within a single networkhowever the high- and low-luminance networksmight be indistinguishable at least at the macroscopicscale of in vivo imaging

In contrast the scalar quantity visual size is linked tolarge-scale retinotopic organization throughout thevisual cortical system and has been proposed as amajor determinant of medial-to-lateral organizationeven at the highest levels of the ventral visualstream (Hasson Levy Behrmann Hendler amp Malach2002) At these higher levels medial-to-lateral organiz-ation appears not to depend on the actual retinalextent of a given object exemplar which varies withdistance from the observer but rather on a ldquotypicalrdquoor generalized perceptual representation Images ofbuildings for example produce stronger activationin medial regions than do images of insects and theopposite pattern holds for lateral regions even whenthe images are the same physical size (Konkle ampOliva 2012) By extension the general theory that con-cepts are represented partly in perceptual systemswould predict similar medialndashlateral activation differ-ences in the ventral visual stream for concepts repre-senting objects that are reliably large or reliably small

There is now strong evidence not only for a fairlyspecialized colour perception network in the humanbrain (Bartels amp Zeki 2000 Beauchamp HaxbyJennings amp DeYoe 1999) but also for activation inor near this processor by concepts with colourcontent (Hsu Frankland amp Thompson-Schill 2012Hsu Kraemer Oliver Schlichting amp Thompson-Schill2011 Kellenbach Brett amp Patterson 2001 MartinHaxby Lalonde Wiggs amp Ungerleider 1995Simmons et al 2007) The question of whether thereare distinguishable neural processors activated by par-ticular colour concepts (eg red vs green vs blue) hasnot been studied but this appears unlikely Colour-sensitive neurons in the monkey inferior temporalcortex form a network of millimetre-sized ldquoglobsrdquowithin which there is evidence for spatial clusteringof neurons by colour preference (Conway amp Tsao2009) However the spatial scale of these clusters ison the order of 50ndash100 microm beyond the spatial resol-ution of current in vivo imaging methods More impor-tantly it is not clear that the spatial organization ofcolour concept representations should necessarilyreflect this perceptual organization Division of thevisible light frequency spectrum into colour conceptsvaries considerably across cultures (Berlin amp Kay1969) whereas the spatial organization of colour-sen-sitive neurons presumably does not As with the attri-bute of luminance we have assumed for these reasonsthat a single neural processor encodes the colour

134 J R BINDER ET AL

content of concepts regardless of the particular colourin question A concept like lemon that is reliably associ-ated with a particular colour is expected to activatethis neural processor to approximately the samedegree as a concept like coffee that is reliably associ-ated with a different colour

Visual motion perception involves a relativelyspecialized neural processor known as MT or V5 (Alb-right Desimone amp Gross 1984 Zeki 1974) Responsesin MT vary with motion velocity and degree of motioncoherence (Lingnau Ashida Wall amp Smith 2009 ReesFriston amp Koch 2000 Tootell et al 1995) Motion vel-ocity is relevant for our purposes as it is strongly rep-resented at a conceptual level For examplemovements may be relatively fast or slow in velocityand this scalar quantity is central to action conceptslike run dash zoom creep ooze walk and so on aswell as entity concepts like bullet jet hare snail tor-toise sloth and so on Movements may also have aspecific pattern or quality as illustrated for exampleby the large number of verbs that express manner ofmotion (turn bounce roll float spin twist etc) Weare not aware of any evidence for large-scale spatialorganization in the cortex according to type ormanner of motion therefore the proposed semanticrepresentation is designed to capture strength ofassociation with visually observable characteristicmotion in general regardless of specific type Theexception to this rule is the perception of biologicalmotion which has been linked with a neural processorthat is clearly distinct from MT located in the posteriorsuperior temporal sulcus (STS Grossman amp Blake2002 Pelphrey Morris Michelich Allison amp McCarthy2005) Biological motion contains higher order com-plexity that is characteristic of face and body partactions Perception of biological motion plays acentral role in understanding face and body gesturesand thus the affective and intentional state of otheragents (ie theory of mind Allison Puce amp McCarthy2000) Biological motion is thus one of several keyattributes distinguishing intentional agents (boy girlwoman man etc) from inanimate entities

Visual shape perception involves an extensiveregion of extrastriate cortex including a large lateralextrastriate area known as the lateral occipitalcomplex (Malach et al 1995) and an adjacent areaon the ventral occipitalndashtemporal surface known asVOT (Grill-Spector amp Malach 2004) In human func-tional magnetic resonance imaging (fMRI) studies

these areas respond more strongly to images ofobjects than to images in which shape contours ofthe objects have been disrupted (ldquoscrambledrdquoimages) similar to response patterns observed inprimate ventral temporal neurons (Tanaka SaitoFukada amp Moriya 1991) Shape information is gener-ally the most important attribute of concrete objectconcepts and distinguishes concrete concepts froma range of abstract (eg affective social and cogni-tive) entities and event concepts Variation in the sal-ience of visual shape also occurs within the domainof concrete entities Shape is typically not a definingfeature of substance nouns (metal plastic water) butit has relevance for some substance-like mass nouns(butter coffee rice) Concrete objects also vary inshape complexity Animals for example tend tohave more complex shapes than plants or tools (Snod-grass amp Vanderwart 1980) We hypothesize that bothshape salience and shape complexity determine thedegree of activation in shape processing regionsduring concept retrieval

More focal areas within or adjacent to these shapeprocessing regions show preferential responses tohuman faces (Kanwisher McDermott amp Chun 1997Schwarzlose Baker amp Kanwisher 2005) and bodyparts (Downing Jiang Shuman amp Kanwisher 2001)A specialized mechanism for face perception is con-sistent with the central role of face recognition insocial interactions Visual perception of body partsprobably plays a crucial role in understandingobserved actions and mapping these onto internalrepresentations of the body schema during actionlearning These processors would be differentially acti-vated by concepts pertaining to faces (nose mouthportrait) and body parts (hand leg finger) As a firstapproximation we propose that both of these neuralprocessors are activated to some degree by all con-cepts referring to human beings which by necessityinclude a face and other human body parts Activationof the face processor may be minimal in the case ofconcepts that represent general human types androles (boy man nurse etc) as these concepts do notinclude information about any specific face whereasconcepts referencing specific individuals (brotherfather Einstein) might include specific facial featureinformation and therefore activate the face processorto a greater degree

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior

COGNITIVE NEUROPSYCHOLOGY 135

parahippocampal region (Epstein amp Kanwisher 1998Epstein Parker amp Feiler 2007) Although this processoris typically studied using visual stimuli and is oftenconsidered a high-level visual area it more likely inte-grates visual proprioceptive and perhaps auditoryinputs all of which provide correlated informationabout three-dimensional space Further discussion ofthis component of experience is provided in thesection below on Spatial Components (Table 2)

Somatosensory components

Somatosensory networks of the cortex represent bodylocation (somatotopic representation) body positionand joint force (proprioception) pain surface charac-teristics of objects (texture and temperature) andobject shape (Friebel Eickhoff amp Lotze 2011Hendry Hsiao amp Bushnell 1999 Lamm Decety ampSinger 2011 Longo Azanon amp Haggard 2010McGlone amp Reilly 2010 Medina amp Coslett 2010Serino amp Haggard 2010) Somatotopy is an inherentlymulti-modal phenomenon because the body locationtouched by a particular object is typically the samebody location as that used during exploratory ormanipulative motor actions involving the object Con-ceptual and linguistic representations are more likelyto encode the action as the salient characteristic ofthe object rather than the tactile experience (we sayldquoI threw the ballrdquo to describe the event not ldquothe balltouched my handrdquo) therefore somatotopic knowledgewas encoded in the proposed representation usingaction-based rather than somatosensory attributes(see Motor Components)

Temperature tactile texture and pain wereincluded as components of the representation aswas a component referring to the salience of objectweight Weight is perceived mainly via proprioceptiverepresentations and is a salient attribute of objectswith which we interact Temperature and weight aretwo more examples of scalar entities that may ormay not have a macroscopic topography in thebrain (Hendry et al 1999 Mazzola Faillenot BarralMauguiere amp Peyron 2012) That is although thereis evidence that these types of somatosensory infor-mation are distinguishable from each other no evi-dence has yet been presented for a scalartopography that separates for example hot fromcold representations Tactile texture can refer to avariety of physical properties we operationalized this

concept as a continuum from smooth to rough andthus the texture component is another scalar entityFor all three of these scalars we elected to representthe entire continuum as a single component Thatis the temperature component is intended tocapture the salience of temperature to a conceptregardless of whether the associated temperature ishot or cold

The published literature on conceptual processingof somatosensory information has focused almostexclusively on impaired knowledge of body parts inneurological patients (Coslett Saffran amp Schwoebel2002 Kemmerer amp Tranel 2008 Laiacona AllamanoLorenzi amp Capitani 2006 Schwoebel amp Coslett2005) though no definitive localization has emergedfrom these studies A number of functional imagingstudies suggest an overlap between networksinvolved in real pain perception and those involvedin empathic pain perception (perception of pain inothers) and perception of ldquosocial painrdquo (Eisenberger2012 Lamm et al 2011) suggesting that retrieval ofpain concepts involves a simulation process To ourknowledge no studies have yet examined the concep-tual representation of temperature tactile texture orweight

Finally object shape is an attribute learned partlythrough haptic experiences (Miqueacutee et al 2008) butthe degree to which haptic shape is learned indepen-dently from visual shape is unclear Some objects areseen but not felt (ie very large and very distantobjects) but the converse is rarely true at least insighted individuals It was therefore decided that thevisual shape query would capture knowledge aboutshape in both modalities and that a separate queryregarding extent of personal manipulation experience(see Motor Components) would capture the impor-tance of haptic shape relative to visual shape

Auditory components

The auditory cortex includes low-level areas tuned tofrequency (tonotopy) amplitude and spatial location(Schreiner amp Winer 2007) as well as higher levelsshowing specialization for auditory ldquoobjectsrdquo such asenvironmental sounds (Leaver amp Rauschecker 2010)and speech sounds (Belin Zatorre Lafaille Ahad ampPike 2000 Liebenthal Binder Spitzer Possing ampMedler 2005) Corresponding auditory attributes ofthe proposed conceptual representation capture the

136 J R BINDER ET AL

degree to which a concept is associated with a low-pitched high-pitched loud (ie high amplitude)recognizable (ie having a characteristic auditoryform) or speech-like sound The attribute ldquolow inamplituderdquo was not included because it was con-sidered to be a salient attribute of relatively fewconceptsmdashthat is those that refer specifically to alow-amplitude variant (eg whisper) Concepts thatfocus on the absence of sound (eg silence quiet)are probably more accurately encoded as a nullvalue on the ldquoloudrdquo attribute as fMRI studies haveshown auditory regions where activity increases withsound intensity but not the converse (Roumlhl amp Uppen-kamp 2012)

A few studies of sound-related knowledge(Goldberg Perfetti amp Schneider 2006 Kellenbachet al 2001 Kiefer Sim Herrnberger Grothe ampHoenig 2008) reported activation in or near the pos-terior superior temporal sulcus (STS) an auditoryassociation region implicated in environmentalsound recognition (Leaver amp Rauschecker 2010 J WLewis et al 2004) Trumpp et al (Trumpp KlieseHoenig Haarmeier amp Kiefer 2013) described apatient with a circumscribed lesion centred on theleft posterior STS who displayed a specific deficit(slower response times and higher error rate) invisual recognition of sound-related words All ofthese studies examined general environmentalsound knowledge nothing is yet known regardingthe conceptual representation of specific types ofauditory attributes like frequency or amplitude

Localization of sounds in space is an importantaspect of auditory perception yet auditory perceptionof space has little or no representation in languageindependent of (multimodal) spatial concepts thereare no concepts with components like ldquomakessounds from the leftrdquo because spatial relationshipsbetween sound sources and listeners are almostnever fixed or central to meaning Like shape attri-butes of concepts spatial attributes and spatial con-cepts are learned through strongly correlatedmultimodal experiences involving visual auditorytactile and motor processes In the proposed semanticrepresentation model spatial attributes of conceptsare represented by modality-independent spatialcomponents (see Spatial Components)

A final salient component of human auditoryexperience is the domain of music Music perceptionwhich includes processing of melody harmony and

rhythm elements in sound appears to engage rela-tively specialized right lateralized networks some ofwhich may also process nonverbal (prosodic)elements in speech (Zatorre Belin amp Penhune2002) Many words refer explicitly to musical phenom-ena (sing song melody harmony rhythm etc) or toentities (instruments performers performances) thatproduce musical sounds Therefore the model pro-posed here includes a component to capture theexperience of processing music

Gustatory and olfactory components

Gustatory and olfactory experiences are each rep-resented by a single attribute that captures the rel-evance of each type of experience to the conceptThese sensory systems do not show clear large-scaleorganization along any dimension and neuronswithin each system often respond to a broad spec-trum of items Both gustatory and olfactory conceptshave been linked with specific early sensory andhigher associative neural processors (Goldberg et al2006 Gonzaacutelez et al 2006)

Motor components

The motor components of the proposed conceptualrepresentation (Table 1) capture the degree to whicha concept is associated with actions involving particu-lar body parts The neural representation of motoraction verbs has been extensively studied withmany studies showing action-specific activation of ahigh-level network that includes the supramarginalgyrus and posterior middle temporal gyrus (BinderDesai Conant amp Graves 2009 Desai Binder Conantamp Seidenberg 2009) There is also evidence for soma-totopic representation of action verb representation inprimary sensorimotor areas (Hauk Johnsrude ampPulvermuller 2004) though this claim is somewhatcontroversial (Postle McMahon Ashton Meredith ampde Zubicaray 2008) Object concepts associated withparticular actions also activate the action network(Binder et al 2009 Boronat et al 2005 Martin et al1995) and there is evidence for somatotopicallyspecific activation depending on which body part istypically used in the object interaction (CarotaMoseley amp Pulvermuumlller 2012) Following precedentin the neuroimaging literature (Carota et al 2012Hauk et al 2004 Tettamanti et al 2005) a three-

COGNITIVE NEUROPSYCHOLOGY 137

way partition into head-related (face mouth ortongue) upper limb (arm hand or fingers) andlower limb (leg or foot) actions was used to assesssomatotopic organization of action concepts A refer-ence to eye actions was felt to be confusingbecause all visual experiences involve the eyes inthis sense any visible object could be construed asldquoassociated with action involving the eyesrdquo

Finally the extent to which action attributes arerepresented in action networks may be partly deter-mined by personal experience (Calvo-Merino GlaserGrezes Passingham amp Haggard 2005 JaumlnckeKoeneke Hoppe Rominger amp Haumlnggi 2009) Mostpeople would rate ldquopianordquo as strongly associatedwith upper limb actions but most people also haveno personal experience with these specific actionsBased on prior research it is likely that the action rep-resentation of the concept ldquopianordquo is more extensivein the case of pianists Therefore a separate com-ponent (called ldquopracticerdquo) was created to capture thelevel of personal experience the rater has with per-forming the actions associated with the concept

Spatial components

Spatial and other more abstract components are listedin Table 2 Neural systems that encode aspects ofspatial experience have been investigated at manylevels (Andersen Snyder Bradley amp Xing 1997Burgess 2008 Kemmerer 2006 Kranjec CardilloSchmidt Lehet amp Chatterjee 2012 Woods et al2014) As mentioned above the acquisition of basicspatial concepts and knowledge of spatial attributesof concepts generally involves correlated multimodalinputs The most obvious spatial content in languageis the set of relations expressed by prepositionalphrases which have been a major focus of study in lin-guistics and semantic theory (Landau amp Jackendoff1993 Talmy 1983) Lesion correlation and neuroima-ging studies have implicated various parietal regionsparticularly the left inferior parietal lobe in the proces-sing of spatial prepositions and depicted locativerelations (Amorapanth Widick amp Chatterjee 2010Noordzij Neggers Ramsey amp Postma 2008 Tranel ampKemmerer 2004 Wu Waller amp Chatterjee 2007) Pre-positional phrases appear to express most if not all ofthe explicit relational spatial content that occurs inEnglish a semantic component targeting this type ofcontent therefore appears unnecessary (ie the set

of words with high values on this component wouldbe identical to the set of spatial prepositions) Thespatial components of the proposed representationmodel focus instead on other types of spatial infor-mation found in nouns verbs and adjectives

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior parahip-pocampal region (Epstein amp Kanwisher 1998 Epsteinet al 2007) The perception of three-dimensionalspace is based largely on visual input but also involvescorrelated proprioceptive information wheneverthree-dimensional space is experienced via move-ment of the body through space Spatial localizationin the auditory modality also probably contributes tothe experience of three-dimensional space Some con-cepts refer directly to interior spaces such as kitchenfoyer bathroom classroom and so on and seem toinclude information about general three-dimensionalsize and layout as do concepts about types of build-ings (library school hospital etc) More generallysuch concepts include the knowledge that they canbe physically entered navigated and exited whichmakes possible a range of conceptual combinations(entered the kitchen ran through the hall went out ofthe library etc) that are not possible for objectslacking large interior spaces (entered the chair) Thesalient property of concepts that determineswhether they activate a three-dimensional represen-tation of space is the degree to which they evoke aparticular ldquoscenerdquomdashthat is an array of objects in athree-dimensional space either by direct referenceor by association Space names (kitchen etc) andbuilding names (library etc) refer directly to three-dimensional scenes Many other object conceptsevoke mental representations of scenes by virtue ofthematic relations such as the evocation of kitchenby oven An object concept like chair would probablyfail to evoke such a representation as chairs are notlinked thematically with any particular scene Thusthere is a graded degree of mental representation ofthree-dimensional space evoked by different con-cepts which we hypothesize results in a correspond-ing variation in activation of the three-dimensionalspace neural processor

Large-scale (topographical) spatial information iscritical for navigation in the environment Entitiesthat are large and have a fixed location such as geo-logical formations and buildings can serve as land-marks within a mental map of the environment We

138 J R BINDER ET AL

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

for which there are likely to be corresponding dis-tinguishable neural processors drawing on evidencefrom animal physiology brain imaging and neurologi-cal studies In particular we focus on macroscopicneural systems that can be distinguished with invivo human imaging methods We do not require orpropose that these neural processors are self-con-tained ldquomodulesrdquo or that they are highly localized inthe brain only that they process information that pri-marily represents a particular aspect of experience

A detailed listing of the queries used to elicit associ-ation ratings for each component is available for down-load (see link prior to the References section)

Visual components

Visual and other sensory components are listed inTable 1 Components of visual experience for whichthere are likely to be corresponding distinguishableneural processors include luminance size colourvisual texture visual shape visual motion and biologi-cal motion Within the visual shape perceptionnetwork are distinguishable networks that primarilyprocess faces human body parts and three-dimen-sional spaces

Luminance is a basic property of vision but is alsorelevant to such concepts as sun light gleam shineink night dark black and so on that are defined bybrightness or darkness Luminance is an example ofa scalar quantitymdashthat is one that represents a con-tinuous one-dimensional variable Linguistic represen-tation of scalars tends to focus on ends of thecontinuum (eg brightdark hotcold heavylightloudsoft etc) which raises a general question ofwhether or to what extent opposite ends of a scalar(eg bright and dark) are represented in distinguish-able neural processors Are there non-identicalneural networks that encode high and low luminanceThe answer depends to some degree on the spatialscale in question It is known that some neuronsrespond somewhat selectively to stimuli with highluminance while others respond more to stimuli withlow luminance (ie ON and OFF cells in retina andV1) resulting in two networks that are distinguishableat a microscopic scale If these and other luminance-tuned neurons intermingle within a single networkhowever the high- and low-luminance networksmight be indistinguishable at least at the macroscopicscale of in vivo imaging

In contrast the scalar quantity visual size is linked tolarge-scale retinotopic organization throughout thevisual cortical system and has been proposed as amajor determinant of medial-to-lateral organizationeven at the highest levels of the ventral visualstream (Hasson Levy Behrmann Hendler amp Malach2002) At these higher levels medial-to-lateral organiz-ation appears not to depend on the actual retinalextent of a given object exemplar which varies withdistance from the observer but rather on a ldquotypicalrdquoor generalized perceptual representation Images ofbuildings for example produce stronger activationin medial regions than do images of insects and theopposite pattern holds for lateral regions even whenthe images are the same physical size (Konkle ampOliva 2012) By extension the general theory that con-cepts are represented partly in perceptual systemswould predict similar medialndashlateral activation differ-ences in the ventral visual stream for concepts repre-senting objects that are reliably large or reliably small

There is now strong evidence not only for a fairlyspecialized colour perception network in the humanbrain (Bartels amp Zeki 2000 Beauchamp HaxbyJennings amp DeYoe 1999) but also for activation inor near this processor by concepts with colourcontent (Hsu Frankland amp Thompson-Schill 2012Hsu Kraemer Oliver Schlichting amp Thompson-Schill2011 Kellenbach Brett amp Patterson 2001 MartinHaxby Lalonde Wiggs amp Ungerleider 1995Simmons et al 2007) The question of whether thereare distinguishable neural processors activated by par-ticular colour concepts (eg red vs green vs blue) hasnot been studied but this appears unlikely Colour-sensitive neurons in the monkey inferior temporalcortex form a network of millimetre-sized ldquoglobsrdquowithin which there is evidence for spatial clusteringof neurons by colour preference (Conway amp Tsao2009) However the spatial scale of these clusters ison the order of 50ndash100 microm beyond the spatial resol-ution of current in vivo imaging methods More impor-tantly it is not clear that the spatial organization ofcolour concept representations should necessarilyreflect this perceptual organization Division of thevisible light frequency spectrum into colour conceptsvaries considerably across cultures (Berlin amp Kay1969) whereas the spatial organization of colour-sen-sitive neurons presumably does not As with the attri-bute of luminance we have assumed for these reasonsthat a single neural processor encodes the colour

134 J R BINDER ET AL

content of concepts regardless of the particular colourin question A concept like lemon that is reliably associ-ated with a particular colour is expected to activatethis neural processor to approximately the samedegree as a concept like coffee that is reliably associ-ated with a different colour

Visual motion perception involves a relativelyspecialized neural processor known as MT or V5 (Alb-right Desimone amp Gross 1984 Zeki 1974) Responsesin MT vary with motion velocity and degree of motioncoherence (Lingnau Ashida Wall amp Smith 2009 ReesFriston amp Koch 2000 Tootell et al 1995) Motion vel-ocity is relevant for our purposes as it is strongly rep-resented at a conceptual level For examplemovements may be relatively fast or slow in velocityand this scalar quantity is central to action conceptslike run dash zoom creep ooze walk and so on aswell as entity concepts like bullet jet hare snail tor-toise sloth and so on Movements may also have aspecific pattern or quality as illustrated for exampleby the large number of verbs that express manner ofmotion (turn bounce roll float spin twist etc) Weare not aware of any evidence for large-scale spatialorganization in the cortex according to type ormanner of motion therefore the proposed semanticrepresentation is designed to capture strength ofassociation with visually observable characteristicmotion in general regardless of specific type Theexception to this rule is the perception of biologicalmotion which has been linked with a neural processorthat is clearly distinct from MT located in the posteriorsuperior temporal sulcus (STS Grossman amp Blake2002 Pelphrey Morris Michelich Allison amp McCarthy2005) Biological motion contains higher order com-plexity that is characteristic of face and body partactions Perception of biological motion plays acentral role in understanding face and body gesturesand thus the affective and intentional state of otheragents (ie theory of mind Allison Puce amp McCarthy2000) Biological motion is thus one of several keyattributes distinguishing intentional agents (boy girlwoman man etc) from inanimate entities

Visual shape perception involves an extensiveregion of extrastriate cortex including a large lateralextrastriate area known as the lateral occipitalcomplex (Malach et al 1995) and an adjacent areaon the ventral occipitalndashtemporal surface known asVOT (Grill-Spector amp Malach 2004) In human func-tional magnetic resonance imaging (fMRI) studies

these areas respond more strongly to images ofobjects than to images in which shape contours ofthe objects have been disrupted (ldquoscrambledrdquoimages) similar to response patterns observed inprimate ventral temporal neurons (Tanaka SaitoFukada amp Moriya 1991) Shape information is gener-ally the most important attribute of concrete objectconcepts and distinguishes concrete concepts froma range of abstract (eg affective social and cogni-tive) entities and event concepts Variation in the sal-ience of visual shape also occurs within the domainof concrete entities Shape is typically not a definingfeature of substance nouns (metal plastic water) butit has relevance for some substance-like mass nouns(butter coffee rice) Concrete objects also vary inshape complexity Animals for example tend tohave more complex shapes than plants or tools (Snod-grass amp Vanderwart 1980) We hypothesize that bothshape salience and shape complexity determine thedegree of activation in shape processing regionsduring concept retrieval

More focal areas within or adjacent to these shapeprocessing regions show preferential responses tohuman faces (Kanwisher McDermott amp Chun 1997Schwarzlose Baker amp Kanwisher 2005) and bodyparts (Downing Jiang Shuman amp Kanwisher 2001)A specialized mechanism for face perception is con-sistent with the central role of face recognition insocial interactions Visual perception of body partsprobably plays a crucial role in understandingobserved actions and mapping these onto internalrepresentations of the body schema during actionlearning These processors would be differentially acti-vated by concepts pertaining to faces (nose mouthportrait) and body parts (hand leg finger) As a firstapproximation we propose that both of these neuralprocessors are activated to some degree by all con-cepts referring to human beings which by necessityinclude a face and other human body parts Activationof the face processor may be minimal in the case ofconcepts that represent general human types androles (boy man nurse etc) as these concepts do notinclude information about any specific face whereasconcepts referencing specific individuals (brotherfather Einstein) might include specific facial featureinformation and therefore activate the face processorto a greater degree

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior

COGNITIVE NEUROPSYCHOLOGY 135

parahippocampal region (Epstein amp Kanwisher 1998Epstein Parker amp Feiler 2007) Although this processoris typically studied using visual stimuli and is oftenconsidered a high-level visual area it more likely inte-grates visual proprioceptive and perhaps auditoryinputs all of which provide correlated informationabout three-dimensional space Further discussion ofthis component of experience is provided in thesection below on Spatial Components (Table 2)

Somatosensory components

Somatosensory networks of the cortex represent bodylocation (somatotopic representation) body positionand joint force (proprioception) pain surface charac-teristics of objects (texture and temperature) andobject shape (Friebel Eickhoff amp Lotze 2011Hendry Hsiao amp Bushnell 1999 Lamm Decety ampSinger 2011 Longo Azanon amp Haggard 2010McGlone amp Reilly 2010 Medina amp Coslett 2010Serino amp Haggard 2010) Somatotopy is an inherentlymulti-modal phenomenon because the body locationtouched by a particular object is typically the samebody location as that used during exploratory ormanipulative motor actions involving the object Con-ceptual and linguistic representations are more likelyto encode the action as the salient characteristic ofthe object rather than the tactile experience (we sayldquoI threw the ballrdquo to describe the event not ldquothe balltouched my handrdquo) therefore somatotopic knowledgewas encoded in the proposed representation usingaction-based rather than somatosensory attributes(see Motor Components)

Temperature tactile texture and pain wereincluded as components of the representation aswas a component referring to the salience of objectweight Weight is perceived mainly via proprioceptiverepresentations and is a salient attribute of objectswith which we interact Temperature and weight aretwo more examples of scalar entities that may ormay not have a macroscopic topography in thebrain (Hendry et al 1999 Mazzola Faillenot BarralMauguiere amp Peyron 2012) That is although thereis evidence that these types of somatosensory infor-mation are distinguishable from each other no evi-dence has yet been presented for a scalartopography that separates for example hot fromcold representations Tactile texture can refer to avariety of physical properties we operationalized this

concept as a continuum from smooth to rough andthus the texture component is another scalar entityFor all three of these scalars we elected to representthe entire continuum as a single component Thatis the temperature component is intended tocapture the salience of temperature to a conceptregardless of whether the associated temperature ishot or cold

The published literature on conceptual processingof somatosensory information has focused almostexclusively on impaired knowledge of body parts inneurological patients (Coslett Saffran amp Schwoebel2002 Kemmerer amp Tranel 2008 Laiacona AllamanoLorenzi amp Capitani 2006 Schwoebel amp Coslett2005) though no definitive localization has emergedfrom these studies A number of functional imagingstudies suggest an overlap between networksinvolved in real pain perception and those involvedin empathic pain perception (perception of pain inothers) and perception of ldquosocial painrdquo (Eisenberger2012 Lamm et al 2011) suggesting that retrieval ofpain concepts involves a simulation process To ourknowledge no studies have yet examined the concep-tual representation of temperature tactile texture orweight

Finally object shape is an attribute learned partlythrough haptic experiences (Miqueacutee et al 2008) butthe degree to which haptic shape is learned indepen-dently from visual shape is unclear Some objects areseen but not felt (ie very large and very distantobjects) but the converse is rarely true at least insighted individuals It was therefore decided that thevisual shape query would capture knowledge aboutshape in both modalities and that a separate queryregarding extent of personal manipulation experience(see Motor Components) would capture the impor-tance of haptic shape relative to visual shape

Auditory components

The auditory cortex includes low-level areas tuned tofrequency (tonotopy) amplitude and spatial location(Schreiner amp Winer 2007) as well as higher levelsshowing specialization for auditory ldquoobjectsrdquo such asenvironmental sounds (Leaver amp Rauschecker 2010)and speech sounds (Belin Zatorre Lafaille Ahad ampPike 2000 Liebenthal Binder Spitzer Possing ampMedler 2005) Corresponding auditory attributes ofthe proposed conceptual representation capture the

136 J R BINDER ET AL

degree to which a concept is associated with a low-pitched high-pitched loud (ie high amplitude)recognizable (ie having a characteristic auditoryform) or speech-like sound The attribute ldquolow inamplituderdquo was not included because it was con-sidered to be a salient attribute of relatively fewconceptsmdashthat is those that refer specifically to alow-amplitude variant (eg whisper) Concepts thatfocus on the absence of sound (eg silence quiet)are probably more accurately encoded as a nullvalue on the ldquoloudrdquo attribute as fMRI studies haveshown auditory regions where activity increases withsound intensity but not the converse (Roumlhl amp Uppen-kamp 2012)

A few studies of sound-related knowledge(Goldberg Perfetti amp Schneider 2006 Kellenbachet al 2001 Kiefer Sim Herrnberger Grothe ampHoenig 2008) reported activation in or near the pos-terior superior temporal sulcus (STS) an auditoryassociation region implicated in environmentalsound recognition (Leaver amp Rauschecker 2010 J WLewis et al 2004) Trumpp et al (Trumpp KlieseHoenig Haarmeier amp Kiefer 2013) described apatient with a circumscribed lesion centred on theleft posterior STS who displayed a specific deficit(slower response times and higher error rate) invisual recognition of sound-related words All ofthese studies examined general environmentalsound knowledge nothing is yet known regardingthe conceptual representation of specific types ofauditory attributes like frequency or amplitude

Localization of sounds in space is an importantaspect of auditory perception yet auditory perceptionof space has little or no representation in languageindependent of (multimodal) spatial concepts thereare no concepts with components like ldquomakessounds from the leftrdquo because spatial relationshipsbetween sound sources and listeners are almostnever fixed or central to meaning Like shape attri-butes of concepts spatial attributes and spatial con-cepts are learned through strongly correlatedmultimodal experiences involving visual auditorytactile and motor processes In the proposed semanticrepresentation model spatial attributes of conceptsare represented by modality-independent spatialcomponents (see Spatial Components)

A final salient component of human auditoryexperience is the domain of music Music perceptionwhich includes processing of melody harmony and

rhythm elements in sound appears to engage rela-tively specialized right lateralized networks some ofwhich may also process nonverbal (prosodic)elements in speech (Zatorre Belin amp Penhune2002) Many words refer explicitly to musical phenom-ena (sing song melody harmony rhythm etc) or toentities (instruments performers performances) thatproduce musical sounds Therefore the model pro-posed here includes a component to capture theexperience of processing music

Gustatory and olfactory components

Gustatory and olfactory experiences are each rep-resented by a single attribute that captures the rel-evance of each type of experience to the conceptThese sensory systems do not show clear large-scaleorganization along any dimension and neuronswithin each system often respond to a broad spec-trum of items Both gustatory and olfactory conceptshave been linked with specific early sensory andhigher associative neural processors (Goldberg et al2006 Gonzaacutelez et al 2006)

Motor components

The motor components of the proposed conceptualrepresentation (Table 1) capture the degree to whicha concept is associated with actions involving particu-lar body parts The neural representation of motoraction verbs has been extensively studied withmany studies showing action-specific activation of ahigh-level network that includes the supramarginalgyrus and posterior middle temporal gyrus (BinderDesai Conant amp Graves 2009 Desai Binder Conantamp Seidenberg 2009) There is also evidence for soma-totopic representation of action verb representation inprimary sensorimotor areas (Hauk Johnsrude ampPulvermuller 2004) though this claim is somewhatcontroversial (Postle McMahon Ashton Meredith ampde Zubicaray 2008) Object concepts associated withparticular actions also activate the action network(Binder et al 2009 Boronat et al 2005 Martin et al1995) and there is evidence for somatotopicallyspecific activation depending on which body part istypically used in the object interaction (CarotaMoseley amp Pulvermuumlller 2012) Following precedentin the neuroimaging literature (Carota et al 2012Hauk et al 2004 Tettamanti et al 2005) a three-

COGNITIVE NEUROPSYCHOLOGY 137

way partition into head-related (face mouth ortongue) upper limb (arm hand or fingers) andlower limb (leg or foot) actions was used to assesssomatotopic organization of action concepts A refer-ence to eye actions was felt to be confusingbecause all visual experiences involve the eyes inthis sense any visible object could be construed asldquoassociated with action involving the eyesrdquo

Finally the extent to which action attributes arerepresented in action networks may be partly deter-mined by personal experience (Calvo-Merino GlaserGrezes Passingham amp Haggard 2005 JaumlnckeKoeneke Hoppe Rominger amp Haumlnggi 2009) Mostpeople would rate ldquopianordquo as strongly associatedwith upper limb actions but most people also haveno personal experience with these specific actionsBased on prior research it is likely that the action rep-resentation of the concept ldquopianordquo is more extensivein the case of pianists Therefore a separate com-ponent (called ldquopracticerdquo) was created to capture thelevel of personal experience the rater has with per-forming the actions associated with the concept

Spatial components

Spatial and other more abstract components are listedin Table 2 Neural systems that encode aspects ofspatial experience have been investigated at manylevels (Andersen Snyder Bradley amp Xing 1997Burgess 2008 Kemmerer 2006 Kranjec CardilloSchmidt Lehet amp Chatterjee 2012 Woods et al2014) As mentioned above the acquisition of basicspatial concepts and knowledge of spatial attributesof concepts generally involves correlated multimodalinputs The most obvious spatial content in languageis the set of relations expressed by prepositionalphrases which have been a major focus of study in lin-guistics and semantic theory (Landau amp Jackendoff1993 Talmy 1983) Lesion correlation and neuroima-ging studies have implicated various parietal regionsparticularly the left inferior parietal lobe in the proces-sing of spatial prepositions and depicted locativerelations (Amorapanth Widick amp Chatterjee 2010Noordzij Neggers Ramsey amp Postma 2008 Tranel ampKemmerer 2004 Wu Waller amp Chatterjee 2007) Pre-positional phrases appear to express most if not all ofthe explicit relational spatial content that occurs inEnglish a semantic component targeting this type ofcontent therefore appears unnecessary (ie the set

of words with high values on this component wouldbe identical to the set of spatial prepositions) Thespatial components of the proposed representationmodel focus instead on other types of spatial infor-mation found in nouns verbs and adjectives

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior parahip-pocampal region (Epstein amp Kanwisher 1998 Epsteinet al 2007) The perception of three-dimensionalspace is based largely on visual input but also involvescorrelated proprioceptive information wheneverthree-dimensional space is experienced via move-ment of the body through space Spatial localizationin the auditory modality also probably contributes tothe experience of three-dimensional space Some con-cepts refer directly to interior spaces such as kitchenfoyer bathroom classroom and so on and seem toinclude information about general three-dimensionalsize and layout as do concepts about types of build-ings (library school hospital etc) More generallysuch concepts include the knowledge that they canbe physically entered navigated and exited whichmakes possible a range of conceptual combinations(entered the kitchen ran through the hall went out ofthe library etc) that are not possible for objectslacking large interior spaces (entered the chair) Thesalient property of concepts that determineswhether they activate a three-dimensional represen-tation of space is the degree to which they evoke aparticular ldquoscenerdquomdashthat is an array of objects in athree-dimensional space either by direct referenceor by association Space names (kitchen etc) andbuilding names (library etc) refer directly to three-dimensional scenes Many other object conceptsevoke mental representations of scenes by virtue ofthematic relations such as the evocation of kitchenby oven An object concept like chair would probablyfail to evoke such a representation as chairs are notlinked thematically with any particular scene Thusthere is a graded degree of mental representation ofthree-dimensional space evoked by different con-cepts which we hypothesize results in a correspond-ing variation in activation of the three-dimensionalspace neural processor

Large-scale (topographical) spatial information iscritical for navigation in the environment Entitiesthat are large and have a fixed location such as geo-logical formations and buildings can serve as land-marks within a mental map of the environment We

138 J R BINDER ET AL

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

content of concepts regardless of the particular colourin question A concept like lemon that is reliably associ-ated with a particular colour is expected to activatethis neural processor to approximately the samedegree as a concept like coffee that is reliably associ-ated with a different colour

Visual motion perception involves a relativelyspecialized neural processor known as MT or V5 (Alb-right Desimone amp Gross 1984 Zeki 1974) Responsesin MT vary with motion velocity and degree of motioncoherence (Lingnau Ashida Wall amp Smith 2009 ReesFriston amp Koch 2000 Tootell et al 1995) Motion vel-ocity is relevant for our purposes as it is strongly rep-resented at a conceptual level For examplemovements may be relatively fast or slow in velocityand this scalar quantity is central to action conceptslike run dash zoom creep ooze walk and so on aswell as entity concepts like bullet jet hare snail tor-toise sloth and so on Movements may also have aspecific pattern or quality as illustrated for exampleby the large number of verbs that express manner ofmotion (turn bounce roll float spin twist etc) Weare not aware of any evidence for large-scale spatialorganization in the cortex according to type ormanner of motion therefore the proposed semanticrepresentation is designed to capture strength ofassociation with visually observable characteristicmotion in general regardless of specific type Theexception to this rule is the perception of biologicalmotion which has been linked with a neural processorthat is clearly distinct from MT located in the posteriorsuperior temporal sulcus (STS Grossman amp Blake2002 Pelphrey Morris Michelich Allison amp McCarthy2005) Biological motion contains higher order com-plexity that is characteristic of face and body partactions Perception of biological motion plays acentral role in understanding face and body gesturesand thus the affective and intentional state of otheragents (ie theory of mind Allison Puce amp McCarthy2000) Biological motion is thus one of several keyattributes distinguishing intentional agents (boy girlwoman man etc) from inanimate entities

Visual shape perception involves an extensiveregion of extrastriate cortex including a large lateralextrastriate area known as the lateral occipitalcomplex (Malach et al 1995) and an adjacent areaon the ventral occipitalndashtemporal surface known asVOT (Grill-Spector amp Malach 2004) In human func-tional magnetic resonance imaging (fMRI) studies

these areas respond more strongly to images ofobjects than to images in which shape contours ofthe objects have been disrupted (ldquoscrambledrdquoimages) similar to response patterns observed inprimate ventral temporal neurons (Tanaka SaitoFukada amp Moriya 1991) Shape information is gener-ally the most important attribute of concrete objectconcepts and distinguishes concrete concepts froma range of abstract (eg affective social and cogni-tive) entities and event concepts Variation in the sal-ience of visual shape also occurs within the domainof concrete entities Shape is typically not a definingfeature of substance nouns (metal plastic water) butit has relevance for some substance-like mass nouns(butter coffee rice) Concrete objects also vary inshape complexity Animals for example tend tohave more complex shapes than plants or tools (Snod-grass amp Vanderwart 1980) We hypothesize that bothshape salience and shape complexity determine thedegree of activation in shape processing regionsduring concept retrieval

More focal areas within or adjacent to these shapeprocessing regions show preferential responses tohuman faces (Kanwisher McDermott amp Chun 1997Schwarzlose Baker amp Kanwisher 2005) and bodyparts (Downing Jiang Shuman amp Kanwisher 2001)A specialized mechanism for face perception is con-sistent with the central role of face recognition insocial interactions Visual perception of body partsprobably plays a crucial role in understandingobserved actions and mapping these onto internalrepresentations of the body schema during actionlearning These processors would be differentially acti-vated by concepts pertaining to faces (nose mouthportrait) and body parts (hand leg finger) As a firstapproximation we propose that both of these neuralprocessors are activated to some degree by all con-cepts referring to human beings which by necessityinclude a face and other human body parts Activationof the face processor may be minimal in the case ofconcepts that represent general human types androles (boy man nurse etc) as these concepts do notinclude information about any specific face whereasconcepts referencing specific individuals (brotherfather Einstein) might include specific facial featureinformation and therefore activate the face processorto a greater degree

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior

COGNITIVE NEUROPSYCHOLOGY 135

parahippocampal region (Epstein amp Kanwisher 1998Epstein Parker amp Feiler 2007) Although this processoris typically studied using visual stimuli and is oftenconsidered a high-level visual area it more likely inte-grates visual proprioceptive and perhaps auditoryinputs all of which provide correlated informationabout three-dimensional space Further discussion ofthis component of experience is provided in thesection below on Spatial Components (Table 2)

Somatosensory components

Somatosensory networks of the cortex represent bodylocation (somatotopic representation) body positionand joint force (proprioception) pain surface charac-teristics of objects (texture and temperature) andobject shape (Friebel Eickhoff amp Lotze 2011Hendry Hsiao amp Bushnell 1999 Lamm Decety ampSinger 2011 Longo Azanon amp Haggard 2010McGlone amp Reilly 2010 Medina amp Coslett 2010Serino amp Haggard 2010) Somatotopy is an inherentlymulti-modal phenomenon because the body locationtouched by a particular object is typically the samebody location as that used during exploratory ormanipulative motor actions involving the object Con-ceptual and linguistic representations are more likelyto encode the action as the salient characteristic ofthe object rather than the tactile experience (we sayldquoI threw the ballrdquo to describe the event not ldquothe balltouched my handrdquo) therefore somatotopic knowledgewas encoded in the proposed representation usingaction-based rather than somatosensory attributes(see Motor Components)

Temperature tactile texture and pain wereincluded as components of the representation aswas a component referring to the salience of objectweight Weight is perceived mainly via proprioceptiverepresentations and is a salient attribute of objectswith which we interact Temperature and weight aretwo more examples of scalar entities that may ormay not have a macroscopic topography in thebrain (Hendry et al 1999 Mazzola Faillenot BarralMauguiere amp Peyron 2012) That is although thereis evidence that these types of somatosensory infor-mation are distinguishable from each other no evi-dence has yet been presented for a scalartopography that separates for example hot fromcold representations Tactile texture can refer to avariety of physical properties we operationalized this

concept as a continuum from smooth to rough andthus the texture component is another scalar entityFor all three of these scalars we elected to representthe entire continuum as a single component Thatis the temperature component is intended tocapture the salience of temperature to a conceptregardless of whether the associated temperature ishot or cold

The published literature on conceptual processingof somatosensory information has focused almostexclusively on impaired knowledge of body parts inneurological patients (Coslett Saffran amp Schwoebel2002 Kemmerer amp Tranel 2008 Laiacona AllamanoLorenzi amp Capitani 2006 Schwoebel amp Coslett2005) though no definitive localization has emergedfrom these studies A number of functional imagingstudies suggest an overlap between networksinvolved in real pain perception and those involvedin empathic pain perception (perception of pain inothers) and perception of ldquosocial painrdquo (Eisenberger2012 Lamm et al 2011) suggesting that retrieval ofpain concepts involves a simulation process To ourknowledge no studies have yet examined the concep-tual representation of temperature tactile texture orweight

Finally object shape is an attribute learned partlythrough haptic experiences (Miqueacutee et al 2008) butthe degree to which haptic shape is learned indepen-dently from visual shape is unclear Some objects areseen but not felt (ie very large and very distantobjects) but the converse is rarely true at least insighted individuals It was therefore decided that thevisual shape query would capture knowledge aboutshape in both modalities and that a separate queryregarding extent of personal manipulation experience(see Motor Components) would capture the impor-tance of haptic shape relative to visual shape

Auditory components

The auditory cortex includes low-level areas tuned tofrequency (tonotopy) amplitude and spatial location(Schreiner amp Winer 2007) as well as higher levelsshowing specialization for auditory ldquoobjectsrdquo such asenvironmental sounds (Leaver amp Rauschecker 2010)and speech sounds (Belin Zatorre Lafaille Ahad ampPike 2000 Liebenthal Binder Spitzer Possing ampMedler 2005) Corresponding auditory attributes ofthe proposed conceptual representation capture the

136 J R BINDER ET AL

degree to which a concept is associated with a low-pitched high-pitched loud (ie high amplitude)recognizable (ie having a characteristic auditoryform) or speech-like sound The attribute ldquolow inamplituderdquo was not included because it was con-sidered to be a salient attribute of relatively fewconceptsmdashthat is those that refer specifically to alow-amplitude variant (eg whisper) Concepts thatfocus on the absence of sound (eg silence quiet)are probably more accurately encoded as a nullvalue on the ldquoloudrdquo attribute as fMRI studies haveshown auditory regions where activity increases withsound intensity but not the converse (Roumlhl amp Uppen-kamp 2012)

A few studies of sound-related knowledge(Goldberg Perfetti amp Schneider 2006 Kellenbachet al 2001 Kiefer Sim Herrnberger Grothe ampHoenig 2008) reported activation in or near the pos-terior superior temporal sulcus (STS) an auditoryassociation region implicated in environmentalsound recognition (Leaver amp Rauschecker 2010 J WLewis et al 2004) Trumpp et al (Trumpp KlieseHoenig Haarmeier amp Kiefer 2013) described apatient with a circumscribed lesion centred on theleft posterior STS who displayed a specific deficit(slower response times and higher error rate) invisual recognition of sound-related words All ofthese studies examined general environmentalsound knowledge nothing is yet known regardingthe conceptual representation of specific types ofauditory attributes like frequency or amplitude

Localization of sounds in space is an importantaspect of auditory perception yet auditory perceptionof space has little or no representation in languageindependent of (multimodal) spatial concepts thereare no concepts with components like ldquomakessounds from the leftrdquo because spatial relationshipsbetween sound sources and listeners are almostnever fixed or central to meaning Like shape attri-butes of concepts spatial attributes and spatial con-cepts are learned through strongly correlatedmultimodal experiences involving visual auditorytactile and motor processes In the proposed semanticrepresentation model spatial attributes of conceptsare represented by modality-independent spatialcomponents (see Spatial Components)

A final salient component of human auditoryexperience is the domain of music Music perceptionwhich includes processing of melody harmony and

rhythm elements in sound appears to engage rela-tively specialized right lateralized networks some ofwhich may also process nonverbal (prosodic)elements in speech (Zatorre Belin amp Penhune2002) Many words refer explicitly to musical phenom-ena (sing song melody harmony rhythm etc) or toentities (instruments performers performances) thatproduce musical sounds Therefore the model pro-posed here includes a component to capture theexperience of processing music

Gustatory and olfactory components

Gustatory and olfactory experiences are each rep-resented by a single attribute that captures the rel-evance of each type of experience to the conceptThese sensory systems do not show clear large-scaleorganization along any dimension and neuronswithin each system often respond to a broad spec-trum of items Both gustatory and olfactory conceptshave been linked with specific early sensory andhigher associative neural processors (Goldberg et al2006 Gonzaacutelez et al 2006)

Motor components

The motor components of the proposed conceptualrepresentation (Table 1) capture the degree to whicha concept is associated with actions involving particu-lar body parts The neural representation of motoraction verbs has been extensively studied withmany studies showing action-specific activation of ahigh-level network that includes the supramarginalgyrus and posterior middle temporal gyrus (BinderDesai Conant amp Graves 2009 Desai Binder Conantamp Seidenberg 2009) There is also evidence for soma-totopic representation of action verb representation inprimary sensorimotor areas (Hauk Johnsrude ampPulvermuller 2004) though this claim is somewhatcontroversial (Postle McMahon Ashton Meredith ampde Zubicaray 2008) Object concepts associated withparticular actions also activate the action network(Binder et al 2009 Boronat et al 2005 Martin et al1995) and there is evidence for somatotopicallyspecific activation depending on which body part istypically used in the object interaction (CarotaMoseley amp Pulvermuumlller 2012) Following precedentin the neuroimaging literature (Carota et al 2012Hauk et al 2004 Tettamanti et al 2005) a three-

COGNITIVE NEUROPSYCHOLOGY 137

way partition into head-related (face mouth ortongue) upper limb (arm hand or fingers) andlower limb (leg or foot) actions was used to assesssomatotopic organization of action concepts A refer-ence to eye actions was felt to be confusingbecause all visual experiences involve the eyes inthis sense any visible object could be construed asldquoassociated with action involving the eyesrdquo

Finally the extent to which action attributes arerepresented in action networks may be partly deter-mined by personal experience (Calvo-Merino GlaserGrezes Passingham amp Haggard 2005 JaumlnckeKoeneke Hoppe Rominger amp Haumlnggi 2009) Mostpeople would rate ldquopianordquo as strongly associatedwith upper limb actions but most people also haveno personal experience with these specific actionsBased on prior research it is likely that the action rep-resentation of the concept ldquopianordquo is more extensivein the case of pianists Therefore a separate com-ponent (called ldquopracticerdquo) was created to capture thelevel of personal experience the rater has with per-forming the actions associated with the concept

Spatial components

Spatial and other more abstract components are listedin Table 2 Neural systems that encode aspects ofspatial experience have been investigated at manylevels (Andersen Snyder Bradley amp Xing 1997Burgess 2008 Kemmerer 2006 Kranjec CardilloSchmidt Lehet amp Chatterjee 2012 Woods et al2014) As mentioned above the acquisition of basicspatial concepts and knowledge of spatial attributesof concepts generally involves correlated multimodalinputs The most obvious spatial content in languageis the set of relations expressed by prepositionalphrases which have been a major focus of study in lin-guistics and semantic theory (Landau amp Jackendoff1993 Talmy 1983) Lesion correlation and neuroima-ging studies have implicated various parietal regionsparticularly the left inferior parietal lobe in the proces-sing of spatial prepositions and depicted locativerelations (Amorapanth Widick amp Chatterjee 2010Noordzij Neggers Ramsey amp Postma 2008 Tranel ampKemmerer 2004 Wu Waller amp Chatterjee 2007) Pre-positional phrases appear to express most if not all ofthe explicit relational spatial content that occurs inEnglish a semantic component targeting this type ofcontent therefore appears unnecessary (ie the set

of words with high values on this component wouldbe identical to the set of spatial prepositions) Thespatial components of the proposed representationmodel focus instead on other types of spatial infor-mation found in nouns verbs and adjectives

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior parahip-pocampal region (Epstein amp Kanwisher 1998 Epsteinet al 2007) The perception of three-dimensionalspace is based largely on visual input but also involvescorrelated proprioceptive information wheneverthree-dimensional space is experienced via move-ment of the body through space Spatial localizationin the auditory modality also probably contributes tothe experience of three-dimensional space Some con-cepts refer directly to interior spaces such as kitchenfoyer bathroom classroom and so on and seem toinclude information about general three-dimensionalsize and layout as do concepts about types of build-ings (library school hospital etc) More generallysuch concepts include the knowledge that they canbe physically entered navigated and exited whichmakes possible a range of conceptual combinations(entered the kitchen ran through the hall went out ofthe library etc) that are not possible for objectslacking large interior spaces (entered the chair) Thesalient property of concepts that determineswhether they activate a three-dimensional represen-tation of space is the degree to which they evoke aparticular ldquoscenerdquomdashthat is an array of objects in athree-dimensional space either by direct referenceor by association Space names (kitchen etc) andbuilding names (library etc) refer directly to three-dimensional scenes Many other object conceptsevoke mental representations of scenes by virtue ofthematic relations such as the evocation of kitchenby oven An object concept like chair would probablyfail to evoke such a representation as chairs are notlinked thematically with any particular scene Thusthere is a graded degree of mental representation ofthree-dimensional space evoked by different con-cepts which we hypothesize results in a correspond-ing variation in activation of the three-dimensionalspace neural processor

Large-scale (topographical) spatial information iscritical for navigation in the environment Entitiesthat are large and have a fixed location such as geo-logical formations and buildings can serve as land-marks within a mental map of the environment We

138 J R BINDER ET AL

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

parahippocampal region (Epstein amp Kanwisher 1998Epstein Parker amp Feiler 2007) Although this processoris typically studied using visual stimuli and is oftenconsidered a high-level visual area it more likely inte-grates visual proprioceptive and perhaps auditoryinputs all of which provide correlated informationabout three-dimensional space Further discussion ofthis component of experience is provided in thesection below on Spatial Components (Table 2)

Somatosensory components

Somatosensory networks of the cortex represent bodylocation (somatotopic representation) body positionand joint force (proprioception) pain surface charac-teristics of objects (texture and temperature) andobject shape (Friebel Eickhoff amp Lotze 2011Hendry Hsiao amp Bushnell 1999 Lamm Decety ampSinger 2011 Longo Azanon amp Haggard 2010McGlone amp Reilly 2010 Medina amp Coslett 2010Serino amp Haggard 2010) Somatotopy is an inherentlymulti-modal phenomenon because the body locationtouched by a particular object is typically the samebody location as that used during exploratory ormanipulative motor actions involving the object Con-ceptual and linguistic representations are more likelyto encode the action as the salient characteristic ofthe object rather than the tactile experience (we sayldquoI threw the ballrdquo to describe the event not ldquothe balltouched my handrdquo) therefore somatotopic knowledgewas encoded in the proposed representation usingaction-based rather than somatosensory attributes(see Motor Components)

Temperature tactile texture and pain wereincluded as components of the representation aswas a component referring to the salience of objectweight Weight is perceived mainly via proprioceptiverepresentations and is a salient attribute of objectswith which we interact Temperature and weight aretwo more examples of scalar entities that may ormay not have a macroscopic topography in thebrain (Hendry et al 1999 Mazzola Faillenot BarralMauguiere amp Peyron 2012) That is although thereis evidence that these types of somatosensory infor-mation are distinguishable from each other no evi-dence has yet been presented for a scalartopography that separates for example hot fromcold representations Tactile texture can refer to avariety of physical properties we operationalized this

concept as a continuum from smooth to rough andthus the texture component is another scalar entityFor all three of these scalars we elected to representthe entire continuum as a single component Thatis the temperature component is intended tocapture the salience of temperature to a conceptregardless of whether the associated temperature ishot or cold

The published literature on conceptual processingof somatosensory information has focused almostexclusively on impaired knowledge of body parts inneurological patients (Coslett Saffran amp Schwoebel2002 Kemmerer amp Tranel 2008 Laiacona AllamanoLorenzi amp Capitani 2006 Schwoebel amp Coslett2005) though no definitive localization has emergedfrom these studies A number of functional imagingstudies suggest an overlap between networksinvolved in real pain perception and those involvedin empathic pain perception (perception of pain inothers) and perception of ldquosocial painrdquo (Eisenberger2012 Lamm et al 2011) suggesting that retrieval ofpain concepts involves a simulation process To ourknowledge no studies have yet examined the concep-tual representation of temperature tactile texture orweight

Finally object shape is an attribute learned partlythrough haptic experiences (Miqueacutee et al 2008) butthe degree to which haptic shape is learned indepen-dently from visual shape is unclear Some objects areseen but not felt (ie very large and very distantobjects) but the converse is rarely true at least insighted individuals It was therefore decided that thevisual shape query would capture knowledge aboutshape in both modalities and that a separate queryregarding extent of personal manipulation experience(see Motor Components) would capture the impor-tance of haptic shape relative to visual shape

Auditory components

The auditory cortex includes low-level areas tuned tofrequency (tonotopy) amplitude and spatial location(Schreiner amp Winer 2007) as well as higher levelsshowing specialization for auditory ldquoobjectsrdquo such asenvironmental sounds (Leaver amp Rauschecker 2010)and speech sounds (Belin Zatorre Lafaille Ahad ampPike 2000 Liebenthal Binder Spitzer Possing ampMedler 2005) Corresponding auditory attributes ofthe proposed conceptual representation capture the

136 J R BINDER ET AL

degree to which a concept is associated with a low-pitched high-pitched loud (ie high amplitude)recognizable (ie having a characteristic auditoryform) or speech-like sound The attribute ldquolow inamplituderdquo was not included because it was con-sidered to be a salient attribute of relatively fewconceptsmdashthat is those that refer specifically to alow-amplitude variant (eg whisper) Concepts thatfocus on the absence of sound (eg silence quiet)are probably more accurately encoded as a nullvalue on the ldquoloudrdquo attribute as fMRI studies haveshown auditory regions where activity increases withsound intensity but not the converse (Roumlhl amp Uppen-kamp 2012)

A few studies of sound-related knowledge(Goldberg Perfetti amp Schneider 2006 Kellenbachet al 2001 Kiefer Sim Herrnberger Grothe ampHoenig 2008) reported activation in or near the pos-terior superior temporal sulcus (STS) an auditoryassociation region implicated in environmentalsound recognition (Leaver amp Rauschecker 2010 J WLewis et al 2004) Trumpp et al (Trumpp KlieseHoenig Haarmeier amp Kiefer 2013) described apatient with a circumscribed lesion centred on theleft posterior STS who displayed a specific deficit(slower response times and higher error rate) invisual recognition of sound-related words All ofthese studies examined general environmentalsound knowledge nothing is yet known regardingthe conceptual representation of specific types ofauditory attributes like frequency or amplitude

Localization of sounds in space is an importantaspect of auditory perception yet auditory perceptionof space has little or no representation in languageindependent of (multimodal) spatial concepts thereare no concepts with components like ldquomakessounds from the leftrdquo because spatial relationshipsbetween sound sources and listeners are almostnever fixed or central to meaning Like shape attri-butes of concepts spatial attributes and spatial con-cepts are learned through strongly correlatedmultimodal experiences involving visual auditorytactile and motor processes In the proposed semanticrepresentation model spatial attributes of conceptsare represented by modality-independent spatialcomponents (see Spatial Components)

A final salient component of human auditoryexperience is the domain of music Music perceptionwhich includes processing of melody harmony and

rhythm elements in sound appears to engage rela-tively specialized right lateralized networks some ofwhich may also process nonverbal (prosodic)elements in speech (Zatorre Belin amp Penhune2002) Many words refer explicitly to musical phenom-ena (sing song melody harmony rhythm etc) or toentities (instruments performers performances) thatproduce musical sounds Therefore the model pro-posed here includes a component to capture theexperience of processing music

Gustatory and olfactory components

Gustatory and olfactory experiences are each rep-resented by a single attribute that captures the rel-evance of each type of experience to the conceptThese sensory systems do not show clear large-scaleorganization along any dimension and neuronswithin each system often respond to a broad spec-trum of items Both gustatory and olfactory conceptshave been linked with specific early sensory andhigher associative neural processors (Goldberg et al2006 Gonzaacutelez et al 2006)

Motor components

The motor components of the proposed conceptualrepresentation (Table 1) capture the degree to whicha concept is associated with actions involving particu-lar body parts The neural representation of motoraction verbs has been extensively studied withmany studies showing action-specific activation of ahigh-level network that includes the supramarginalgyrus and posterior middle temporal gyrus (BinderDesai Conant amp Graves 2009 Desai Binder Conantamp Seidenberg 2009) There is also evidence for soma-totopic representation of action verb representation inprimary sensorimotor areas (Hauk Johnsrude ampPulvermuller 2004) though this claim is somewhatcontroversial (Postle McMahon Ashton Meredith ampde Zubicaray 2008) Object concepts associated withparticular actions also activate the action network(Binder et al 2009 Boronat et al 2005 Martin et al1995) and there is evidence for somatotopicallyspecific activation depending on which body part istypically used in the object interaction (CarotaMoseley amp Pulvermuumlller 2012) Following precedentin the neuroimaging literature (Carota et al 2012Hauk et al 2004 Tettamanti et al 2005) a three-

COGNITIVE NEUROPSYCHOLOGY 137

way partition into head-related (face mouth ortongue) upper limb (arm hand or fingers) andlower limb (leg or foot) actions was used to assesssomatotopic organization of action concepts A refer-ence to eye actions was felt to be confusingbecause all visual experiences involve the eyes inthis sense any visible object could be construed asldquoassociated with action involving the eyesrdquo

Finally the extent to which action attributes arerepresented in action networks may be partly deter-mined by personal experience (Calvo-Merino GlaserGrezes Passingham amp Haggard 2005 JaumlnckeKoeneke Hoppe Rominger amp Haumlnggi 2009) Mostpeople would rate ldquopianordquo as strongly associatedwith upper limb actions but most people also haveno personal experience with these specific actionsBased on prior research it is likely that the action rep-resentation of the concept ldquopianordquo is more extensivein the case of pianists Therefore a separate com-ponent (called ldquopracticerdquo) was created to capture thelevel of personal experience the rater has with per-forming the actions associated with the concept

Spatial components

Spatial and other more abstract components are listedin Table 2 Neural systems that encode aspects ofspatial experience have been investigated at manylevels (Andersen Snyder Bradley amp Xing 1997Burgess 2008 Kemmerer 2006 Kranjec CardilloSchmidt Lehet amp Chatterjee 2012 Woods et al2014) As mentioned above the acquisition of basicspatial concepts and knowledge of spatial attributesof concepts generally involves correlated multimodalinputs The most obvious spatial content in languageis the set of relations expressed by prepositionalphrases which have been a major focus of study in lin-guistics and semantic theory (Landau amp Jackendoff1993 Talmy 1983) Lesion correlation and neuroima-ging studies have implicated various parietal regionsparticularly the left inferior parietal lobe in the proces-sing of spatial prepositions and depicted locativerelations (Amorapanth Widick amp Chatterjee 2010Noordzij Neggers Ramsey amp Postma 2008 Tranel ampKemmerer 2004 Wu Waller amp Chatterjee 2007) Pre-positional phrases appear to express most if not all ofthe explicit relational spatial content that occurs inEnglish a semantic component targeting this type ofcontent therefore appears unnecessary (ie the set

of words with high values on this component wouldbe identical to the set of spatial prepositions) Thespatial components of the proposed representationmodel focus instead on other types of spatial infor-mation found in nouns verbs and adjectives

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior parahip-pocampal region (Epstein amp Kanwisher 1998 Epsteinet al 2007) The perception of three-dimensionalspace is based largely on visual input but also involvescorrelated proprioceptive information wheneverthree-dimensional space is experienced via move-ment of the body through space Spatial localizationin the auditory modality also probably contributes tothe experience of three-dimensional space Some con-cepts refer directly to interior spaces such as kitchenfoyer bathroom classroom and so on and seem toinclude information about general three-dimensionalsize and layout as do concepts about types of build-ings (library school hospital etc) More generallysuch concepts include the knowledge that they canbe physically entered navigated and exited whichmakes possible a range of conceptual combinations(entered the kitchen ran through the hall went out ofthe library etc) that are not possible for objectslacking large interior spaces (entered the chair) Thesalient property of concepts that determineswhether they activate a three-dimensional represen-tation of space is the degree to which they evoke aparticular ldquoscenerdquomdashthat is an array of objects in athree-dimensional space either by direct referenceor by association Space names (kitchen etc) andbuilding names (library etc) refer directly to three-dimensional scenes Many other object conceptsevoke mental representations of scenes by virtue ofthematic relations such as the evocation of kitchenby oven An object concept like chair would probablyfail to evoke such a representation as chairs are notlinked thematically with any particular scene Thusthere is a graded degree of mental representation ofthree-dimensional space evoked by different con-cepts which we hypothesize results in a correspond-ing variation in activation of the three-dimensionalspace neural processor

Large-scale (topographical) spatial information iscritical for navigation in the environment Entitiesthat are large and have a fixed location such as geo-logical formations and buildings can serve as land-marks within a mental map of the environment We

138 J R BINDER ET AL

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

degree to which a concept is associated with a low-pitched high-pitched loud (ie high amplitude)recognizable (ie having a characteristic auditoryform) or speech-like sound The attribute ldquolow inamplituderdquo was not included because it was con-sidered to be a salient attribute of relatively fewconceptsmdashthat is those that refer specifically to alow-amplitude variant (eg whisper) Concepts thatfocus on the absence of sound (eg silence quiet)are probably more accurately encoded as a nullvalue on the ldquoloudrdquo attribute as fMRI studies haveshown auditory regions where activity increases withsound intensity but not the converse (Roumlhl amp Uppen-kamp 2012)

A few studies of sound-related knowledge(Goldberg Perfetti amp Schneider 2006 Kellenbachet al 2001 Kiefer Sim Herrnberger Grothe ampHoenig 2008) reported activation in or near the pos-terior superior temporal sulcus (STS) an auditoryassociation region implicated in environmentalsound recognition (Leaver amp Rauschecker 2010 J WLewis et al 2004) Trumpp et al (Trumpp KlieseHoenig Haarmeier amp Kiefer 2013) described apatient with a circumscribed lesion centred on theleft posterior STS who displayed a specific deficit(slower response times and higher error rate) invisual recognition of sound-related words All ofthese studies examined general environmentalsound knowledge nothing is yet known regardingthe conceptual representation of specific types ofauditory attributes like frequency or amplitude

Localization of sounds in space is an importantaspect of auditory perception yet auditory perceptionof space has little or no representation in languageindependent of (multimodal) spatial concepts thereare no concepts with components like ldquomakessounds from the leftrdquo because spatial relationshipsbetween sound sources and listeners are almostnever fixed or central to meaning Like shape attri-butes of concepts spatial attributes and spatial con-cepts are learned through strongly correlatedmultimodal experiences involving visual auditorytactile and motor processes In the proposed semanticrepresentation model spatial attributes of conceptsare represented by modality-independent spatialcomponents (see Spatial Components)

A final salient component of human auditoryexperience is the domain of music Music perceptionwhich includes processing of melody harmony and

rhythm elements in sound appears to engage rela-tively specialized right lateralized networks some ofwhich may also process nonverbal (prosodic)elements in speech (Zatorre Belin amp Penhune2002) Many words refer explicitly to musical phenom-ena (sing song melody harmony rhythm etc) or toentities (instruments performers performances) thatproduce musical sounds Therefore the model pro-posed here includes a component to capture theexperience of processing music

Gustatory and olfactory components

Gustatory and olfactory experiences are each rep-resented by a single attribute that captures the rel-evance of each type of experience to the conceptThese sensory systems do not show clear large-scaleorganization along any dimension and neuronswithin each system often respond to a broad spec-trum of items Both gustatory and olfactory conceptshave been linked with specific early sensory andhigher associative neural processors (Goldberg et al2006 Gonzaacutelez et al 2006)

Motor components

The motor components of the proposed conceptualrepresentation (Table 1) capture the degree to whicha concept is associated with actions involving particu-lar body parts The neural representation of motoraction verbs has been extensively studied withmany studies showing action-specific activation of ahigh-level network that includes the supramarginalgyrus and posterior middle temporal gyrus (BinderDesai Conant amp Graves 2009 Desai Binder Conantamp Seidenberg 2009) There is also evidence for soma-totopic representation of action verb representation inprimary sensorimotor areas (Hauk Johnsrude ampPulvermuller 2004) though this claim is somewhatcontroversial (Postle McMahon Ashton Meredith ampde Zubicaray 2008) Object concepts associated withparticular actions also activate the action network(Binder et al 2009 Boronat et al 2005 Martin et al1995) and there is evidence for somatotopicallyspecific activation depending on which body part istypically used in the object interaction (CarotaMoseley amp Pulvermuumlller 2012) Following precedentin the neuroimaging literature (Carota et al 2012Hauk et al 2004 Tettamanti et al 2005) a three-

COGNITIVE NEUROPSYCHOLOGY 137

way partition into head-related (face mouth ortongue) upper limb (arm hand or fingers) andlower limb (leg or foot) actions was used to assesssomatotopic organization of action concepts A refer-ence to eye actions was felt to be confusingbecause all visual experiences involve the eyes inthis sense any visible object could be construed asldquoassociated with action involving the eyesrdquo

Finally the extent to which action attributes arerepresented in action networks may be partly deter-mined by personal experience (Calvo-Merino GlaserGrezes Passingham amp Haggard 2005 JaumlnckeKoeneke Hoppe Rominger amp Haumlnggi 2009) Mostpeople would rate ldquopianordquo as strongly associatedwith upper limb actions but most people also haveno personal experience with these specific actionsBased on prior research it is likely that the action rep-resentation of the concept ldquopianordquo is more extensivein the case of pianists Therefore a separate com-ponent (called ldquopracticerdquo) was created to capture thelevel of personal experience the rater has with per-forming the actions associated with the concept

Spatial components

Spatial and other more abstract components are listedin Table 2 Neural systems that encode aspects ofspatial experience have been investigated at manylevels (Andersen Snyder Bradley amp Xing 1997Burgess 2008 Kemmerer 2006 Kranjec CardilloSchmidt Lehet amp Chatterjee 2012 Woods et al2014) As mentioned above the acquisition of basicspatial concepts and knowledge of spatial attributesof concepts generally involves correlated multimodalinputs The most obvious spatial content in languageis the set of relations expressed by prepositionalphrases which have been a major focus of study in lin-guistics and semantic theory (Landau amp Jackendoff1993 Talmy 1983) Lesion correlation and neuroima-ging studies have implicated various parietal regionsparticularly the left inferior parietal lobe in the proces-sing of spatial prepositions and depicted locativerelations (Amorapanth Widick amp Chatterjee 2010Noordzij Neggers Ramsey amp Postma 2008 Tranel ampKemmerer 2004 Wu Waller amp Chatterjee 2007) Pre-positional phrases appear to express most if not all ofthe explicit relational spatial content that occurs inEnglish a semantic component targeting this type ofcontent therefore appears unnecessary (ie the set

of words with high values on this component wouldbe identical to the set of spatial prepositions) Thespatial components of the proposed representationmodel focus instead on other types of spatial infor-mation found in nouns verbs and adjectives

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior parahip-pocampal region (Epstein amp Kanwisher 1998 Epsteinet al 2007) The perception of three-dimensionalspace is based largely on visual input but also involvescorrelated proprioceptive information wheneverthree-dimensional space is experienced via move-ment of the body through space Spatial localizationin the auditory modality also probably contributes tothe experience of three-dimensional space Some con-cepts refer directly to interior spaces such as kitchenfoyer bathroom classroom and so on and seem toinclude information about general three-dimensionalsize and layout as do concepts about types of build-ings (library school hospital etc) More generallysuch concepts include the knowledge that they canbe physically entered navigated and exited whichmakes possible a range of conceptual combinations(entered the kitchen ran through the hall went out ofthe library etc) that are not possible for objectslacking large interior spaces (entered the chair) Thesalient property of concepts that determineswhether they activate a three-dimensional represen-tation of space is the degree to which they evoke aparticular ldquoscenerdquomdashthat is an array of objects in athree-dimensional space either by direct referenceor by association Space names (kitchen etc) andbuilding names (library etc) refer directly to three-dimensional scenes Many other object conceptsevoke mental representations of scenes by virtue ofthematic relations such as the evocation of kitchenby oven An object concept like chair would probablyfail to evoke such a representation as chairs are notlinked thematically with any particular scene Thusthere is a graded degree of mental representation ofthree-dimensional space evoked by different con-cepts which we hypothesize results in a correspond-ing variation in activation of the three-dimensionalspace neural processor

Large-scale (topographical) spatial information iscritical for navigation in the environment Entitiesthat are large and have a fixed location such as geo-logical formations and buildings can serve as land-marks within a mental map of the environment We

138 J R BINDER ET AL

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

way partition into head-related (face mouth ortongue) upper limb (arm hand or fingers) andlower limb (leg or foot) actions was used to assesssomatotopic organization of action concepts A refer-ence to eye actions was felt to be confusingbecause all visual experiences involve the eyes inthis sense any visible object could be construed asldquoassociated with action involving the eyesrdquo

Finally the extent to which action attributes arerepresented in action networks may be partly deter-mined by personal experience (Calvo-Merino GlaserGrezes Passingham amp Haggard 2005 JaumlnckeKoeneke Hoppe Rominger amp Haumlnggi 2009) Mostpeople would rate ldquopianordquo as strongly associatedwith upper limb actions but most people also haveno personal experience with these specific actionsBased on prior research it is likely that the action rep-resentation of the concept ldquopianordquo is more extensivein the case of pianists Therefore a separate com-ponent (called ldquopracticerdquo) was created to capture thelevel of personal experience the rater has with per-forming the actions associated with the concept

Spatial components

Spatial and other more abstract components are listedin Table 2 Neural systems that encode aspects ofspatial experience have been investigated at manylevels (Andersen Snyder Bradley amp Xing 1997Burgess 2008 Kemmerer 2006 Kranjec CardilloSchmidt Lehet amp Chatterjee 2012 Woods et al2014) As mentioned above the acquisition of basicspatial concepts and knowledge of spatial attributesof concepts generally involves correlated multimodalinputs The most obvious spatial content in languageis the set of relations expressed by prepositionalphrases which have been a major focus of study in lin-guistics and semantic theory (Landau amp Jackendoff1993 Talmy 1983) Lesion correlation and neuroima-ging studies have implicated various parietal regionsparticularly the left inferior parietal lobe in the proces-sing of spatial prepositions and depicted locativerelations (Amorapanth Widick amp Chatterjee 2010Noordzij Neggers Ramsey amp Postma 2008 Tranel ampKemmerer 2004 Wu Waller amp Chatterjee 2007) Pre-positional phrases appear to express most if not all ofthe explicit relational spatial content that occurs inEnglish a semantic component targeting this type ofcontent therefore appears unnecessary (ie the set

of words with high values on this component wouldbe identical to the set of spatial prepositions) Thespatial components of the proposed representationmodel focus instead on other types of spatial infor-mation found in nouns verbs and adjectives

Perception of three-dimensional spaces involves aspecialized neural processor in the posterior parahip-pocampal region (Epstein amp Kanwisher 1998 Epsteinet al 2007) The perception of three-dimensionalspace is based largely on visual input but also involvescorrelated proprioceptive information wheneverthree-dimensional space is experienced via move-ment of the body through space Spatial localizationin the auditory modality also probably contributes tothe experience of three-dimensional space Some con-cepts refer directly to interior spaces such as kitchenfoyer bathroom classroom and so on and seem toinclude information about general three-dimensionalsize and layout as do concepts about types of build-ings (library school hospital etc) More generallysuch concepts include the knowledge that they canbe physically entered navigated and exited whichmakes possible a range of conceptual combinations(entered the kitchen ran through the hall went out ofthe library etc) that are not possible for objectslacking large interior spaces (entered the chair) Thesalient property of concepts that determineswhether they activate a three-dimensional represen-tation of space is the degree to which they evoke aparticular ldquoscenerdquomdashthat is an array of objects in athree-dimensional space either by direct referenceor by association Space names (kitchen etc) andbuilding names (library etc) refer directly to three-dimensional scenes Many other object conceptsevoke mental representations of scenes by virtue ofthematic relations such as the evocation of kitchenby oven An object concept like chair would probablyfail to evoke such a representation as chairs are notlinked thematically with any particular scene Thusthere is a graded degree of mental representation ofthree-dimensional space evoked by different con-cepts which we hypothesize results in a correspond-ing variation in activation of the three-dimensionalspace neural processor

Large-scale (topographical) spatial information iscritical for navigation in the environment Entitiesthat are large and have a fixed location such as geo-logical formations and buildings can serve as land-marks within a mental map of the environment We

138 J R BINDER ET AL

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

hypothesize that such concepts are partly encoded inenvironmental navigation systems located in the hip-pocampal parahippocampal and posterior cingulategyri (Burgess 2008 Maguire Frith Burgess Donnettamp OrsquoKeefe 1998 Wallentin Oslashstergaarda LundOslashstergaard amp Roepstorff 2005) and we assess the sal-ience of this ldquotopographicalrdquo attribute by inquiring towhat degree the concept in question could serve asa landmark on a map

Spatial cognition also includes spatial aspects ofmotion particularly direction or path of motionthrough space (location change) Some verbs thatdenote location change refer directly to a directionof motion (ascend climb descend rise fall) or type ofpath (arc bounce circle jump launch orbit swerve)Others imply a horizontal path of motion parallel tothe ground (run walk crawl swim) while othersrefer to paths toward or away from an observer(arrive depart leave return) Some entities are associ-ated with a particular type of path through space(bird butterfly car rocket satellite snake) Visualmotion area MT features a columnar spatial organiz-ation of neurons according to preferred axis ofmotion however this organization is at a relativelymicroscopic scale such that the full 180deg of axis ofmotion is represented in 500 microm of cortex (Albrightet al 1984) In human fMRI studies perception ofthe spatial aspects of motion has been linked with aneural processor in the intraparietal sulcus (IPS) andsurrounding dorsal visual stream (Culham et al1998 Wu Morganti amp Chatterjee 2008)

A final aspect of spatial cognition relates to periper-sonal proximity Intuitively the coding of peripersonalproximity is a central aspect of action representationand performance threat detection and resourceacquisition all of which are neural processes criticalfor survival Evidence from both animal and humanstudies suggests that the representation of periperso-nal space involves somewhat specialized neural mech-anisms (Graziano Yap amp Gross 1994 Makin Holmes ampZohary 2007) We hypothesize that these systems alsopartly encode the meaning of concepts that relate toperipersonal spacemdashthat is objects that are typicallywithin easy reach and actions performed on theseobjects Given the importance of peripersonal spacein action and object use two further componentswere included to explore the relevance of movementaway from or out of versus toward or into the selfSome action verbs (give punch tell throw) indicate

movement or relocation of something away fromthe self whereas others (acquire catch receive eat)indicate movement or relocation toward the selfThis distinction may also modulate the representationof objects that characteristically move toward or awayfrom the self (Rueschemeyer Pfeiffer amp Bekkering2010)

Mathematical cognition particularly the represen-tation of numerosity is a somewhat specialized infor-mation processing domain with well-documentedneural correlates (Dehaene 1997 Harvey KleinPetridou amp Dumoulin 2013 Piazza Izard PinelBihan amp Dehaene 2004) In addition to numbernames which refer directly to numerical quantitiesmany words are associated with the concept ofnumerical quantity (amount number quantity) orwith particular numbers (dime hand egg) Althoughnot a spatial process per se since it does not typicallyinvolve three dimensions numerosity processingseems to depend on directional vector represen-tations similar to those underlying spatial cognitionThus a component was included to capture associ-ation with numerical quantities and was consideredloosely part of the spatial domain

Temporal and causal components

Event concepts include several distinct types of infor-mation Temporal information pertains to the durationand temporal order of events Some types of eventhave a typical duration in which case the durationmay be a salient attribute of the event (breakfastlecture movie shower) Some event-like conceptsrefer specifically to durations (minute hour dayweek) Typical durations may be very short (blinkflash pop sneeze) or relatively long (childhoodcollege life war) Events of a particular type may alsorecur at particular times and thus occupy a character-istic location within the temporal sequence of eventsExamples of this phenomenon include recurring dailyevents (shower breakfast commute lunch dinner)recurring weekly or monthly events (Mondayweekend payday) recurring annual events (holidaysand other cultural events seasons and weatherphenomena) and concepts associated with particularphases of life (infancy childhood college marriageretirement) The neural correlates of these types ofinformation are not yet clear (Ferstl Rinck amp vonCramon 2005 Ferstl amp von Cramon 2007 Grondin

COGNITIVE NEUROPSYCHOLOGY 139

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143

2010 Kranjec et al 2012) Because time seems to bepervasively conceived of in spatial terms (Evans2004 Kranjec amp Chatterjee 2010 Lakoff amp Johnson1980) it is likely that temporal concepts share theirneural representation at least in part with spatial con-cepts Components of the proposed conceptual rep-resentation model are designed to capture thedegree to which a concept is associated with a charac-teristic duration the degree to which a concept isassociated with a long or a short duration and thedegree to which a concept is associated with a particu-lar or predictable point in time

Causal information pertains to cause and effectrelationships between concepts Events and situationsmay have a salient precipitating cause (infection killlaugh spill) or they may occur without an immediateapparent cause (eg some natural phenomenarandom occurrences) Events may or may not havehighly likely consequences Imaging evidencesuggests involvement of several inferior parietal andtemporal regions in low-level perception of causality(Blos Chatterjee Kircher amp Straube 2012 Fonlupt2003 Fugelsang Roser Corballis Gazzaniga ampDunbar 2005 Woods et al 2014) and a few studiesof causal processing at the conceptual level implicatethese and other areas (Kranjec et al 2012 Satputeet al 2005) The proposed conceptual representationmodel includes components designed to capture thedegree to which a concept is associated with a clearpreceding cause and the degree to which an eventor action has probable consequences

Social components

Theory of mind (TOM) refers to the ability to infer orpredict mental states and mental processes in othervolitional beings (typically though not exclusivelyother people) We hypothesize that the brainsystems underlying this ability are involved whenencoding the meaning of words that refer to inten-tional entities or actions because learning these con-cepts is frequently associated with TOM processingEssentially this attribute captures the semanticfeature of intentionality In addition to the query per-taining to events with social interactions as well as thequery regarding mental activities or events weincluded an object-directed query assessing thedegree to which a thing has human or human-likeintentions plans or goals and a corresponding verb

query assessing the degree to which an action reflectshuman intentions plans or goals The underlyinghypothesis is that such concepts which mostly referto people in particular roles and the actions theytake are partly understood by engaging a TOMprocess

There is strong evidence for the involvement of aspecific network of regions in TOM These regionsmost consistently include the medial prefrontalcortex posterior cingulateprecuneus and the tem-poroparietal junction (Schilbach et al 2012 SchurzRadua Aichhorn Richlan amp Perner 2014 VanOverwalle 2009 van Veluw amp Chance 2014) withseveral studies also implicating the anterior temporallobes (Ross amp Olson 2010 Schilbach et al 2012Schurz et al 2014) The temporoparietal junction(TPJ) is an area of particular focus in the TOM literaturebut it is not a consistently defined region and mayencompass parts of the posterior superior temporalgyrussulcus supramarginal gyrus and angulargyrus Collapsing across tasks TOM or intentionalityis frequently associated with significant activation inthe angular gyri (Carter amp Huettel 2013 Schurz et al2014) but some variability in the localization of peakactivation within the TPJ is seen across differentTOM tasks (Schurz et al 2014) In addition whilesome studies have shown right lateralization of acti-vation in the TPJ (Saxe amp Wexler 2005) severalmeta-analyses have suggested bilateral involvement(Schurz et al 2014 Spreng Mar amp Kim 2009 VanOverwalle 2009 van Veluw amp Chance 2014)

Another aspect of social cognition likely to have aneurobiological correlate is the representation of theself There is evidence to suggest that informationthat pertains to the self may be processed differentlyfrom information that is not considered to be self-related Some behavioural studies have suggestedthat people show better recall (Rogers Kuiper ampKirker 1977) and faster response times (Kuiper ampRogers 1979) when information is relevant to theself a phenomenon known as the self-referenceeffect Neuroimaging studies have typically implicatedcortical midline structures in the processing of self-referential information particularly the medial pre-frontal and parietal cortices (Araujo Kaplan ampDamasio 2013 Northoff et al 2006) While both self-and other-judgments activate these midline regionsgreater activation of medial prefrontal cortex for self-trait judgments than for other-trait judgments has

140 J R BINDER ET AL

been reported with the reverse pattern seen in medialparietal regions (Araujo et al 2013) In addition thereis evidence for a ventral-to-dorsal spatial gradientwithin medial prefrontal cortex for self- relative toother-judgments (Denny Kober Wager amp Ochsner2012) Preferential activation of left ventrolateral pre-frontal cortex and left insula for self-judgments andof the bilateral temporoparietal junction and cuneusfor other-judgments has also been seen (Dennyet al 2012) The relevant query for this attributeassesses the degree to which a target noun isldquorelated to your own view of yourselfrdquo or the degreeto which a target verb is ldquoan action or activity that istypical of you something you commonly do andidentify withrdquo

A third aspect of social cognition for which aspecific neurobiological correlate is likely is thedomain of communication Communication whetherby language or nonverbal means is the basis for allsocial interaction and many noun (eg speech bookmeeting) and verb (eg speak read meet) conceptsare closely associated with the act of communicatingWe hypothesize that comprehension of such items isassociated with relative activation of language net-works (particularly general lexical retrieval syntacticphonological and speech articulation components)and gestural communication systems compared toconcepts that do not reference communication orcommunicative acts

Cognition component

A central premise of the current approach is that allaspects of experience can contribute to conceptacquisition and therefore concept composition Forself-aware human beings purely cognitive eventsand states certainly comprise one aspect of experi-ence When we ldquohave a thoughtrdquo ldquocome up with anideardquo ldquosolve a problemrdquo or ldquoconsider a factrdquo weexperience these phenomena as events that are noless real than sensory or motor events Many so-called abstract concepts refer directly to cognitiveevents and states (decide judge recall think) or tothe mental ldquoproductsrdquo of cognition (idea memoryopinion thought) We propose that such conceptsare learned in large part by generalization acrossthese cognitive experiences in exactly the same wayas concrete concepts are learned through generaliz-ation across perceptual and motor experiences

Domain-general cognitive operations have beenthe subject of a very large number of neuroimagingstudies many of which have attempted to fractionatethese operations into categories such as ldquoworkingmemoryrdquo ldquodecisionrdquo ldquoselectionrdquo ldquoreasoningrdquo ldquoconflictsuppressionrdquo ldquoset maintenancerdquo and so on No clearanatomical divisions have arisen from this workhowever and the consensus view is that there is alargely shared bilateral network for what are variouslycalled ldquoworking memoryrdquo ldquocognitive controlrdquo orldquoexecutiverdquo processes involving portions of theinferior and middle frontal gyri (ldquodorsolateral prefron-tal cortexrdquo) mid-anterior cingulate gyrus precentralsulcus anterior insula and perhaps dorsal regions ofthe parietal lobe (Badre amp DrsquoEsposito 2007 DerrfussBrass Neumann amp von Cramon 2005 Duncan ampOwen 2000 Nyberg et al 2003) We hypothesizethat retrieval of concepts that refer to cognitivephenomena involves a partial simulation of thesephenomena within this cognitive control networkanalogous to the partial activation of sensory andmotor systems by object and action concepts

Emotion and evaluation components

There is strong evidence for the involvement of aspecific network of regions in affective processing ingeneral These regions principally include the orbito-frontal cortex dorsomedial prefrontal cortex anteriorcingulate gyri (subgenual pregenual and dorsal)inferior frontal gyri anterior temporal lobes amygdalaand anterior insula (Etkin Egner amp Kalisch 2011Hamann 2012 Kober et al 2008 Lindquist WagerKober Bliss-Moreau amp Barrett 2012 Phan WagerTaylor amp Liberzon 2002 Vytal amp Hamann 2010) Acti-vation in these regions can be modulated by theemotional content of words and text (Chow et al2013 Cunningham Raye amp Johnson 2004 Ferstlet al 2005 Ferstl amp von Cramon 2007 Kuchinkeet al 2005 Nakic Smith Busis Vythilingam amp Blair2006) However the nature of individual affectivestates has long been the subject of debate One ofthe primary families of theories regarding emotionare the dimensional accounts which propose thataffective states arise from combinations of a smallnumber of more fundamental dimensions most com-monly valence (hedonic tone) and arousal (Russell1980) In current psychological construction accountsthis input is then interpreted as a particular emotion

COGNITIVE NEUROPSYCHOLOGY 141

using one or more additional cognitive processessuch as appraisal of the stimulus or the retrieval ofmemories of previous experiences or semantic knowl-edge (Barrett 2006 Lindquist Siegel Quigley ampBarrett 2013 Posner Russell amp Peterson 2005)Another highly influential family of theories character-izes affective states as discrete emotions each ofwhich have consistent and discriminable patterns ofphysiological responses facial expressions andneural correlates Basic emotions are a subset of dis-crete emotions that are considered to be biologicallypredetermined and culturally universal (Hamann2012 Lench Flores amp Bench 2011 Vytal amp Hamann2010) although they may still be somewhat influ-enced by experience (Tracy amp Randles 2011) Themost frequently proposed set of basic emotions arethose of Ekman (Ekman 1992) which include angerfear disgust sadness happiness and surprise

There is substantial neuroimaging support for dis-sociable dimensions of valence and arousal withinindividual studies however the specific regionsassociated with the two dimensions or their endpointshave been inconsistent and sometimes contradictoryacross studies (Anders Eippert Weiskopf amp Veit2008 Anders Lotze Erb Grodd amp Birbaumer 2004Colibazzi et al 2010 Gerber et al 2008 P A LewisCritchley Rotshtein amp Dolan 2007 Nielen et al2009 Posner et al 2009 Wilson-Mendenhall Barrettamp Barsalou 2013) With regard to the discreteemotion model meta-analyses have suggested differ-ential activation patterns in individual regions associ-ated with basic emotions (Lindquist et al 2012Phan et al 2002 Vytal amp Hamann 2010) but there isa lack of specificity Individual regions are generallyassociated with more than one emotion andemotions are typically associated with more thanone region (Lindquist et al 2012 Vytal amp Hamann2010) Overall current evidence is not consistentwith a one-to-one mapping between individual brainregions and either specific emotions or dimensionshowever the possibility of spatially overlapping butdiscriminable networks remains open Importantlystudies of emotion perception or experience usingmultivariate pattern classifiers are providing emergingevidence for distinct patterns of neural activity associ-ated with both discrete emotions (Kassam MarkeyCherkassky Loewenstein amp Just 2013 Kragel ampLaBar 2014 Peelen Atkinson amp Vuilleumier 2010Said Moore Engell amp Haxby 2010) and specific

combinations of valence and arousal (BaucomWedell Wang Blitzer amp Shinkareva 2012)

The semantic representation model proposed hereincludes separate components reflecting the degree ofassociation of a target concept with each of Ekmanrsquosbasic emotions The representation also includes com-ponents corresponding to the two dimensions of thecircumplex model (valence and arousal) There is evi-dence that each end of the valence continuum mayhave a distinctive neural instantiation (Anders et al2008 Anders et al 2004 Colibazzi et al 2010 Posneret al 2009) and therefore separate components wereincluded for pleasantness and unpleasantness

Drive components

Drives are central associations of many concepts andare conceptualized here following Mazlow (Mazlow1943) as needs that must be filled to reach homeosta-sis or enable growth They include physiological needs(food restsleep sex emotional expression) securitysocial contact approval and self-development Driveexperiences are closely related to emotions whichare produced whenever needs are fulfilled or fru-strated and to the construct of reward conceptual-ized as the brain response to a resolved or satisfieddrive Here we use the term ldquodriverdquo synonymouslywith terms such as ldquoreward predictionrdquo and ldquorewardanticipationrdquo Extensive evidence links the ventralstriatum orbital frontal cortex and ventromedial pre-frontal cortex to processing of drive and reward states(Diekhof Kaps Falkai amp Gruber 2012 Levy amp Glimcher2012 Peters amp Buumlchel 2010 Rolls amp Grabenhorst2008) The proposed semantic components representthe degree of general motivation associated with thetarget concept and the degree to which the conceptitself refers to a basic need

Attention components

Attention is a basic process central to information pro-cessing but is not usually thought of as a type of infor-mation It is possible however that some concepts(eg ldquoscreamrdquo) are so strongly associated with atten-tional orienting that the attention-capturing eventbecomes part of the conceptual representation Acomponent was included to assess the degree towhich a concept is ldquosomeone or something thatgrabs your attentionrdquo

142 J R BINDER ET AL

Attribute salience ratings for 535 Englishwords

Ratings of the relevance of each semantic componentto word meanings were collected for 535 Englishwords using the internet crowdsourcing survey toolAmazon Mechanical Turk ( httpswwwmturkcommturk) The aim of this work was to ascertain thedegree to which a relatively unselected sample ofthe population associates the meaning of a particularword with particular kinds of experience We refer tothe continuous ratings obtained for each word asthe attributes comprising the wordrsquos meaning Theresults reflect subjective judgments rather than objec-tive truth and are likely to vary somewhat with per-sonal experience and background presence ofidiosyncratic associations participant motivation andcomprehension of the instructions With thesecaveats in mind it was hoped that mean ratingsobtained from a sufficiently large sample would accu-rately capture central tendencies in the populationproviding representative measures of real-worldcomprehension

As discussed in the previous section the attributesselected for study reflect known neural systems Wehypothesize that all concept learning depends onthis set of neural systems and therefore the sameattributes were rated regardless of grammatical classor ontological category

The word set

The concepts included 434 nouns 62 verbs and 39adjectives All of the verbs and adjectives and 141 ofthe nouns were pre-selected as experimentalmaterials for the Knowledge Representation inNeural Systems (KRNS) project (Glasgow et al 2016)an ongoing multi-centre research program sponsoredby the US Intelligence Advanced Research ProjectsActivity (IARPA) Because the KRNS set was limited torelatively concrete concepts and a restricted set ofnoun categories 293 additional nouns were addedto include more abstract concepts and to enablericher comparisons between category types Table 3summarizes some general characteristics of the wordset Table 4 describes the content of the set in termsof major category types A complete list of thewords and associated lexical data are available fordownload (see link prior to the References section)

Query design

Queries used to elicit ratings were designed to be asstructurally uniform as possible across attributes andgrammatical classes Some flexibility was allowedhowever to create natural and easily understoodqueries All queries took the general form of headrelationcontent where head refers to a lead-inphrase that was uniform within each word classcontent to the specific content being assessed andrelation to the type of relation linking the targetword and the content The head for all queriesregarding nouns was ldquoTo what degree do you thinkof this thing as rdquo whereas the head for all verbqueries was ldquoTo what degree do you think of thisas rdquo and the head for all adjective queries wasldquoTo what degree do you think of this propertyas rdquo Table 5 lists several examples for each gram-matical class

For the three scalar somatosensory attributesTemperature Texture and Weight it was felt necess-ary for the sake of clarity to pose separate queriesfor each end of the scalar continuum That is ratherthan a single query such as ldquoTo what degree do youthink of this thing as being hot or cold to thetouchrdquo two separate queries were posed about hotand cold attributes The Temperature rating wasthen computed by taking the maximum rating forthe word on either the Hot or the Cold query Similarlythe Texture rating was computed as the maximumrating on Rough and Smooth queries and theWeight rating was computed as the maximum ratingon Heavy and Light queries

A few attributes did not appear to have a mean-ingful sense when applied to certain grammaticalclasses The attribute Complexity addresses thedegree to which an entity appears visuallycomplex and has no obvious sensible correlate in

Table 3 Characteristics of the 535 rated wordsCharacteristic Mean SD Min Max Median IQR

Length in letters 595 195 3 11 6 3Log(10) orthographicfrequency

108 080 minus161 305 117 112

Orthographic neighbours 419 533 0 22 2 6Imageability (1ndash7 scale) 517 116 140 690 558 196

Note Orthographic frequency values are from the Westbury Lab UseNetCorpus (Shaoul amp Westbury 2013) Orthographic neighbour counts arefrom CELEX (Baayen Piepenbrock amp Gulikers 1995) Imageability ratingsare from a composite of published norms (Bird Franklin amp Howard 2001Clark amp Paivio 2004 Cortese amp Fugett 2004 Wilson 1988) IQR = interquar-tile range

COGNITIVE NEUROPSYCHOLOGY 143