Cogni&onandLanguage:InterfacesandMechanismsinCommon

TimShalliceEx-UniversityCollegeLondonEx-SISSATrieste

Approachesinpresenta&on

EmpiricalData

•  Neuropsychology:forra&onaleandmethodologicalanalysis:

•  SeeShalliceCogni&veNeuropsychology2015

Theore0calframework

•  Connec&onism(butnotminimalistvariety):

•  SeeShallice&Cooper–TheOrganisa&onofMindOxford,2011.

Syndromestobediscussed

•  1.DynamicAphasia•  2.Seman&cDemen&a,Category-SpecificSeman&cImpairments

•  3.PhonologicalOutputBufferandGraphemicOutputBufferImpairments

Issues•  1..Whatarethe

interfacesbetweenlanguageandnon-languageprocesses(mainlysyndrome1;somesyndromes2)

•  2.Towhatextentdoeslanguageusemechanismsspecifictoitselfandtowhatextentdogeneral-purposecor&calprinciplesapply?(mainlysyndromes3;somesyndromes2)

•  1.DynamicAphasia•  2.Seman&cDemen&a,

Category-SpecificSeman&cImpairments

•  3.PhonologicalOutputBufferandGraphemicOutputBufferImpairments

DynamicAphasia(Luria)

•  Subtypeoftranscor&calmotoraphasia•  Luria(1970)reportedthatwhenpa&entswithdynamicaphasiawereengagedinataskrequiringthemtotellastorytheycomplainedofan‘...emp&nessinthehead...’asiftheirthoughts‘...stands&llanddon’tmove...’

•  Ontheotherhandweresaidtoanswerques&onsappropriately

DynamicAphasicPa&entsType1•  ANG(Robinson,Blair&Cipolo0,1998)

–  59yrold,female,re&redgene&cslecturer–  malignantmeningioma.

•  CH(Robinson,Shallice&Cipolo0,2005)–  60yrold,male,re&redengineer–  Frontotemporaldemen&a→non-fluentprogressiveaphasia.

Quan&ta&veProduc&onAnalysis:Berndtetal,2000(SamplefromDescrip&onofComplexScenes)

Type 1 Controls

ANG

CH

Speech Rate (words per min)

29.2 12.0 160.8 (SD 37)

Proportion of Verbs (V/N+V)

0.39 0.44 0.48 (SD 0.06)

TheliflespontaneousspeechANGdidproducewaswellar&culatedwithnormalprosodyandcorrectsyntac&cstructure.(Seelater)

TwoDynamicAphasiaPa&ents:lesionsites

ANG:frontalmeningioma * anterior part of the lehinferiorfrontalgyrus*BA45+++,BA44++

CH:focalatrophy*fronto-temporaldemen&a*lehBA44“moderatelyatrophic”lehBA43,45,46“mildlyatrophic”;righmrontalnormal

Le:InferiorFrontal+44,45and47

LanguageExamina&on:WordProcessing

ANG CH Word Comprehension Synonyms Test British Picture Vocab. Scale

25-50th %ile

-

75-90th %ile

145/150

Oral Naming Graded Naming Test

75th %ile

75-90th %ile

Repetition Single Words

30/30

169/180

Reading NART 75-90th %ile

25-50th %ile

Problemspecifictolanguage

•  Fluencytasks–generateasmanyXaspossibleinafixed&meeg60s

•  1.Verballyspecified•  2.Designsconsis&ngof4lines•  3.Gesturesusingtheupperlimbs•  4.Movementsofajoys&ck

CH:NonverbalGenera&on–purelyverbalproblem Total Number Generated

Gesture Fluency eg Make different positions with your hands.

CH Controls (n=10)

a. Meaningful movements b. Meaningless movements

13 26

16.0 (4.9) 22.0 (5.8)

Design Fluency eg Draw abstract designs with 4 straight lines. a. Free Condition 11 11.8 (4.4)

b. Fixed Condition 17 12.6 (4.3)

Random Motor Movement Generation eg Move joystick at tone.

% Total Responses (s.d.) CH Controls (n=10)

4 Options: Up/Down/Left/Right

Repeats 39 26.2 (5.8)

Opposites 24 27.0 (8.6)

Other 37 46.8 (10.0)

TwoDynamicAphasiaPa&ents:lesionsites

ANG:frontalmeningioma * anterior part of the lehinferiorfrontalgyrus*BA45+++,BA44++

CH:focalatrophy*fronto-temporaldemen&a*lehBA44“moderatelyatrophic”lehBA43,45,46“mildlyatrophic”;righmrontalnormal

Le:InferiorFrontal+44,45and47

PhonemicversusDesignandGestureFluency(RobinsonetalBrain2012)

02468101214161820

LeftLat RightLat

SupMedial

Healthy

phonemicdesigngesture

40 frontal patients: Specific Left Lateral problem in fluency is restricted to phonemic fluency

SentenceGenera&onTasks Type1 ANG CH

SentenceGenera&onfrom: a singlecommonworde.g.phone2/1511/20pictureofsingleobjecte.g0/6nt

pictureofscenee.g.34/3420/20 e.g.(ANG)

“aboyandagirlridinganelephant”Reporter’sTest 14/1415/15

(TokenTestinreverse)e.g.(ANG)“Youhaveselectedfoursquaresandfourcircles.Youhavetappedthecirclesharderthanthesquares”

DynamicAphasia;Func&onalLocalisa&on

•  Levelt’smodelofspeechproduc&on

•  GivenANGisnotagramma&candhasnophonologicalproblems

•  Mostplausibleloca&on-Conceptualiser

Jackendoff(2002)“BeethovenlikesthatSchubertwritesmusic”

Phrasal semantics – Preverbal message- impaired in dynamic aphasia type I

LIFG UNIFICATION – Binding of content to an abstract (programmable?) node in a hierarchical structure

Badre&D’Esposito(JCN2007)•  Fourtypesofexperiment•  Eachtype–2linesonthe

diagramegAandB,CandD....•  Foreachtypeeither1,2or4

choicesofresponseindifferentblocksoftrials

•  (Forthefirstlineofeachexp(i.e.A,C...)theresponsesindicatedareforchoicesetof2)

•  Whichaspectofs&mulusiscri&calonthattrialisdeterminedbythecolouroftheborder

Badre&D’Esposito(2007)

•  Asthedecisionbecomesmoreabstractcri&calregionbecomesmoreanterior

•  ieA->B->C->D

SentenceGenera&onTest:S&muliandPredic&ons

Generateawholesentencethatincludestheword…

Frontal Patients Posterior Patients Healthy Controls

LIFG Non-LIFG

High Frequency Words glass

X

√

√

Low Frequency Words kite

√

√

√

Proper Nouns Gandhi

√

√

√

Selection Demands

* = p < 0.001, LIFG patients vs. Non-LIFG patients & Controls

Mapsintoselec&ondemandsstudiesinfunc&onalimaging

•  EgThompson-Schilletal1997

•  Badreetal2005–judgementspecificity

Crescen&nietal2009

•  Genera&onofnoungivenverbandviceversa

•  Lowselec&ondemands(LS)egcan->todrink*54%vs.can->toopen9%

•  Highselec&ondemands(HS)eglamp>toturnon46%vslamp->tolightup37%

•  Alsoweak(WA)vsstrong(SA)associa&vestrength

TransfromItalian

DynamicAphasiaStudiesConclusion:Selec&onandSentenceGenera&onI

1.  Low frequency words or proper names – have smaller number of associations so much more limited competition of associations than for high frequency

2.  Plausibly due to an analogue of the cue-overload (Watkins & Watkins, 1976) or fan effect in memory: A-B A-C vs A-B D-E

DynamicAphasiaStudiesConclusion:Selec&onandSentenceGenera&onII

3. Effects occurring at the conceptualiser level (on Levelt’s framework) and appear to be specific to language. Hence at the level of generation of preverbal message (which may be misnamed!). Note from a linguistic perspective – it plausibly involves Jackendoff’s abstract semantic hierarchy – events, situations, objects 4. Yet a simple phenomenon known from the memory literature - cue overload - also operates exceedingly strongly within the highest level of the language production system – presumably because it derives from a very general property of neural nets, out of which the language system is built.

Thecomprehensioninterface-Seman&cs,languageandembodiment:

twosyndromes

•  Seman&cdemen&a •  Category-specificdisorders

AB(Warrington1975)–spontaneouswri&ng

HowdidWarrington(1975)detectthescien&ficinterestofABini&allyclinically?

•  ProgressiveMatrices–top5%ile

•  WAIS–PictureArrangementsubtest–secondeasiestitem;whatismissing?

•  AB–“Ihaveneverbeeninterestedindogs”

Seman&cDemen&a:Dissocia&ons•  1.IntactIQ(egRaven’sMatrices)•  2.Intactsensoryandperceptualprocesses(priortolevelofmeaning)

•  3.Intactshort-termmemory(egspan)•  4.Intactepisodicmemoryofnon-seman&ccharacteris&cs(Hodgesgroup)

•  5.Rela&velyintactsyntax,phonologyandorthography

•  BUTalltypesofknowledgeegofthesignificance(andname)ofobjects,wordmeaningsetcgrosslyreduced

Seman&cdemen&aasasyndromeegHodgesetal(1992)

•  5Demen&ngPa&ents•  EgPicturesor&ng:threelevels•  1.LivingthingvsArtefact•  2.Categories:landanimalvsseacreaturevsbird

•  3.Afribute/Subordinate:Bri&shvsnon-Bri&shanimal;electricalvsnon-electricalitem

,

Seman&cdemen&aasafunc&onalsyndromeegHodgesetal(1992)

•  5Demen&ngPa&ents•  Downonpurelyverbalseman&cmemoryteststoo(egdefini&ons;categoryfluency)

controls (mean, SD)

Mionetal(Brain2010)•  Differencesbetween

normalisedcerebralmetabolicrateofglucosebetweenseman&cdemen&apa&ents(n=21)andhealthycontrol

•  Glucoseisaprimarysourceofenergyforthebrain,andhenceitsavailabilityinfluencespsychologicalprocesses.

Rogersetal(2004)‘Hub’modelofseman&cs

•  Fullconceptualmodelandpartsimulated

•  FromLambon-Ralph,LoweandRogers(Brain2007)

•  Seman&cs–heteromodal(ieequallyverbal/non-verbal)

Rogersetal‘hub’`modelsimula&on

Semantic dementia patients and model (with disconnection lesions) on picture-naming

Psychol Rev 2004

Aproblemforthehubmodel:visualseman&cs-RM(Lauro-Grofoetal,1997)

•  Seman&cdemen&a–lehtemporalmoreatrophiedthanright

•  Whichof3items(egdetergent,cartaxs&cker,scarf)goeswithanother(egwindscreenwiper):

Verbal30%(chance);Visual69%

Mionetal(Brain2010):Rela&velyunilateralseman&cdemen&a:lehvsright

•  Camelandcactustestof‘visualseman&cs’Presentedinpictureswitha4-alterna&veforcedchoiceegforcamel:cactus(thetarget),tree,sunflower,orrose.

Verbal semantics (involving object naming and category fluency) specifically correlated with analogous left temporal region

Forrightgroup

Abiggerproblemforthehubmodel:Herpessimplexencephali&sandcategoryspecificity•  Veryrare•  Veryrapid&me-courseofillness•  Priorto1975mostpa&entsdied•  Acyclovirstoppeddiseasebutmedialtemporallobesohengravelydamaged

•  Rapidityofillnessprobablymeansthatliflereorganisa&onoffunc&onoccurs(unlikelowgradeglioma)

•  Butrestofbrainunaffectedbydiseasewillbeingoodshape(unlikestroke)

Non-Classical(Strong)Dissocia&ons–herpessimplexencephali&s

0

10

20

30

40

50

60

70

80

JBR SBY

ObjectsAnimalsFoods

From original descriptions in Warrington & Shallice Brain 1984

Proposal : Sensory Quality vs Functional Knowledge

TASK – give distinguishing meaning of (as assessed by independent judges)

Gainoy(Cortex2000)

•  20+herpesencephali&cpa&entsreviewedwithasimilarpafernacrosscategories–‘categoryspecificity’

•  Nowconsiderablymore(seealsoCapitanietalCogni&veNeuropsychology2003)

•  Prototypiclesions(generallylarge)–bilateralanteriorinferiortemporallobe,par&cularlymedial–overlapslesionssuteforseman&cdemen&a

TyleretalJCogNeuro2004

Red = Domain level naming

Green = Basic level Naming

Normal functional imaging superimposed on herpes patients

Black area – lesion of herpes patients

Buthowtoaccountforcatspeconthehubmodelespeciallyasherpesandseman&c

demen&ahavesimilarlesionsites?

•  LambonRalphetal2007

•  Twodifferenttypesofdamagetothehubitself

•  NO-Adhocassump&onsandunsuitablemodelling

Bramba&etal(2006)&Campanellaetal(2010)Living(L)differsfromNon-Living(NL)

Dementing patients L>NL Udine Tumour patients

NL>L

L>NL

MUCHBETTER:Chen(2016):sensory-func&oninthespokes

Cri&calregions–apartfromthehub–derivedfrommetaanalysisofcatspecfunc&onalimagingeffectsConnec&onsbasedonprobabilis&ctractography.Simula&on(pa&ent)forverbalinput

ImpairmentsofLanguageOutputBuffers

•  Phonological:Caramazzaetal86

•  IGR-nonwords

•  Lengtheffects•  Errors–phonemesubs&tu&ons,inser&ons,dele&onsandtransposi&ons

•  1,2ormoreinnonword•  Essen&allythesamepafernwithrepe&&on,readingaloudandwri&ng

•  Graphemic:Caramazzaetal,87:Caramazza&Miceli90

•  LB-words

•  Lengtheffects•  Errors–lefersubs&tu&ons,inser&ons,dele&onsandtransposi&ons

•  1,2ormoreinword•  U-shapedserialposi&oncurves

ImpairmentofPhonologicalOutputBuffer(Caramazza-Miceliposi&on)

•  Papermethodologicallyhighlyinnova&ve

•  Similareffectsacross3differentinput-outputtasks(readingaloud,repe&&on,wri&ngtodicta&on)

•  Indicatesdeficitbeforetheinternalprocessingtrajectoriesofthethreetasksseparate

•  Togetherwithnatureoferrors

•  PhonologicalOutputBuffer

•  TwoPhonologicalOutputBufferpa&ents

•  TwoPhonologicalOutputBufferpa&ents:%ofdifferenttypesoferrors

EarlyGraphemicBufferpa&ents:ErrorSerialPosi&onCurves-LBandAS(Jonsdoyretal1996)

AS:Errorsinwri&ng(black)andoralspelling(white)

LB:Errorsinwri&ng

Compe&&veQueueingMechanism(Houghton,1990)

(a)ThestructureofthemechanismfromI(ini&al)andE(end)nodestotheCompe&&veFilter.(b)Ac&va&onofI/Enodesover&me–bothatlearningandretrieval.Inthissimula&onseparatenetforeachword.

Compe&&veQueueingDynamics•  CQac&va&ondynamicsof

nodesrepresen&nglefersduringproduc&onoftheword“CINEMA”.Theac&va&onlevelofeachleferisshownateach&mestepduringproduc&onoftheword.Thetraceforeachleferislabelledatthepointwhereitwinsthecompe&&onforoutput.

•  Notethatpost-selec&oninhibi&onpreventsimmediaterepe&&onsofaleferbeinglearned

•  Henceanaddi&onalgeminatemechanismisrequired

CQmodelofspellingofHoughton,Glasspool&Shallice(1994)

•  Notetheaddi&onofageminate(doublingnode)

GraphemicBufferPa&entsandCQmodel–effectsof(i)wordlength;(ii)

errortype

Glasspooletal(2006)distributedcompe&&vequeuing(CQ)model

Seman&cac&va&ngsystemtrainedfor400wordsusingBP–when95%correct,weights‘frozen’.Restofnetworkthentrainedwithseman&cinputusingalazylearningrule-weightschangedonlyforaleferincorrectlyselected

TypeAandTypeBGraphemicOutputBufferImpairment–SerialPosi&onCurves

TypeBalsomake‘fragment’errors.AlsoTypeBtendtoshowdeepdysgraphiccharacteris&cs

Glasspooletal(2006)distributedcompe&&vequeuing(CQ)model

TYPEB

TYPEA

Hartley-HoughtonCQmodelofthephonologicaloutputbuffer(single

syllableversion)

(a)Givestheoverall2-routeCQmodelarchitecturefornovelphonologicalforms(egrepe&&onofnon-words)(b)givesthein-builtinternalsyllabicrepresenta&onsforthestructurepathway.Aseachphonemepresented,onlythephonemerepsinthenextwithin-syllableslotareini&allycandidates

%ofsubs&tu&onerrors(asopposedtoinserts,deletes’transla&ons)ieerrorsthatretainwithin-syllablestructure

•  Phonologicaloutputbuffer

•  IGR,LT–75%,72%

•  Graphemicoutputbuffer

•  JH,LB,AS,HE-45%,53%,32%,31%

Effectsof(syllabic)structureweakerforgraphemicoutputbufferthanforphonologicaloutputbuffer->greater%oferrorsthatbreakstructure.

Conclusions1•  Cogni&on-Languageinterfaceforconcretenouns–hub

plusspokes•  Specificallynon-languageandmainlylanguagesubsystems

interconnectedinacomplexfashion.Butconceptofembodimentoverlysimplis&candinadequate(lehtemporalhub).

•  Cogni&on-Languageinterfaceforabstractnounsandverbs–muchlessclear(butseeShallice&Cooper,2013)

•  Theprocessesunderlyingtheproduc&onofLevelt’spreverbalmessageremainsprefyvirginterritorybutitseemstoexistasaninputtotherestofthelanguagesystemthatcanbeselec&velyimpaired.

Conclusion2

•  Thelanguagesystemusesthebasicneuralarchitectureoftherestofcogni&on(eghub+spokemodel)

•  BUTlanguagesubsystemshaveaddi&onalsubsystem-specificelements(egstructuralpathwayinoutputbuffermodels)

•  Someoftheseaddi&onalelementsmustbelearned(wri&ngmodels)butsomeareprobablyinatelyspecified(egHartley-Houghtonassump&onsofstructuralpathwayforthephonologicaloutputbuffer).

•  Syntaxwillrequirealotofspecialpurposeaddi&ons.

OutputBufferModelling

•  Needtocombinesymbolicandconnec&onistaccounts

•  Hasbeendonefothephonologicaloutputbuffer(Hartley&Houghton,1996)

•  Remainstobedoneforgraphemicoutputbuffer(tomyknowledge)

Canessaetal(Cer.Cor2008)Manipulability(A)vsFunc&on(F)Judgements

ROIanalysesininferiorparietal(leh)toparieto-occipitalregions(right).NOTEA>Fasonegoesmoreanterior

ANDfunc&onlocalisa&ondoesnotfitLivingSuperioritypa&ents