SomaticMarkerHypothesisTashaPoppa

AffectiveComputingGuestLectureFall2017

Outline• Historicalcontext• SomeUsefulFunctionalNeuroanatomy• VMPFCpatients:Clinicalobservations• SomaticMarkerHypothesis• TestingTheSMH• IowaGamblingTask(IGT)• EmpiricalStudies• ComputationalModeling

HistoricalContext:OntheFunctionsofthePrefrontalCortex

• PhineasGage’sfamoustampingrodaccidentof1848

• Damagedidnotresultinlossofintellect,motor,languageabilities

• Instead,demonstratedlossofsocialinhibitions/inappropriatebehavior

• Exactlocationofthedamageisuncertain• Ventralvs.Dorsal?• H.andA.Damasio revivedinterestinthestudyoftheprefrontalcortex

BriefTouroftheFrontalLobes

AVentralPerspective

• Theorbitofrontalcortex• LateralandMedialdivisions

• Sulcus/Gyrusrectusdefinestheboundary

AVentralPerspective

• Lateralorbitalregions:• Multimodalsensoryintegration• Esp.sensitivetorewardvalueof

sensorystimuli• Associative

• Medialorbitalregions:• Motor,butofvisceraloutputs• PAG&Hypothalamus

• Medialnetworkoverlapswithlateralnetwork• Sensory-visceromotor integration

Ongur &Price,2000

VMPFCPatients:ClinicalObservations• Normal:

• Semantic/factualknowledge• Intellect(IQ)• Reasoning(incl.aboutmorals)• Language• WorkingMemory• Attention

VMPFCPatients:ClinicalObservations• Abnormal:

• Difficultyplanningdailyactivities• Future-orientedplanning• Emotionallylabile/blunted• Behaviors/choicesleadtofinanciallosses,lossoffriends/family/socialstature

• Acquiredpsychopathy/sociopathy

• Asystems-levelneuroanatomicalandcognitiveframeworkfordecision-makinganditsinfluencebyemotion

• SMHpartofatraditionofmodelsofemotionwhichpositthatemotion/feelingsarerepresentationsofembodiedchangesinvisceral/musculoskeletalstates

• WilliamJames &CarlLange(late1800s)

• ThemodelalsoechoeswiththewritingsofNauta (1971)• Affectivereferencepoints,“navigationalmarkers”,assistadaptivebehaviorwrt.frontallobefunction

SomaticMarkerHypothesis(Damasio,1991;Damasio,etal.,1994)

• Emotion:• “…changesinbodyandbrainstatestriggeredbyadedicatedbrainsystemthatrespondstospecificcontentsofone’sperceptions,actualorrecalledrelativetoaparticularobjectorevent”(Bechara andDamasio,2005)

• Feelings:• Concernshowbodilystatesarerepresentedinsomatosensingcortices(S1,S2,operculum,insula)

SomaticMarkerHypothesis(Damasio,1991;Damasio,etal.,1994)

• Endocrinerelease:• Epinephrine/Adrenaline• Oxytocin

• Autonomicmodifications:• Cardiac• Vascular• Sweating

• Musculoskeletal:• Posture• Facialexpression• Freeze/Flight/Flight

SomaticMarkerHypothesis(Damasio,1991;Damasio,etal.,1994)

• Neurotransmittersystems:• Dopamine• Serotonin• Acetylcholine

• Alterationinsomatosensorymaps• Esp.insularcortex

• Afferentfeedbacktobrainfrombody• Vagus nerve• Spinalpathways

BODY BRAIN

• Inductionofsomaticstates:

• PrimaryInducers• Innateorlearnedstimulithatgeneratepleasurableoraversivefeelingstates• Automatically,obligatorilyelicitbodilyresponse• Amygdala

• SecondaryInducers• Generatedbyrecallofpersonalorhypotheticalevent• i.e.thoughtsabout/memoriesofaprimaryinducer• VMPFC

• Secondaryinductionreliesonintactmechanismsofprimaryinduction• Afterasomaticstatehasbeentriggeredbyaprimaryinducerandexperiencedatleastonce,aneuralpatternforthissomaticstateisformed

SomaticMarkerHypothesis(Damasio,1991;Damasio,etal.,1994)

• VMPFC:• Triggerstructureforsomaticstatesfromsecondaryinducers

• VMPFCneuronensemblescouple:• Memories/sensoryinputfromassociationcortices• Effectorstructurestoactualizeasomaticstate• Neuralpatternsrelatedtofeelingstates

• Insum:“whatitfeelsliketobeinagivensituation”

SomaticMarkerHypothesis(Damasio,1991;Damasio,etal.,1994)

• The“bodyloop”mechanismofsomaticmarkers

SomaticMarkerHypothesis(Damasio,1991;Damasio,etal.,1994)

• Insum:• Reasoningandknowledgeareinsufficientforadvantageouschoicesunderuncertainty

• VMPFCpatients’inabilitytomakeadvantageousdecisionsduetodefectinemotionalmechanismthatsignalsprospectiveconsequencesofanaction/assistsinselectionofchoices

SomaticMarkerHypothesis(Damasio,1991;Damasio,etal.,1994)

TestingtheSMH:IGT

• 4-armedbandit• DecksA&B• DecksC&D

• DecksA&C• DecksB&D

NegativeExpectedValue

PositiveExpectedValue

HighVariance

LowVariance

• PatientswithlesionstoVMPFCandAmygdaladonotlearntoassociatedeckswithoutcomes

• Butthroughdifferentmechanisms?

TestingtheSMH:IGT

TestingtheSMH:IGT

• SkinConductanceResponses(SCR)• Puresympatheticinnervationtoeccrineglandsfoundonpalmarsurface

• Correlateswitharousal(butnotvalence)

• Amygdaladamage:• Noarousaltoreward/punishment• Noanticipatoryresponse

• VMPFCdamage:• Arousaltoreward/punishment• Noanticipatoryresponse

IGTsensitivetoVMPFClesions

• N=344patients

• Variousneuropsychologicaltestsofexecutivefunction:• TrailMakingTest(TMT)• WisconsinCardSortingTask(WCST)• STROOP• ControlledOralWordAssociation

(COWA)• IGT

• OnlydamagetoVMPFCissignificantlylinkedtoIGTperformance

AdditionalEmpiricalStudies

• Applicationstoclinicalpopulations:

• GreaterdopaminereleaseinventralstriatuminpathologicalgamblerswhileplayingIGT(Linnet,etal.,2010)

• OpiateuserstendtopreferdeckC(frequentreward/negativeEVdeck)(Upton,etal.,2012)

• WorseperformanceinObsessiveCompulsiveDisorder(FilardidaRocha,etal.,2011)

ReinforcementLearningModelofIGT:Expectancy–ValenceLearningModel(Busemeyer andStout,2002)

• Lesions,variouspsychiatric,otherneurologicalconditionscangenerateimpairmentsontheIGT

• IGTisacomplextaskthatmayrequireinvolvementofanumberoflatentcognitiveprocesses

• Scoringcriterionof[(C+D)– (A+B)]isnotverysensitiveatdistinguishingtheunderlyingsourceofthedeficit

• Structuralmodelsthatformalizelatentprocessesinmathematicaltermsmayrevealwheretheimpairmentoriginates

ReinforcementLearningModelofIGT:Expectancy–ValenceLearningModel(Busemeyer andStout,2002)

• EVLmodelhas3freeparameters(w, c,anda)thatcontrolchoicebehaviorintheIGT

• Valence– parameterwdescribestherelativeweightgiventogainsvs.lossesinagiventrial

ReinforcementLearningModelofIGT:Expectancy–ValenceLearningModel(Busemeyer andStout,2002)

• Expectancylearning:adecisionmakerlearnsexpectanciesabouteachdeckoverthecourseofexperience

• Updatingmechanismwitharecency parametera:

ReinforcementLearningModelofIGT:Expectancy–ValenceLearningModel(Busemeyer andStout,2002)

• ThechoicemadeoneachtrialisaprobabilisticfunctionofdeckexpectanciesbasedonLuce’sratioofstrengthrule(1959)

• cisthesensitivityparameter.Determinessensitivityofchoiceprobabilitiestotheexpectancies• ThewayΘ isdefinedallowsforsensitivitytochangeovertrials• Explorevs.exploit

ReinforcementLearningModelofIGT:Expectancy–ValenceLearningModel(Busemeyer andStout,2002)

• 1)

• 2)

• 3)

Freeparameters:w,c,anda

ReinforcementLearningModelsofIGT

• Modelperformance:• Thickline=predictedchoices• Thinline=observedchoiceproportions

• Modelperformanceevaluatedagainstabaselinemodelthatsimulateschoicesbasedonobservedchoiceproportions usinglog-likelihoods

ReinforcementLearningModelsintheBrain

Ahn,etal.,2011

• Usesindividuallyorgroupidentifiedparameterstogeneratetrial-by-trialvaluesrelevanttothestageofdecision-making

• Typically:atdecisionperiod(D)&afterfeedback(F)fromachoice

• Atdecisionperiod,weighttrialsbyexpectedvaluesforthesubsequentlychosendeck

• Atoutcomestage,weighttrialbydifferencebetweenreceived– expectedreward(i.e.PredictionError)

• BOLD=D1*w1 +F1*z1+….+Dn*wn +Fn*zn +Error

ReinforcementLearningModelsintheBrain

Tanabe,etal.,2013

ReinforcementLearningModelsandtheSMH

• TheyarebehavioralmodelswithassumptionsaboutlatentprocessesHowdotheselatentprocessesactuallymapontobrainfunction?

• Theydonotdirectlyincorporaterelevantbiologicalsignals– e.g.frombrainandbody

• Don’tsaymuchabouttheSMHthiswaynorthatway• JustmodelsIGTasabasicreinforcementlearningprocess• Fromneuroimagingperspective,model-derivedregressors tendtocorrelatewithventralstriatumandposteriorVMPFC

ReinforcementLearningModelsandtheSMH

• Incorporatingbiologicalsignalsdirectlyintopredictivemodelsmaybemoreinsightful

• CorrespondmoredirectlytothetheoreticalprocessesdescribedinSMH

Questions?

References• Ahn,W.-Y.,etal.(2011)Amodel-basedfMRIanalysiswithhierarchicalBayesianparameterestimation.JournalofNeuroscience,Psychology,

andEconomics,4(2),95-110.• Bechara,A.;Damasio,A.R.(2005).Thesomaticmarkerhypothesis:Aneuraltheoryofeconomicdecision.GamesandEconomicBehavior.52,

336–372.• Busemeyer,J.R.,andStout,J.C.(2002).AContributionofCognitiveDecisionModelstoClinicalAssessment:DecomposingPerformance on

theBechara GamblingTask.PsychologicalAssessment,14,253-62.• Damasio,A.(1991).SomaticMarkersandtheGuidanceofBehavior.NewYork:OxfordUniversityPress.• Damasio,AntonioR.(2008)[1994].Descartes'Error:Emotion,ReasonandtheHumanBrain.RandomHouse.ISBN 978-1-4070-7206-7.• Damasio AR.(1996).Thesomaticmarkerhypothesisandthepossiblefunctionsoftheprefrontalcortex.PhilosophicalTransactionsofthe

RoyalSocietyB:BiologicalSciences.351,1413–1420.• Filardi daRocha,etal.(2011).Decision-makingimpairmentinobsessive-compulsivedisorderasmeasuredbytheIowaGamblingTask.Arq.

Neuropsiquiat.9,642-47.• Glascher,etal.(2012).Lesionmappingofcognitivecontrolandvalue-baseddecisionmakingintheprefrontalcortex.Proceedingsofthe

NationalAcademyofSciences,109,14686-86.• Linnet,J.,etal.(2010).DopaminereleaseinventralstriatumduringIowagamblingtaskperformanceisassociatedwithincreased

excitementlevelsinpathologicalgambling.Addiction, 106,383–390.• Tanabe,J.etal.(2012).ReducedNeuralTrackingofPredictionErrorinSubstanceDependentIndividuals.AmericanJournalofPsychiatry,

170,1356-63.• Upton,D.,etal.(2012).ComparingtheIowaandSoochowgamblingtasksinopiateusers.FrontiersinNeuroscience,doi:

10.3389/fnins.2012.00034