Prefrontal Cortex and Prefrontal Cortex and RewardReward

Chai BishopChai Bishop

5/35/3

I.I. Pathological Gambling is Linked to Pathological Gambling is Linked to Reduced Activation of the Mesolimbic Reduced Activation of the Mesolimbic

Reward SystemReward System

Jan Reuter, thomas Raedler, Michael Rose, Iver hand, Jan Jan Reuter, thomas Raedler, Michael Rose, Iver hand, Jan Glascher, christian BuchelGlascher, christian Buchel

Pathological gambling:Pathological gambling:

• -defined as gambling that interferes with -defined as gambling that interferes with interpersonal relationships and negatively affects interpersonal relationships and negatively affects financial and socioeconomic status financial and socioeconomic status

• -a common disorder with a prevalence of 1.6% in -a common disorder with a prevalence of 1.6% in adultsadults

Premise of study:Premise of study:– Linkage between Drug Addiction and Pathological GamblingLinkage between Drug Addiction and Pathological Gambling -symptoms-symptoms -mesolimbic reward system-mesolimbic reward system -drug self administration experiments-drug self administration experiments

*By analogy to drug addiction, it has been speculated that pathological gambling might also *By analogy to drug addiction, it has been speculated that pathological gambling might also be related to a deficiency of the mesolimbic dopaminergic reward systembe related to a deficiency of the mesolimbic dopaminergic reward system

Methods:Methods:-Using fMRI imaging and a guessing task known to -Using fMRI imaging and a guessing task known to robustly activate the ventral striatum (Knutson, robustly activate the ventral striatum (Knutson, Delgado)Delgado)- the experimenters studied 12 pathological - the experimenters studied 12 pathological gamblers (assessed with a questionnaire) and 12 gamblers (assessed with a questionnaire) and 12 closely matched healthy controlsclosely matched healthy controls

Results:Results:Fig.1Fig.1-there was significantly -there was significantly greater activity during greater activity during winning than during winning than during losing in the ventral losing in the ventral striatum which served striatum which served as verification that the as verification that the task robustly activated task robustly activated the ventral striatumthe ventral striatum-fewer voxels (volume -fewer voxels (volume pixel, smallest pixel, smallest distinguishable boxed distinguishable boxed shaped part of a 3D shaped part of a 3D image) were activated image) were activated in pathological in pathological gamblers than in gamblers than in controlscontrols-only the controls -only the controls showed additional showed additional activation in the activation in the ventromedial and ventromedial and ventrolateral prefrontal ventrolateral prefrontal cortex (VMPFC) (fig. cortex (VMPFC) (fig. 1c)1c)

Fig.2Fig.2-direct -direct comparison of comparison of both groups both groups showed showed significantly significantly lower activation lower activation of the right of the right ventral striatum ventral striatum in pathological in pathological gamblers than in gamblers than in controls (fig 2b)controls (fig 2b)-pathological -pathological gamblers showed gamblers showed significantly significantly weaker weaker activation in the activation in the VMPFC (fig 2c)VMPFC (fig 2c)

Additional data:Additional data:-Regression analysis to correlate signal changes in the ventral -Regression analysis to correlate signal changes in the ventral striatum with the severity of gambling in each patient showed striatum with the severity of gambling in each patient showed that the severity of gambling in pathological gamblers showed that the severity of gambling in pathological gamblers showed a significant negative correlation w/ the response in the right a significant negative correlation w/ the response in the right ventral striatum (r = -0.53) and the response in the VMPFC (r ventral striatum (r = -0.53) and the response in the VMPFC (r = -.53) (fig 2a,d)= -.53) (fig 2a,d)

-To ensure this result was not related to depression or -To ensure this result was not related to depression or smoking habits in some pathological gamblers 2 additional smoking habits in some pathological gamblers 2 additional analyses were performed:analyses were performed:

1. A categorical analysis comparing non-depressed 1. A categorical analysis comparing non-depressed pathological gamblers to a smaller control group perfectly pathological gamblers to a smaller control group perfectly matched for smoking that showed the same patternmatched for smoking that showed the same pattern

2. A correlation analysis w/o the depressed patients 2. A correlation analysis w/o the depressed patients that confirmed the initial data and actually showed an even that confirmed the initial data and actually showed an even stronger relationship between gambling severity and stronger relationship between gambling severity and hypoactivation of the right ventral striatum (r = -0.88) and hypoactivation of the right ventral striatum (r = -0.88) and VMPFC (r = -0.67)VMPFC (r = -0.67)

Conclusion:Conclusion:--through studying pathological gamblers and through studying pathological gamblers and

controls during a guessing game using fMRI, controls during a guessing game using fMRI, the experimenters observed a reduction of the experimenters observed a reduction of ventral striatal and ventromedial prefrontal ventral striatal and ventromedial prefrontal activation in the pathological gamblers that activation in the pathological gamblers that was negatively correlated with gambling was negatively correlated with gambling severely linking hypoactivation of these areas severely linking hypoactivation of these areas to disease severity which may represent the to disease severity which may represent the neural basis for impaired impulse control in neural basis for impaired impulse control in pathological gamblers. pathological gamblers.

- amongst pathological gamblers natural - amongst pathological gamblers natural enforcers are not strong enough for enforcers are not strong enough for dopamine to reach and maintain a dopamine to reach and maintain a homeostatic baseline level.homeostatic baseline level.

II. Prediction Error for Free Monetary Reward II. Prediction Error for Free Monetary Reward in the Human Prefrontal Cortexin the Human Prefrontal Cortex

N. Ramnani, R. Elliot, B.S Athwal, and R.E. PassinghamN. Ramnani, R. Elliot, B.S Athwal, and R.E. Passingham

Premise of the study:Premise of the study: -“The execution of goal-directed behavior is always -“The execution of goal-directed behavior is always

followed by monitoring for the successful followed by monitoring for the successful achievement of the goal. For this process to work achievement of the goal. For this process to work effectively, a representation of the expected goal effectively, a representation of the expected goal must be compared with the outcome.”must be compared with the outcome.”

-Midbrain dopamine systems of the primate brain send -Midbrain dopamine systems of the primate brain send projections to the basal ganglia and widespread projections to the basal ganglia and widespread regions of the frontal lobes (Ghashghaei and regions of the frontal lobes (Ghashghaei and Barbas). These routes are important for conveying Barbas). These routes are important for conveying reward-related information to frontostriatal circuitry reward-related information to frontostriatal circuitry involved in cognitive processing-the firing involved in cognitive processing-the firing characteristics of dopamine neurons are determined characteristics of dopamine neurons are determined by the ability of animals to predict rewards in by the ability of animals to predict rewards in advance of their occurrence and whether the advance of their occurrence and whether the predictions about outcomes are violated or verified.predictions about outcomes are violated or verified.

Primary AimPrimary Aim::-to localize regions of the human brain that were -to localize regions of the human brain that were responsive to violations of expectations related to responsive to violations of expectations related to freely delivered monetary rewardsfreely delivered monetary rewards

Methods:Methods: -event related fMRI in 6 human subjects to examine activity -event related fMRI in 6 human subjects to examine activity

related to the failure of expected rewards and the occurrence related to the failure of expected rewards and the occurrence of unexpected rewards, where outcomes were not contingent of unexpected rewards, where outcomes were not contingent on any behavioron any behavior

--An earlier study Ramnani and Miall (2003) used the same exact --An earlier study Ramnani and Miall (2003) used the same exact conditioned stimuli and reported that when conditioned conditioned stimuli and reported that when conditioned stimuli predict reward activity is found in the ventral pallidum stimuli predict reward activity is found in the ventral pallidum –on the basis of that study Ramnani et al. used a region of –on the basis of that study Ramnani et al. used a region of interest approach to test for the possibility that one could interest approach to test for the possibility that one could predict activity in the same region in this study for predict activity in the same region in this study for conditioned stimuli that predict rewards compared conditioned stimuli that predict rewards compared w/conditioned stimuli that predict nonrewards, even if the w/conditioned stimuli that predict nonrewards, even if the outcomes are as predictedoutcomes are as predicted

Results:Results:Fig 2.Fig 2.

-prediction errors evoke activity in 2 -prediction errors evoke activity in 2 separate frontotemporal networks, the separate frontotemporal networks, the anterior prefrontal cortex (a medial part of anterior prefrontal cortex (a medial part of the orbitofrontal cortex) and in lateral the orbitofrontal cortex) and in lateral parts of the dorsal prefrontal cortex (Table parts of the dorsal prefrontal cortex (Table 2, fig. 2B), 2, fig. 2B),

-unexpected rewards evoked phasic event -unexpected rewards evoked phasic event related decreases in the anterior prefrontal related decreases in the anterior prefrontal cortex and parahippocampal gyrus in most cortex and parahippocampal gyrus in most subjects but increases in the inferior subjects but increases in the inferior frontal sulcus and cerebellar cortexfrontal sulcus and cerebellar cortex

Fig 3.Fig 3.-This study found high activations within its region -This study found high activations within its region of interest, the ventral striatum which supports their of interest, the ventral striatum which supports their conclusion that conditioned stimuli that predict conclusion that conditioned stimuli that predict monetary reward cause activity changes in the monetary reward cause activity changes in the basal ganglia.basal ganglia.

Conclusion:Conclusion:--Projections between orbital and polar areas of the prefrontal cortex and Projections between orbital and polar areas of the prefrontal cortex and

areas of the temporal lobes found in primates (Barbas et al.) represent areas of the temporal lobes found in primates (Barbas et al.) represent important connections that enable visual information from the temporal important connections that enable visual information from the temporal lobe ‘object’ processing pathways to reach the prefrontal cortex, nearly lobe ‘object’ processing pathways to reach the prefrontal cortex, nearly all fibers that connect the temporal pole with the prefrontal cortex pass all fibers that connect the temporal pole with the prefrontal cortex pass through the uncinate fascicle (Ungerleider)through the uncinate fascicle (Ungerleider)

-The ability to form specific object-rewards associations was severely -The ability to form specific object-rewards associations was severely impaired after uncinate fascicle lesions (Parker and Gaffan)impaired after uncinate fascicle lesions (Parker and Gaffan)

-This study has demonstrated that violations of predictions made on the -This study has demonstrated that violations of predictions made on the basis of such associations specifically cause activity changes in basis of such associations specifically cause activity changes in frontotemporal circuitry. frontotemporal circuitry.

--Due to the experimental design Ramnani et al. were able to separately Due to the experimental design Ramnani et al. were able to separately determine activity changes related to unexpected rewards and determine activity changes related to unexpected rewards and unexpected failures of reward in reversal trials, changes were found in unexpected failures of reward in reversal trials, changes were found in the anterior prefrontal cortex for both types of prediction error but the the anterior prefrontal cortex for both types of prediction error but the medial part of the orbitofrontal cortex shows activity changes that are medial part of the orbitofrontal cortex shows activity changes that are specific to the presentation of unexpected rewardspecific to the presentation of unexpected reward

-The results are consistent with the view that changes in activity in -The results are consistent with the view that changes in activity in frontotemporal circuits do not simply represent the processing of frontotemporal circuits do not simply represent the processing of predictive stimuli and rewards but actively encode the associations predictive stimuli and rewards but actively encode the associations between thembetween them

*Lesions may therefore impair learning because frontotemporal circuitry is *Lesions may therefore impair learning because frontotemporal circuitry is required for the processing of error feedbackrequired for the processing of error feedback

III. Separate Neural systems Value Immediate III. Separate Neural systems Value Immediate Delayed Monetary RewardsDelayed Monetary Rewards

Samuel M. McClure, David I. Laibson, George Loewenstein, Jonathan Samuel M. McClure, David I. Laibson, George Loewenstein, Jonathan D. CohenD. Cohen

Premise:Premise:-Human decision makers are apparently torn between impulsiveness and -Human decision makers are apparently torn between impulsiveness and

patiencepatience-consumers behave impatiently today but prefer/plan to act patiently in the -consumers behave impatiently today but prefer/plan to act patiently in the

futurefuture Ex. “Someone offered the choice between $10 today and $11 tomorrow Ex. “Someone offered the choice between $10 today and $11 tomorrow

might be tempted to choose the immediate option. However, if asked might be tempted to choose the immediate option. However, if asked today to choose between $10 in a year and $11 in a year and a day, the today to choose between $10 in a year and $11 in a year and a day, the same person is likely to prefer the slightly delayed but larger amount.”same person is likely to prefer the slightly delayed but larger amount.”

Hypothesis:Hypothesis:-The discrepancy between short-run and long –run preferences reflects the -The discrepancy between short-run and long –run preferences reflects the

differential activation of distinguishable neural systems. differential activation of distinguishable neural systems. - β , short run impatience is driven by the limbic system, which responds - β , short run impatience is driven by the limbic system, which responds

preferentially to immediate rewards and is less sensitive to the value of preferentially to immediate rewards and is less sensitive to the value of future rewardsfuture rewards

- δ , long run patience is mediated by the lateral prefrontal cortex and - δ , long run patience is mediated by the lateral prefrontal cortex and associated structures, which are able to evaluate tradeoffs between associated structures, which are able to evaluate tradeoffs between abstract rewards, including rewards in the more distant futureabstract rewards, including rewards in the more distant future

*Based on a time discounting function referred to as beta-delta preference *Based on a time discounting function referred to as beta-delta preference (Pollack)(Pollack)

Methods:Methods:participants made a series of binary choices between participants made a series of binary choices between smaller/earlier and larger/later monetary amounts while their smaller/earlier and larger/later monetary amounts while their brains were scanned using fMRIbrains were scanned using fMRIspecific amounts ranging from $5-$40 and times of availability specific amounts ranging from $5-$40 and times of availability ranging from the day of the experiment to 6 weeks laterranging from the day of the experiment to 6 weeks later

Results:Results:Fig. 1Fig. 1-brain areas disproportionately activated by choices involving -brain areas disproportionately activated by choices involving an immediate outcome (β areas) include the ventral striatum, an immediate outcome (β areas) include the ventral striatum, medial orbitoprefrontal cortex, and medial prefrontal cortex. medial orbitoprefrontal cortex, and medial prefrontal cortex. As predicted these are the classic limbic structures and closely As predicted these are the classic limbic structures and closely related paralimbic cortical projectionsrelated paralimbic cortical projections

Fig.2Fig.2-voxels that loaded on “all decisions” variable are δ areas, -voxels that loaded on “all decisions” variable are δ areas, these were not activated by all decision epochs and were not these were not activated by all decision epochs and were not preferentially activated by experimental choices that included preferentially activated by experimental choices that included an option for a reward today, areas of the LPFC as predictedan option for a reward today, areas of the LPFC as predicted

Conclusions:Conclusions:

-When humans are offered the choice between rewards -When humans are offered the choice between rewards available at different points in time, the relative values of available at different points in time, the relative values of the options are discounted according to their expected the options are discounted according to their expected delays until deliverydelays until delivery

-2 separate systems are involved in such decisions:-2 separate systems are involved in such decisions:1. Parts of the limbic system (associated with the midbrain 1. Parts of the limbic system (associated with the midbrain

dopamine system including paralimbic cortex) are dopamine system including paralimbic cortex) are preferentially activated by decisions involving immediately preferentially activated by decisions involving immediately available rewards, βavailable rewards, β

2. Regions of the lateral prefrontal cortex and posterior 2. Regions of the lateral prefrontal cortex and posterior parietal cortex are engaged uniformly by intertemporal parietal cortex are engaged uniformly by intertemporal choices irrespective of delay, δchoices irrespective of delay, δ

-this analysis shows that the degree of engagement of the -this analysis shows that the degree of engagement of the these areas is consistent with a key role in future planningthese areas is consistent with a key role in future planning

Overall Conclusions on PFC and Overall Conclusions on PFC and RewardReward

PFC plays an important role inPFC plays an important role inI. Gambling: negative correlation between I. Gambling: negative correlation between

PFC activation and addiction severity, PFC activation and addiction severity, implicating that it is not a choiceimplicating that it is not a choice

II. Prediction Error: predictions based on II. Prediction Error: predictions based on monetary reward irrespective of behavior, monetary reward irrespective of behavior, implicates PFC in stimuli association implicates PFC in stimuli association processes processes

III. Decisions based on monetary reward: III. Decisions based on monetary reward: irrespective of delay, future planning skillsirrespective of delay, future planning skills