a model-based fmri analysis with hierarchical ... - ccs lab

A Model-Based fMRI Analysis With Hierarchical BayesianParameter Estimation

Woo-Young Ahn, Adam Krawitz, Woojae Kim, Jerome R. Busemeyer, andJoshua W. Brown

Indiana University Bloomington

A recent trend in decision neuroscience is the use of model-based functional magneticresonance imaging (fMRI) using mathematical models of cognitive processes. How-ever, most previous model-based fMRI studies have ignored individual differencesbecause of the challenge of obtaining reliable parameter estimates for individualparticipants. Meanwhile, previous cognitive science studies have demonstrated thathierarchical Bayesian analysis is useful for obtaining reliable parameter estimates incognitive models while allowing for individual differences. Here we demonstrate theapplication of hierarchical Bayesian parameter estimation to model-based fMRI usingthe example of decision making in the Iowa Gambling Task. First, we used a simulationstudy to demonstrate that hierarchical Bayesian analysis outperforms conventional(individual- or group-level) maximum likelihood estimation in recovering true param-eters. Then we performed model-based fMRI analyses on experimental data to examinehow the fMRI results depend on the estimation method.

Keywords: model-based fMRI, functional magnetic resonance imaging, reinforcement learning,hierarchical Bayesian analysis, model comparison

How we make decisions to obtain rewardsand avoid punishments is a fundamental topicacross research areas including psychology,economics, neuroscience, and computer sci-ence. In the past decade, decision neuroscienceresearchers have begun to approach decisionmaking from an interdisciplinary perspective—integrating quantitative models with neural sig-nals. For example, early pioneering studiesdemonstrated that the phasic responses of mid-brain dopamine neurons can be well describedby the temporal-difference reinforcement learn-ing algorithm (Montague, Dayan, & Sejnowski,1996; Schultz, Dayan, & Montague, 1997).More recently, human functional magnetic res-onance imaging (fMRI) studies have shown that

blood-oxygen-level-dependent (BOLD) activa-tions in brain regions including the striatum andorbitofrontal cortex correlate with prediction er-ror signals from the temporal-difference learn-ing model (McClure, Berns, & Montague, 2003;O’Doherty, Dayan, Friston, Critchley, & Dolan,2003). These fMRI studies used the method of“model-based fMRI,” in which a mathematicalmodel of behavior provides a framework tostudy neural mechanisms of reward learning. Inmodel-based fMRI, predictions derived from amathematical model of choice behavior are cor-related with fMRI data to determine brain areasrelated to postulated decision-making pro-cesses. This method is increasingly popular indecision neuroscience because it provides in-sight into the neural correlates of predicted cog-nitive processes and can be useful to discrimi-nate competing theories of brain function (seeO’Doherty, Hampton, & Kim, 2007, for a re-view and methodological recipe).

The first step in model-based fMRI is toestimate the free parameters in the mathematicalmodel of behavior. Getting accurate parameterestimates is important not only because param-eter estimates reflect psychological traits (e.g.,the learning rate or the balance of exploitation

Woo-Young Ahn, Adam Krawitz, Woojae Kim, Je-rome R. Busemeyer, and Joshua W. Brown, Departmentof Psychological and Brain Sciences, Indiana UniversityBloomington.

Adam Krawitz is now at Department of Psychology,University of Victoria, Victoria, Canada.

Correspondence concerning this article should be ad-dressed to Joshua W. Brown, Department of Psychologicaland Brain Sciences, 1101 East 10th Street, Indiana Univer-sity, Bloomington, IN 47405. E-mail: [email protected]

Journal of Neuroscience, Psychology, and Economics © 2011 American Psychological Association2011, Vol. 4, No. 2, 95–110 1937-321X/11/$12.00 DOI: 10.1037/a0020684

95

and exploration), but also because they affectthe results of the subsequent model-based fMRIanalysis (e.g., Tanaka et al., 2004). Most previ-ous studies with model-based fMRI have usedgroup-level analysis with maximum likelihoodestimation (MLE). In group-level analysis, freeparameters are assumed to be homogenousacross participants and no individual differ-ences are taken into account. It might be toler-able to ignore individual differences in simpleconditioning tasks where most participants be-have similarly, but it is inappropriate in com-plex tasks where substantial individual differ-ences exist. To take into account individualdifferences, an alternative approach is individual-level analysis. However, using MLE for individ-ual-level analysis can lead to noisy and unreliableestimates if the amount of information from eachindividual is limited, as is often the case in neu-roimaging studies. We propose the use of hierar-chical Bayesian analysis (HBA) to reconcile thetension between individual differences and reli-able parameter estimation.

HBA is an advanced branch of Bayesian sta-tistics (Berger, 1985) that uses the basic princi-ples of Bayesian statistical inference (Gelman,Carlin, Stern, & Rubin, 2004). One of the ad-vantages of HBA is that, whereas MLE finds apoint estimate for each parameter that maxi-mizes the likelihood of the data, HBA finds afull posterior distribution of belief across therange of values a parameter can take on. Asecond advantage of HBA is that it allows forindividual differences while pooling informa-tion across individuals in a coherent way. Bothindividual and group parameter estimates arefound simultaneously in a mutually constrain-ing fashion. By capturing the commonalitiesamong individuals, each individual’s param-eter estimates tend to be more stable andreliable because they are informed by thegroup tendencies.

In this article, we apply HBA to model-basedfMRI analysis of the Iowa Gambling Task(IGT; Bechara, Damasio, Damasio, & Ander-son, 1994) and compare it with other estimationmethods. The remainder of this article is orga-nized as follows. First, we briefly explain theIGT and a mathematical model for the task.Second, we briefly explain the procedures ofMLE and HBA and then use simulated datawith known true parameter values to examinewhich method recovers the values more accu-

rately. To tease apart the relative contributionsof Bayesian estimation and the use of a hierar-chical approach, we estimate parameters usingeach of the following methods: MLE at thegroup level (MLE-Group), MLE at the individ-ual level (MLE-Ind), nonhierarchical Bayesiananalysis (Bayes-Ind), and HBA. Third, we pres-ent model-based fMRI analyses using parame-ter estimates from MLE-Group, MLE-Ind, andHBA and show how the choice of method in-fluences the fMRI results.

IGT

The IGT is a well-established task used tostudy decision-making processes in control par-ticipants and decision-making deficits in clini-cal populations including substance abusers andpatients with brain lesions (Bechara, Damasio,Tranel, & Damasio, 1997; Bechara et al., 2001).The goal of the task is to maximize monetarygains while repeatedly choosing cards from oneof four decks. Each selection results in a mon-etary gain, draw, or loss. There are two “good”decks with long-term gains and two “bad” deckswith long-term losses. However, the typical winis larger for the bad decks than the good decks,putting the magnitude of the potential immedi-ate gain into opposition with the long-term cu-mulative outcome from the decks. Furthermore,one bad deck and one good deck have infre-quent larger losses, and the other bad and onegood deck have more frequent smaller losses.Participants need to learn from experiencewhich decks are advantageous for good perfor-mance. It is a challenging task and even healthycontrol participants show substantial individualdifferences in behavioral performance.

Prospect Valence Learning (PVL) Model

We used the PVL model (Ahn, Busemeyer,Wagenmakers, & Stout, 2008) to mathemati-cally model participants’ decision-making pro-cesses. The model assumes that the evaluationof outcomes follows the prospect utility func-tion, which has diminishing sensitivity to in-creases in magnitude and loss aversion; that is,different sensitivity to losses versus gains. Spe-cifically, the utility, u!t" on trial t of each netoutcome x!t" is expressed as:

u!t" ! ! x!t"# if x!t" " 0$%"x!t""# if x!t" # 0. (1)

96 AHN, KRAWITZ, KIM, BUSEMEYER, AND BROWN

Here # is a shape parameter (0 # # # 1) thatgoverns the shape of the utility function, and % isa loss aversion parameter (0 # % # 5) thatdetermines the sensitivity of losses comparedwith gains. Net outcomes were scaled for cog-nitive modeling so that the highest net gainbecomes 1 and the largest net loss becomes$11.5 (Busemeyer & Stout, 2002). As # getsclose to 1, the utility gets close to the objectiveoutcome amount, and as # gets close to 0, itbecomes more like a step function. If an indi-vidual has a value of loss aversion (%) greaterthan 1, it indicates that the individual is moresensitive to losses than gains. A value of % lessthan 1 indicates that the individual is moresensitive to gains than losses.

On the basis of the outcome of the chosenoption, the expectancies of decks were updatedon each trial using the decay-reinforcementlearning rule (Erev & Roth, 1998). When testedwith the Bayesian information criterion(Schwarz, 1978), previous studies consistentlyshowed that this rule had the best post hocmodel fit (Ahn et al., 2008; Yechiam & Buse-meyer, 2005, 2008) compared with others in-cluding the delta rule (Rescorla & Wagner,1972). The decay-reinforcement learning ruleassumes that the expectancies of all decks decay(are discounted) with time, and then the expec-tancy of the chosen deck is added to the currentoutcome utility:

Ej!t" ! A · Ej!t $ 1" % &j!t" ! u!t". (2)

The parameter A is a recency parameter!0 # A # 1", which determines how muchthe past expectancy is discounted. &j!t" is adummy variable that is 1 if deck j is chosenand 0 otherwise.

With the updated expectancies, the softmaxchoice rule (Luce, 1959) was then used to com-pute the probability of choosing each deck j. Ithas a sensitivity (or exploitiveness) parameter,' !t", which governs the degree of exploitationversus exploration.

Pr(D!t % 1" ! j) !e'!t"!Ej!t"

#k*1

4

e'!t"!Ek!t"

. (3)

Sensitivity, ' !t", is assumed to be trial inde-pendent and was set to 3c $ 1 (Ahn et al.,

2008; Yechiam & Ert, 2007). Here c is calledconsistency and was limited from 0 to 5 so thatthe sensitivity ranges from 0 (random) to 242(almost deterministic). In summary, the PVLmodel used in this study has four free parame-ters reflecting distinct psychological constructs:A, recency; #, utility shape; c, choice sensitiv-ity; and %, loss aversion.

Simulation Study

To compare the performance of HBA to theother methods in accurately recovering param-eter values, we simulated 30 participants per-forming the IGT (100 trials per participant)assuming that they behaved according to thePVL model. To be as realistic as possible, webased the simulated data on the actual fMRIstudy reported in a later section. The number ofparticipants and trials were matched to those ofthe actual study, and when generating stimu-lated data, we used parameters estimated fromthe actual data set. Specifically, the parametersof the simulation agents were the posteriormeans of the parameters found with HBA forthe real participants.1

The four free model parameters of each sim-ulated participant were then estimated in fourdifferent ways and compared to determinewhich method recovered the true parameter val-ues most accurately. The four methods are:MLE at the group level (MLE-Group), MLE atthe individual level (MLE-Ind), nonhierarchicalBayesian analysis at the individual level(Bayes-Ind), and HBA. HBA yields posteriordistributions of the group parameters (HBA-Group) and each individual’s parameters(HBA-Ind), so we report them separately. MLE-Group and MLE-Ind are the most widely usedapproaches in the decision neuroscience field,so we were most interested in comparing esti-mates from these approaches with those fromHBA. The inclusion of Bayes-Ind allowed us tolearn something about the degree to which theadvantages of HBA were due to the use of

1 We also compared the estimation methods using anindependent data set (i.e., randomly generated parametersfor simulation agents), and it yielded the same conclusionsas reported in the main text. We report the findings using the“nonindependent” data set because the parameters of thesimulated agents are more realistic and more likely to bedirectly comparable with the actual data.

97SPECIAL ISSUE: HIERARCHICAL BAYESIAN MODEL-BASED fMRI ANALYSIS

Bayesian estimation versus being due to the useof a hierarchical model.

Parameter Estimation Methods

Maximum likelihood estimation. ForMLE-Group, point estimates were made foreach parameter that maximized the sum of log-likelihood across all participants (e.g., Daw,O’Doherty, Dayan, Seymour, & Dolan, 2006).We used a combination of grid-search (100different initial grid positions) and simplexsearch methods (Nelder & Mead, 1965) imple-mented in the R programming language (R De-velopment Core Team, 2009).2 For MLE-Ind, aset of parameters were estimated that maxi-mized the log-likelihood of each individual’sone-step-ahead model predictions, again usingthe combination of grid-search and simplexsearch methods in R.

Bayesian estimation. For HBA, it is as-sumed that the parameters of individual partic-ipants are generated from parent distributions,which are modeled with independent beta dis-tributions for each parameter:

#i $ Beta!+#, ,#", (4)

%i- $ Beta!+%, ,%", %i ! 5 ! %i

-, (5)

Ai $ Beta!+A, ,A", (6)

ci- $ Beta!+c, ,c", ci ! 5 ! ci

-. (7)

As implied in these equations, #i and Ai werelimited to values between 0 and 1 and %i and ci

were limited to values between 0 and 5. Here +zand ,z are the mean and the standard deviationof the beta distribution of each group-level pa-rameter z (i.e., #, %, A, and c).3 The Bayesianmethod needs the specification of the prior dis-tributions for the parameters, and we used uni-form distributions for +z and ,z, which provideflat priors and thus assume no a priori knowl-edge about these parameters. The range of each+z was set between 0 and 1 and each ,z was setbetween 0 and %+z!1 $ +z"/3 so that each dis-tribution was prevented from being bimodal.Different diffuse priors were also tested (e.g.,uniform distributions for the beta means anddiffuse gamma distributions for their preci-sions), and they resulted in almost identical

posterior distributions. The hierarchical Bayes-ian model is represented graphically in Figure 1.

Finally, the observed data matrix from allparticipants was used to compute the posteriordistributions for all parameters according toBayes rule (Kruschke, 2010). Posterior infer-ence was performed with the Markov chainMonte Carlo (MCMC) sampling scheme inOpenBUGS (Thomas, O’Hara, Ligges, &Sturtz, 2006), an open source version of Win-BUGS (Spiegelhalter, Thomas, Best, & Lunn,2003), and BRugs, its interface to R (R Devel-opment Core Team, 2009). WinBUGS (andOpenBUGS) implements various MCMC com-putational methods, and it is relatively userfriendly and easy to program; thus, it has greatlyfacilitated Bayesian modeling and its applica-tions (Cowles, 2004). A total of 50,000 sampleswere drawn after 70,000 burn-in samples withthree chains.

For nonhierarchical Bayesian analysis, weagain used beta distributions for each partici-pant’s parameters. However, no hyper (group)parameters were assumed, and priors of eachparticipant’s parameters were uniform beta dis-tributions; for example, #i $ Beta!1,1". Weused the same number of burn-in samples andreal samples as the HBA analyses. For eachparameter, the Gelman–Rubin test (Gelman etal., 2004) was conducted to confirm the conver-gence of the chains. All parameters had R val-ues of 1.00 (in most parameters) or, atmost, 1.04, which suggested MCMC chainsconverged to the target posterior distributions.OpenBUGS codes used for all Bayesian analy-ses are available at http://www.ahnlab.org/home/research/.

Results and Discussion

Figure 2 shows the results of the simulationstudy focusing on comparisons between HBA-Ind and MLE (MLE-Ind and MLE-Group)methods. Wetzels, Vandekerckhove, Tuer-linckx, and Wagenmakers (2010) showed that

2 See Section 4 of Ahn et al. (2008) for more details.3 Typically a beta distribution is denoted by its own # (it is

not a shape parameter of the utility function) and . parameters.The mean (+) of a beta distribution is #/!# % .", and thevariance (,2) is #./!!# % ."2!# % . % 1""; thus, it canbe easily shown that # ! +(+!1 $ +"/,2 $ 1) and. ! !1 $ +"(+!1 $ +"/,2 $ 1).


parameter estimates of an individual’s data arenot reliable in the expectancy valence learning(EVL) model (Busemeyer & Stout, 2002),which closely resembles the PVL model. How-ever, their claim was based on parameter valuesestimated with each individual’s data only, notall individuals’ data in a group. In our data set,parameter estimates of the PVL also show somediscrepancy from actual values when using eachindividual’s data only (MLE-Ind in Figure 2). Inparticular, note that some MLE-Ind estimates areon the parameters’ boundary limits, indicating in-sufficient information in each individual partici-pant’s data. However, when estimated with HBA,parameter estimates (HBA-Ind in Figure 2) showmuch less discrepancy with actual values com-pared with MLE-Ind estimates.

The HBA results lead to two interesting obser-vations. First, by capturing the dependency be-tween all participants, the HBA estimates regresstoward the group mean when there is not much

information in individual participants’ data(“shrinkage”; Gelman et al., 2004). Second, HBAcan also be sensitive to individual differenceswhen there is enough information, which is dem-onstrated in the estimates for the learning param-eter (A). Although the performance of the HBAwas not perfect, estimated values from HBA wereoverall more accurate than those from MLE. Notethat we reached the same conclusion when run-ning additional simulations with different true pa-rameter values and different numbers of partici-pants and trials. Regarding MLE-Group estimates,A and c parameter estimates are around the meanvalues of individual values, but # and % estimatesare quite different from them.

Figure 3 plots the distribution of posteriormeans for individual participants’ parameter es-timates computed from each method (except forHBA-Group, for which distribution densities ofgroup parameters are plotted). When compared

Figure 1. Graphical depiction of the hierarchical Bayesian analysis for the prospect valencelearning model. Clear shapes indicate latent variables; shaded shapes indicate observedvariables. Single outlines indicate probabilistic functions of input; double outlines indicatedeterministic functions of input. Circles indicate continuous variables; squares indicatediscrete variables; rounded rectangular plates indicate replication over the indexing variable.See the main text for a description of the individual parameters.


with the true values, the results suggest thatHBA performs better than Bayes-Ind and MLE-Ind in recovering true values. The fact that theHBA-Ind estimates are closer to the true valuesthan Bayes-Ind estimates provides empiricalsupport that using a hierarchical approach in-deed helps get more accurate estimates.

The posterior distributions of parameters forHBA-Group closely resemble the true parame-ter distributions. However, these distributionsare narrower because they represent the vari-ance in the group estimates, not the varianceamong individuals.

The results in Figure 3 also give some insightinto why the MLE-Group estimates of # and %are so far from the actual values. The MLE-Group estimates are around the mode of theMLE-Ind distributions. For parameter A, mostparticipants’ true values are around the mode,so the MLE-Group estimate looks acceptable.However, for parameters # and %, the modes ofMLE-Ind parameters are near the lower bounds;thus, MLE-Group estimates are dissimilar fromthe overall patterns of actual values. Note thatthese results are not because of the estimationmethod (MLE) but because of the assumption

Figure 2. For the simulation study, parameter estimates of the four free parameters in theprospect valence learning model for each simulated participant from both HBA and MLE.Black circles indicate true parameter values; red triangles indicate MLE-Ind; large bluesquares indicate mean HBA-Ind estimates; green horizontal dashed lines indicate MLE-Groupestimates; and small sky-blue squares indicate 50 random samples from the posterior distri-bution of the HBA estimate for each participant. A * recency; # * utility shape; c * choicesensitivity; % * loss aversion. HBA * hierarchical Bayesian analysis; MLE * maximumlikelihood estimation; Ind * individual-level estimates; Group * group-level estimates.


made in the estimation—namely, that all partic-ipants use the same parameter values. Indeed, ifthe same assumption is used with Bayesian es-timation, the posterior means of the parametersare very close to the MLE-Group estimates (forbrevity, results are not shown here). In sum-mary, the results of the simulation study suggestthat HBA is the best among these methods forestimating free parameters of the PVL modelaccurately.

Model-Based fMRI Study

Having confirmed the utility of HBA in esti-mating behavioral parameters, we then wantedto examine how fMRI results would depend onthe choice of estimation method. We conductedmodel-based fMRI analyses of actual data usingparameter estimates from three of the estimationmethods (HBA, MLE-Ind, and MLE-Group).We included both the individual and group es-timates from HBA (HBA-Ind and HBA-Group).

By using group and individual estimates withboth MLE and HBA, we could consider theeffects of method and analysis level separately.We hypothesized that HBA would yield morepower than MLE because its parameter esti-mates would be more accurate (as revealed bythe simulation study) and, thus, its model pre-dictions would better reflect participants’ actualcognitive processes. We also predicted that,when MLE-Group and HBA-Group are com-pared with each other, MLE-Group would yieldmore power at the second level (group) than thefirst level (individual), whereas HBA-Groupwould conversely yield more power at the firstlevel (individual) than the second level (group).This follows from the fact that MLE-Groupsimply finds a group average parameter, butHBA-Group provides more individualized fitsconstrained toward the group mean. Regardingcomparisons between individual estimates andgroup estimates, we predicted that MLE-Groupwould yield more power than MLE-Ind be-

Recency

A

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

06 TRUE

Utility Shape

αα

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

06

Consistency

c

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

04

8

Loss Aversion

λλ

Freq

uenc

y

0 1 2 3 4 5

04

8

Recency

A

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

06 HBA−Ind

Utility Shape

ααFr

eque

ncy

0.0 0.2 0.4 0.6 0.8 1.0

08

Consistency

c

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

06

Loss Aversion

λλ

Freq

uenc

y

0 1 2 3 4 5

06

Recency

A

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

06 HBA−Group

Utility Shape

αα

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

03

Consistency

c

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

04

Loss Aversion

λλ

Den

sity

0 1 2 3 4 50.0

1.0

Recency

A

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

03

6

Baye−Ind

Utility Shape

α

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

06

Consistency

c

Freq

uenc

y0.0 0.2 0.4 0.6 0.8 1.0

06

Loss Aversion

λλ

Freq

uenc

y

0 1 2 3 4 5

06

Recency

A

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

010

MLE−GroupMLE−Ind

Utility Shape

αα

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

06

MLE−Group

Consistency

c

Freq

uenc

y

0.0 0.2 0.4 0.6 0.8 1.0

04

MLE−Group

Loss Aversion

λλ

Freq

uenc

y

0 1 2 3 4 5

06

MLE−Group

Figure 3. For the simulation study, histograms of the true parameter values and theparameter estimates from each estimation method for all of the simulated participants (exceptfor the third row, which shows the posterior distributions of the group parameters forHBA-Group). HBA * hierarchical Bayesian analysis; MLE * maximum likelihood estima-tion; Bayes * nonhierarchical Bayesian analysis; Ind * individual-level estimates; Group *group-level estimates.


cause, according to the simulation study, MLE-Ind estimates would be noisy and unreliable.

Method

Participants. We recruited participantsfrom the student body of Indiana UniversityBloomington. They were required to be atleast 18 years of age, to be right handed, and tomeet standard health and safety requirementsfor entry into the MRI scanner. They were paid$25/hr for participation plus performance bo-nuses based on points earned during the task. Atotal of 30 participants (mean age * 21.7 years;age range * 18–29; 16 women, 14 men) wereused in all reported analyses.

Design and procedure. Participants per-formed the IGT for a block of 100 trials.4 Dur-ing the task, there were 4 decks of cards labeledA, B, C, and D from left to right. Two of thedecks (A and B) are considered bad decks:(Deck A) a net-loss/frequent-loss deck, with50% loss trials, a mean loss of 25 points pertrial, and a gain of 100 on nonloss trials; and(Deck B) a net-loss/rare-loss deck, with 10%loss trials, a mean loss of 25 points per trial, anda gain of 100 on nonloss trials. The other twodecks (C and D) are considered good decks:(Deck C) a net-gain/frequent-loss deck, with50% loss trials, a mean gain of 25 points pertrial, and a gain of 50 on nonloss trials; and(Deck D) a net-gain/rare-loss deck, with 10%loss trials, a mean gain of 25 points per trial, anda gain of 50 on nonloss trials. The two baddecks were always adjacent with the frequent-loss deck to the left of the rare-loss deck. Thetwo good decks were also kept adjacent with thefrequent-loss deck to the left of the rare-lossdeck. The order of the bad and good decks wascounterbalanced across participants.

The specific sequences of gains and losses foreach deck were the same as in the original taskdesign (Bechara et al., 1994). However, unlikein the original design, each trial outcome waspresented as a net gain, draw, or loss; the par-ticipant started with an initial sum of 1,000points; and the entire task was performed on acomputer using E-Prime 1.2 (Psychology Soft-ware Tools Inc., 2006).

Participants were instructed to select cardsfrom the decks. They were informed that thegoal was to maximize earnings and that theywould receive a monetary bonus based on the

number of points they accumulated. They wereinformed of the 3 second period in which tomake each selection and the running point totalat the bottom of the screen. They were toldnothing about the order of the decks or how theorder might change from block to block.

The timing and presentation of a trial is pre-sented schematically in Figure 4. The partici-pant’s running point total was displayedthroughout the block at the bottom of thescreen. At the start of a trial, a message (“Somedecks are better than others”) and the 4 decks ofcards (labeled A, B, C, and D from left to right)were presented. The participant had 3 s to selecta deck by pressing one of their middle or indexfingers on buttons corresponding in a spatiallycompatible way to the decks. If the participantfailed to respond within 3 s, then the trial wasconsidered a no-response trial. After the re-sponse deadline, there was an exponentially dis-tributed delay of 0, 2, 4, or 6 s. After thevariable delay, a card from the chosen deck wasflipped over to reveal the outcome as a negative,zero, or positive point value; and the runningtotal was updated. On no-response trials, theoutcome was always a loss of 100 points toencourage participants to make a choice on ev-ery trial. The feedback remained visiblefor 0.8 s, after which the message, cards, andoutcome were removed for an exponentiallydistributed intertrial interval of 0.2, 2.2, 4.2,or 6.2 s before the next trial began. The vari-able-length delays between choice and outcomeand between trials were designed to allow thebrain activity associated with the decision pe-riod to be estimated separately from that asso-ciated with response to the outcome (Ollinger,Corbetta, & Shulman, 2001; Ollinger, Shulman,& Corbetta, 2001).

fMRI collection and preprocessing. Im-aging data were collected on a 3.0 Tesla Sie-

4 During fMRI data collection, participants performed theIGT for three blocks of 100 trials each for a larger study thatexamined the effect of persuasive messages on risky deci-sion making. For each block, a different hint message waspresented to the participant. Only data from the blocks withthe control message, “Some decks are better than others,”were analyzed here. Also, participants whose data sets con-tained unacceptable spike artifacts were excluded from fur-ther analysis. Spike artifacts were due to a technical prob-lem with the scanner and were unrelated to individualparticipants’ anatomy or performance. For the details of thefull study, see Krawitz, Fukunaga, and Brown (2010).


mens Magnetom Trio. For each participant, wecollected functional BOLD data using echo pla-nar imaging with free induction decay for ablock of 360 whole brain volumes with an echotime of 25 ms, a repetition time of 2,000 ms,and a flip angle of 70°. Images were collectedin 33 axial slices in interleaved order with 3-mmthickness and 1-mm spacing between slices.Each slice consisted of a 64 / 64 gridof 3.4375 / 3.4375 / 3-mm voxels. For eachparticipant, we collected a structural scan usingthree-dimensional TurboFLASH imaging.

We checked each participant’s functionalvolumes for transient spike artifacts in individ-ual slices using a custom algorithm imple-mented in MATLAB R2007a 7.4.0. Preprocess-ing of the data was conducted with SPM5(Wellcome Trust Centre for Neuroimaging,2005). The raw DICOM images were first con-verted to NIfTI format. Then we performedslice timing correction on the functional imagesusing SPM5’s Fourier phase shift interpolationwith the first slice as reference. We then per-formed motion correction using SPM5’s leastsquares six parameter rigid body transforma-tion. The structural scan was then skull-strippedusing BET2 (Pechaud, Jenkinson, & Smith,2006) with default parameters. The structuralscan was then coregistered with the motion-corrected functional scans using SPM5’s affinetransformation with the mean functional imageas reference. We then registered the images foreach participant to MNI space using SPM5’s

normalization procedure by first performing a12-parameter affine transformation within aBayesian framework using regularization to theICBM space template, followed by a nonlineardeformation using discrete cosine transform ba-sis functions. The source image was the struc-tural scan with an 8-mm Gaussian smoothingkernel, and the template was SPM5’s MNIAvg152, T1 at 2 mm3 along with its associatedweighting mask. The resulting normalized im-ages were written with 2-mm3 voxels. Finally,the normalized images were smoothed with an8-mm3 FWHM (full width at half maximum) ofthe Gaussian kernel.

Model-based fMRI analysis. For model-based fMRI analysis (Daw et al., 2006;McClure et al., 2003; O’Doherty et al., 2004;O’Doherty, et al., 2007; Tanaka et al., 2004),model-generated regressors were convolvedwith a canonical hemodynamic response func-tion and then correlated against BOLD fMRIsignals to determine brain areas related to thespecific decision-making processes (using para-metric modulation in SPM5). We used choiceprobability of the chosen option at the time ofdecision (onset of deck presentation) as the re-gressor of interest. The choice probability is arelative measure of the expected value signal(Daw et al., 2006). Ventromedial prefrontal cor-tex (vmPFC) is known to encode reward (Dawet al., 2006; Knutson, Taylor, Kaufman, Peter-son, & Glover, 2005), and IGT performance isimpaired in vmPFC lesion patients (Bechara et

Figure 4. Time course of the Iowa Gambling Task in a rapid event-related functionalmagnetic resonance imaging design.


al., 1994), so we hypothesized that the choiceprobability computed by the model for the cho-sen option on each trial would be correlatedwith activation of vmPFC at the time of deci-sion making.

To increase statistical power, we usedsummed probabilities for choosing either goodor bad decks. In other words, if a good deck waschosen on a given trial, we used the summedprobability of choosing either good deck, and ifa bad deck was chosen on a trial, we used thesummed probability of choosing either baddeck. Analyzing the IGT results in terms ofgood and bad decks is a common practice (e.g.,Bechara et al., 1994; Bechara et al., 1997).

For model-based fMRI with HBA, the regres-sors were generated directly from the posteriordistributions of each participant’s parameters(HBA-Ind) or group parameters (HBA-Group),and the means of the posterior predictive distri-butions were used as input to SPM5.5 For re-gressors from MLE, regressors were generatedfrom each participant’s point estimates (MLE-Ind) or group point estimates (MLE-Group). Inevery case, all regressors were standardized into zscores within each participant (M * 0, standarddeviation * 1) before being entered into SPM5.

We performed first-level analysis of the pre-processed fMRI data using SPM5. A generallinear model was conducted for each participantwith SPM5’s canonical hemodynamic responsefunction with no derivatives, a microtime reso-lution of 16 time-bins per scan, a high-pass filtercutoff at 128 s using a residual forming matrix,autoregressive AR(1) to account for serial corre-lations, and restricted maximum likelihood formodel estimation. The model included a constantterm, six motion regressors using the parametersof the motion correction performed during prepro-cessing, the model-generated regressor, and nui-sance regressors. Nuisance regressors includedobjective outcomes and the absolute value of lossoutcomes at the feedback period, reaction times,and onsets of no-response trials (mean number ofno-response trials * 1.8).

Group-level analyses were conducted withtwo approaches. The first approach was awhole-brain analysis, which assumes that acti-vations for a given contrast will be in the samelocation for all participants. This approach usedvoxel-by-voxel one-sample t tests to testwhether the beta coefficients across participantswere significantly different from zero. For acti-

vation maps in vmPFC, we used a threshold ofp 0 .001, uncorrected, with a cluster thresholdof 8 contiguous voxels. This approach was se-lected because it is the most commonly usedgroup-level analysis for model-based fMRIanalysis. The second approach was a form ofregion-of-interest (ROI) analysis where thevoxel with peak activation within an a prioriROI is selected for each participant. Our ROIwas vmPFC, defined as Brodmann area 25 di-lated by 1 mm using the WFU Pick Atlas (Lan-caster et al., 2000; Maldjian, Laurienti, Kraft, &Burdette, 2003). We tested the peak values foreach participant from this ROI at the secondlevel using an uncorrected one-sample t test.This approach was selected because past workhas suggested that the peak voxel best reflectsthe underlying neural activity (Arthurs & Bon-iface, 2003), and because this approach allowsfor individual differences in the particular loca-tion of activity.

Results and Discussion

Parameter estimation. Model parameterswere estimated from the actual behavioral datawith three different methods (HBA, MLE-Ind,and MLE-Group). Table 1 shows the parameterestimates from each method, and Figure 5 illus-trates the histograms of parameter estimatesfrom each method. Again, MLE-Ind estimatesare often on the boundaries of parameter rangesand MLE-Group estimates of # and % are nearthe lower bound of zero. Unlike the simulationstudy, we do not know true values of these pa-rameters, so we cannot tell which method is thebest for estimating them. However, the previoussimulation study and other studies (Fridberg et al.,2010; Shiffrin, Lee, Kim, & Wagenmakers, 2008;Wetzels et al., 2010) that examined this issuesuggest that the HBA estimates probably betterreflect individuals’ true internal characteristics.

Model-based fMRI. To focus on the com-parison of estimation methods, we limited ourfMRI analysis to correlations between choiceprobability for the chosen option and decision-time activation in the vmPFC. We found suchcorrelations with all four estimation methods

5 Alternatively, regressors can be generated from themeans of posterior distributions. For most participants,model-generated choice probabilities were very similar bothways.


using both approaches to the group-level fMRIanalysis. The results from each estimationmethod and each analysis approach are summa-rized in Table 2, and the activation maps fromthe whole-brain analysis are shown in Figure 6.To compare the models in more detail, we plot-ted z values for all four methods from thewhole-brain analysis within a rectangle of MNI

space ranging from 0 to 4 in x-coordinates, 10to 42 in y-coordinates, and $14 to $10 inz-coordinates (see Figure 7). This region in-cludes the vmPFC peak activations found witheach of the four methods.

With the whole brain analysis, for HBA andespecially MLE, group methods showed higherpeak z values and larger regions of activationthan individual methods. The MLE-Groupmethod produced the largest vmPFC region cor-relating with the choice probabilities. The acti-vation maps for HBA-Ind, HBA-Group, andMLE-Ind are quite similar in shape, and theirpeak coordinates match, but the map for MLE-Group is qualitatively different. The peak ofactivation is shifted forward along the y-axis forMLE-Group compared with the other methods.

With the ROI analysis, which searches theROI to find the voxel with the best model fitseparately for each individual, the group meth-ods again showed higher peak z values than theindividual methods. However, unlike thewhole-brain approach, both of the HBA meth-

Table 1Comparison of Prospect Valence Learning ModelParameters by Method of Parameter Estimation

Method A # c %

+ (HBA-Group) 0.86 0.34 0.29 1.25Mean of MLE-Ind

estimates 0.80 0.48 0.26 2.71MLE-Group 0.91 0.01 0.25 0.38

Note. Parameter estimates from the 30 participants in thefMRI study. A * recency; # * utility shape; c * choicesensitivity; % * loss aversion; HBA * hierarchical Bayes-ian analysis; MLE * maximum likelihood estimation;Ind * individual-level estimates; Group * group-levelestimates.

Figure 5. For the model-based fMRI study, histograms of each participant’s parameterestimates with each estimation method. Note that the HBA-Ind parameters in the first rowwere used as the true parameters in the simulation study. HBA * hierarchical Bayesiananalysis; MLE * maximum likelihood estimation; Ind * individual-level estimates; Group *group-level estimates.


ods yielded higher z values than either of theMLE methods. Furthermore, the mean locationsof the peaks were almost the same for all fourmethods.

The differences found here between estima-tion methods in terms of their ability to accountfor the fMRI data are relatively small. However,it is still interesting to note that to the extent thatthere are differences, HBA was more successfulwhen the method of analysis allowed for indi-vidual differences in the locations of activation.This suggests that, for HBA’s advantage in ac-counting for individual differences in behaviorto carry over to the fMRI analysis, that analysismust also allow for individual differences. Inaddition, HBA-Ind yielded more power thanMLE-Ind in both whole brain and ROI analyses,which suggests that allowing some extra param-eters in the hierarchical model is helpful interms of fMRI power when estimating eachsubject’s parameters separately.

What is less clear is why the HBA-Groupmethod outperformed HBA-Ind in the ROIanalysis, given that the simulation study sug-gested that HBA-Ind does the best job of account-ing for individual differences in parameter esti-mates. Further investigation should be conductedto characterize these patterns across a wider rangeof brain regions, tasks, and model predictions.

General Discussion

The simulation study compared two Bayesianmethods (HBA and Bayes-Ind) and two MLEmethods (MLE-Ind and MLE-Group) in theirperformance recovering true parameters and theresults suggest that HBA is the best method for

obtaining accurate individual and group param-eter estimates. Whereas the HBA estimateswere close to the true parameter values, theMLE-Ind estimates were often stuck on boundsand some MLE-Group estimates divergedmarkedly from the true parameter means. Indi-vidual estimates from the nonhierarchicalBayesian method (Bayes-Ind) were uninformedby other participants’ data and were often toovague and less accurate than HBA estimates.

Subsequent model-based fMRI results weregenerally consistent with the behavioral results.We predicted that a method that performed wellin the stimulation study would yield more sig-nificant activations when used for model-basedfMRI. We evaluated this prediction in thevmPFC for the correlation of decision-time ac-tivity and choice probabilities for the chosenoptions. As predicted, the HBA-Ind methodyielded more vmPFC activation than MLE-Ind,MLE-Group yielded more activation than MLE-Ind, and HBA-Group outperformed MLE-Groupin the ROI analysis. However, in the whole-brainanalysis MLE-Group yielded the largest vmPFCactivation, which was not expected. The best over-all performance with respect to power was foundwith HBA-Group, with the more flexible priorassumption that there may be individual differ-ences in peak voxel location. Nonetheless,when the simplifying assumption was made thatthere are no individual differences in peak voxellocation, then MLE-Group yielded more powerthan HBA-Group. To our knowledge, this is thefirst model-based fMRI study with HBA esti-mation, so further research is needed to confirmand extend the results of this study.

Table 2Comparison of PVL Model-Based fMRI Findings by Method of Parameter Estimation andAnalysis Approach

Estimation method

Whole-brain analysis ROI analysis

Maximum zvalue

Cluster size(kE)

Peak MNIcoordinates

Mean peak zvalue

Mean MNIcoordinates

HBA-Ind 3.84 54 [2, 16, $10] 5.20 [0, 12, $14]HBA-Group 4.17 74 [2, 16, $10] 5.31 [0, 12, $12]MLE-Ind 3.49 14 [4, 18, $10] 4.99 [0, 12, $14]MLE-Group 4.41 444 [0, 24, $14] 5.06 [0, 12, $12]

Note. Analysis focused on regions in the ventromedial prefrontal cortex (vmPFC) that correlated significantly with thechoice probability for the chosen option assigned by the prospect valence learning (PVL) model. Cluster size is in voxels.ROI * region of interest; HBA * hierarchical Bayesian analysis; MLE * maximum likelihood estimation; Ind *individual-level estimates; Group * group-level estimates.


We demonstrated the use of hierarchicalBayesian parameter estimation with model-based fMRI, and compared this method to othernonhierarchical and non-Bayesian methods. Asour results suggest, there may be several advan-tages to using HBA in studies correlating mod-eling predictions with both behavior and neuralsignals. In particular, obtaining more reliableparameter estimates from behavior may in turnbetter capture individual differences in neuralactivations underlying the behavior. If partici-pants’ data contain enough information, then

using nonhierarchical individual-level analysis(MLE-Ind or Bayes-Ind) for model-based fMRImight be sufficient. However, for many real-world scenarios, where only a limited numberof trials are collected for each participant, ourfindings support the use of a hierarchical anal-ysis (e.g., HBA). Although we applied the HBAto model-based fMRI, it can also be used formodel-based EEG analysis (Mars et al., 2008)and model-based single-cell monkey electro-physiology analysis. HBA has been recentlyutilized in many areas including biostatistics,

Figure 6. Brain regions whose decision-time activation correlates significantly with thechoice probability assigned by the prospect valence learning model using parameter estimatesfrom each estimation method. Thresholded at p 0 .001, uncorrected, cluster size " 8 voxels.HBA * hierarchical Bayesian analysis; MLE * maximum likelihood estimation; Ind *individual-level estimates; Group * group-level estimates.


economics, and cognitive modeling and hasproven to be a useful method. We believe that itwill also be a valuable tool in model-basedfMRI and hope that this study will help re-searchers in decision neuroscience better under-stand and adopt HBA for their work.

References

Ahn, W.-Y., Busemeyer, J. R., Wagenmakers, E.-J.,& Stout, J. C. (2008). Comparison of decisionlearning models using the generalization criterionmethod. Cognitive Science, 32, 1376–1402.

Arthurs, O. J., & Boniface, S. J. (2003). What aspectof the fMRI BOLD signal best reflects the under-lying electrophysiology in human somatosensorycortex? Clinical Neurophysiology, 114, 1203–1209.

Bechara, A., Damasio, A. R., Damasio, H., & An-derson, S. W. (1994). Insensitivity to future con-sequences following damage to human prefrontalcortex. Cognition, 50, 7–15.

Bechara, A., Damasio, H., Tranel, D., & Damasio,A. R. (1997). Deciding advantageously beforeknowing the advantageous strategy. Science, 275,1293–1295.

Bechara, A., Dolan, S., Denburg, N., Hindes, A.,Anderson, S. W., & Nathan, P. E. (2001). Deci-sion-making deficits, linked to a dysfunctionalventromedial prefrontal cortex, revealed in alcoholand stimulant abusers. Neuropsychologia, 39,376–389.

Berger, J. O. (1985). Statistical decision theory andBayesian analysis. New York: Springer.

Busemeyer, J. R., & Stout, J. C. (2002). A contri-bution of cognitive decision models to clinicalassessment: Decomposing performance on theBechara gambling task. Psychological assess-ment, 14, 253–262.

Cowles, M. (2004). Review of WinBUGS 1.4. TheAmerican Statistician, 58, 330–336.

Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour,B., & Dolan, R. J. (2006). Cortical substrates forexploratory decisions in humans. Nature, 441,876–879.

10 14 18 22 26 30 34 38 42

1

2

3

4

Z−va

lue

X−axis MNI = −2, Z−axis MNI = −10

HBA−IndHBA−GroupMLE−IndMLE−Group

10 14 18 22 26 30 34 38 42

1

2

3

4

X−axis MNI = 0, Z−axis MNI = −10

10 14 18 22 26 30 34 38 42

1

2

3

4


10 14 18 22 26 30 34 38 42

1

2

3

4

Z−va

lue



10 14 18 22 26 30 34 38 42

1

2

3

4


10 14 18 22 26 30 34 38 42

1

2

3

4


10 14 18 22 26 30 34 38 42

1

2

3

4

Y−axis MNI coordinate

Z−va

lue



10 14 18 22 26 30 34 38 42

1

2

3

4



10 14 18 22 26 30 34 38 42

1

2

3

4



Figure 7. The z values as a function of MNI coordinates for decision-time activationcorrelated with the choice probability assigned by the prospect valence learning model usingparameter estimates from each estimation method. Red dashed lines indicate thresholds forp 0 .001 (z * 3.06), and purple dashed lines indicate thresholds for p 0 .01 (z * 2.33).HBA * hierarchical Bayesian analysis; MLE * maximum likelihood estimation; Ind *individual-level estimates; Group * group-level estimates.


Erev, I., & Roth, A. (1998). Predicting how peopleplay games: Reinforcement learning in experimen-tal games with unique, mixed strategy equilibria.American Economic Review, 88, 848–881.

Fridberg, D. J., Queller, S., Ahn, W.-Y., Kim, W.,Bishara, A. J., Busemeyer, J. R., . . . Stout, J. C.(2010). Cognitive mechanisms underlying disad-vantageous decision-making in chronic cannabisabusers. Journal of Mathematical Psychology, 54,28–38.

Gelman, A., Carlin, J. B., Stern, H. S., & Rubin,D. B. (2004). Bayesian data analysis (2nd ed.).London: Chapman & Hall.

Knutson, B., Taylor, J., Kaufman, M., Peterson, R.,& Glover, G. (2005). Distributed neural represen-tation of expected value. Journal of Neurosci-ence, 25, 4806–4812.

Krawitz, A., Fukunaga, R., & Brown, J. W. (2010).Anterior insula activity predicts the influence ofpositively framed messages on decision making.Cognitive, Affective, & Behavioral Neuroscience,10(3), 392–405.

Kruschke, J. K. (2010). Doing Bayesian Data Anal-ysis: A tutorial with R and BUGS. AcademicPress/Elsevier Science.

Lancaster, J. L., Woldorff, M. G., Parsons, L. M.,Liotti, M., Freitas, C. S., Rainey, L., . . . Fox, P. T.(2000). Automated Talairach Atlas labels for func-tional brain mapping. Human Brain Mapping, 10,120–131.

Luce, R. (1959). Individual choice behavior. NewYork:Wiley.

Maldjian, J. A., Laurienti, P. J., Kraft, R. A., &Burdette, J. H. (2003). An automated method forneuroanatomic and cytoarchitectonic atlas-basedinterrogation of fMRI data sets. NeuroImage, 19,1233–1239.

Mars, R. B., Debener, S., Gladwin, T. E., Harrison,L. M., Haggard, P., Rothwell, J. C., . . . Bestmann,S. (2008). Trial-by-trial fluctuations in the event-related electroencephalogram reflect dynamicchanges in the degree of surprise. Journal of Neu-roscience, 28, 12539–12545.

McClure, S. M., Berns, G. S., & Montague, P. R.(2003). Temporal prediction errors in a passivelearning task activate human striatum. Neuron, 38,339–346.

Montague, P. R., Dayan, P., & Sejnowski, T. J.(1996). A framework for mesencephalic dopaminesystems based on predictive Hebbian learning.Journal of Neuroscience, 16, 1936–1947.

Nelder, J. A., & Mead, R. (1965). A simplex methodfor function minimization. Computer Journal, 7,308–313.

O’Doherty, J. P., Dayan, P., Friston, K., Critchley,H., & Dolan, R. J. (2003). Temporal differencemodels and reward-related learning in the humanbrain. Neuron, 38, 329–337.

O’Doherty, J. P., Dayan, P., Schultz, J., Deichmann,R., Friston, K., & Dolan, R. J. (2004). Dissociableroles of ventral and dorsal striatum in instrumentalconditioning. Science, 304, 452–454.

O’Doherty, J. P., Hampton, A., & Kim, H. (2007).Model-based fMRI and its application to rewardlearning and decision making. Annals of the NewYork Academy of Sciences, 1104, 35–53.

Ollinger, J. M., Corbetta, M., & Shulman, G. L.(2001). Separating processes within a trial inevent-related functional MRI—II. Analysis. Neu-roImage, 13, 218–229.

Ollinger, J. M., Shulman, G. L., & Corbetta, M.(2001). Separating processes within a trial inevent-related functional MRI—I. The method.NeuroImage, 13, 210–217.

Pechaud, M., Jenkinson, M., & Smith, S. (2006).Brain extraction tool (BET). Oxford, England: Ox-ford University Centre for Functional MRI of theBrain.

Psychology Software Tools Inc. (2006). E-Prime.Pittsburgh, PA: Author.

R Development Core Team. (2009). R: A languageand environment for statistical computing. Vienna:R Foundation for Statistical Computing.

Rescorla, R., & Wagner, A. (1972). A theory ofPavlovian conditioning: Classical conditioning, II.New York: Appleton-Century-Crofts.

Schultz, W., Dayan, P., & Montague, P. R. (1997). Aneural substrate of prediction and reward. Science,275, 1593–1599.

Schwarz, G. (1978). Estimating the dimension of amodel. The annals of statistics, 461–464.

Shiffrin, R., Lee, M., Kim, W., & Wagenmakers, E.(2008). A survey of model evaluation approacheswith a tutorial on hierarchical Bayesian methods.Cognitive Science: A Multidisciplinary Jour-nal, 32, 1248–1284.

Spiegelhalter, D., Thomas, A., Best, N., & Lunn, D.(2003). WinBUGS version 1.4 user manual. Cam-bridge, United Kingdom: MRC Biostatistics Unit.

Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Oka-moto, Y., & Yamawaki, S. (2004). Prediction ofimmediate and future rewards differently recruitscortico-basal ganglia loops. Nature Neurosci-ence, 7, 887–893.

Thomas, A., O’Hara, B., Ligges, U., & Sturtz, S.(2006). Making BUGS Open. R News, 6, 12–17.

Wellcome Trust Centre for Neuroimaging. (2005).Statistical parametric mapping (SPM). London:Author.

Wetzels, R., Vandekerckhove, J., Tuerlinckx, F., &Wagenmakers, E. (2010). Bayesian parameter es-timation in the expectancy valence model of theIowa gambling task. Journal of Mathematical Psy-chology, 54, 14–27.


Yechiam, E., & Busemeyer, J. R. (2005). Compari-son of basic assumptions embedded in learningmodels for experience-based decision making.Psychonomic Bulletin & Review, 12, 387–402.

Yechiam, E., & Busemeyer, J. R. (2008). Evaluatinggeneralizability and parameter consistency in

learning models. Games and Economic Behav-ior, 63, 370–394.

Yechiam, E., & Ert, E. (2007). Evaluating thereliance on past choices in adaptive learningmodels. Journal of Mathematical Psychol-ogy, 51, 75– 84.

Online First Publication

APA-published journal articles are now available Online First in the PsycARTICLESdatabase. Electronic versions of journal articles will be accessible prior to the printpublication, expediting access to the latest peer-reviewed research.

All PsycARTICLES institutional customers, individual APA PsycNET" database pack-age subscribers, and individual journal subscribers may now search these records as anadded benefit. Online First Publication (OFP) records can be released within as little as 30days of acceptance and transfer into production, and are marked to indicate the postingstatus, allowing researchers to quickly and easily discover the latest literature. OFParticles will be the version of record; the articles have gone through the full productioncycle except for assignment to an issue and pagination. After a journal issue’s printpublication, OFP records will be replaced with the final published article to reflect thefinal status and bibliographic information.


a model-based fmri analysis with hierarchical ... - ccs lab

Documents