neuroeconomics || basic methods from neoclassical economics

C H A P T E R

1

Basic Methods from Neoclassical EconomicsAndrew Caplin and Paul W. Glimcher

O U T L I N E

Introduction 3

Rational Choice and Utility Theory: SomeBeginnings 4

Early Price Theory and the Marginal Revolution 4Early Decision Theory and Utility Maximization 5

The Ordinal Revolution and the Logic of Choice 6

Quantitative Tests of Qualitative Theories:Revealed Preference 7

GARP 10Understanding Rationality 10Axiomatic Approaches: Strengths 11Axiomatic Approaches: Weaknesses 11

Expected Utility Theory 12Defining the Objects of Choice: Probabilistic

Outcomes 12Continuity Axiom 12Independence 13The Expected Utility Theorem 13Axioms and Axiomatic Reasoning 14

Using Axioms: The Neoclassical Approachin Neuroeconomics 14

The Reward Prediction Error Hypothesis 14The DRPE Axioms and the Ideal Data Set 16Concluding Remarks 17

References 17

INTRODUCTION

A basic introduction to the economic theory of choiceis crucial for those starting out in the field of neuroeco-nomics. One reason for this is that the theory involvesabstractions, such as “utility functions,” that are offirst-order importance in organizing our understandingof choice behavior. Another reason is that it providestools for testing theories of choice against data. The thirdreason is even more fundamental. A history of the eco-nomic theory of choice carries lessons for the evolutionof neureconomics that will both help shape the field inthe coming decades and will allow neuroeconomists toavoid critical errors that crippled early economists intheir studies of human choice behavior. Those whostudy behavior from the economic point of view haveintroduced not only specific models and tests but alsosome remarkable methodological advances, all ofwhich can be leveraged by modern neuroeconomists.For all of these reasons a deep understanding of this“economic” way of thinking will prove increasinglyimportant as the field matures.

One reason that economic methods are of such greatvalue in neuroeconomics is that they are designed forrelatively complex decision-making situations whencompared to standard neurobiological tasks. Beingforced to face-up to the complexities of decision mak-ing and social interaction has forced economic theoriststo develop methods of a qualitative as well as a quanti-tative nature that are unique in the social and naturalsciences. Over the centuries, economists have beenforced to ask themselves again and again the questionof whether or not they are approaching their questionsfrom the right angle. As a result, they have developedmethods that are ideal when one is qualitativelyunsure about the appropriate modeling framework.That is particularly relevant because it is just the situa-tion in which neureconomics finds itself today. Therelationship between neural activity and decision mak-ing is just beginning to be unraveled. The fact that thetheoretical underpinnings of neuroeconomics remainemergent, explains in part the appeal and importanceof the methods and models of economic theory toneuroeconomists.

3Neuroeconomics. DOI: http://dx.doi.org/10.1016/B978-0-12-416008-8.00001-2 © 2014 Elsevier Inc. All rights reserved.

http://dx.doi.org/10.1016/B978-0-12-416008-8.00001-2

This chapter provides a short guide to some ofthe methods and models economists and others inrelated disciplines have developed in their struggle tosystematize our understanding of choice behavior. Thechapter is organized around the theory of utility maxi-mization, often referred to as the theory of rationalchoice, because it is the centerpiece of contemporaryeconomic theories of individual choice. The goal ofthis chapter is to show just how powerful and flexiblethese tools can be, and illustrate the help they mayoffer to those seeking to understand how neural phe-nomena and choice interact. A second goal of the chap-ter is to highlight some of the limitations of theseapproaches; limitations that can often be addressed orsupplemented by the approaches from psychology,neuroscience and anthropology presented in the restof this book. To those ends, we turn now to a highlyselective tour through some of the highlights of theeconomic theory of individual decision making. In theprocess, we will encounter the general methods thateconomists have developed for testing their theories.

The first section of this chapter opens with a sketchof several early ideas in the economic theory of choice.That brings us, in section two, to the standard theoryof utility maximization that achieved its mature formin the work of Vilfredo Pareto during the opening dec-ades of the twentieth century. In section three weintroduce the revealed preference approach to choice the-ory introduced by Paul Samuelson (1938). In sectionfour we describe the expected utility theory of Johnvon Neumann and Oskar Morgenstern (1944). In sec-tion five we present an illustration of how these meth-ods can be used in contemporary Neuroeconomics,with a discussion of the Dopaminergic Reward PredictionError Model.

RATIONAL CHOICE AND UTILITYTHEORY: SOME BEGINNINGS

One can view modern utility theory as derivingfrom two streams of thought in early economics: pricetheoretic and decision theoretic reasoning.

Early Price Theory and the Marginal Revolution

A key goal of traditional economics has always beento understand patterns in the prices and quantities ofgoods that are traded in markets. It is intuitively clearthat these exchange-values, the prices people actuallypay for things, must reflect a balance of forces betweencertain technologically impacted costs of production andsome notion of use-value; how valuable and useful thegoods are to those who are interested in obtaining

them. But early researchers struggled to understandhow these distinct notions of value (exchange-value,production costs, and use-value) could be related.Although it seemed obvious that the usefulness of agood must be an important determinant of how muchsomeone would pay for that good (the exchange-value), it was unclear how this could be reconciledwith the fact that relatively useless objects like dia-monds were expensive, while life-necessities like waterwere essentially free.

David Ricardo (1817), perhaps the most importanteconomist of the early nineteenth century, was a cen-tral figure in developing the earliest workable answerto these questions concerning the determination ofprices. He focused the majority of his energies onstudying the costs of producing objects, in the beliefthat the costs of production were central to determin-ing exchange-value. He reasoned that the prices ofthose commodities produced by human effort weredetermined by the prices of the inputs necessary fortheir production, such as the price of labor and theprice of, for example, land. In its crudest form, thisbecame known as the “labor theory of value,” accord-ing to which, a good’s value derives from the numberof labor hours it takes to produce. Thus, according tothe labor theory of value, diamonds cost more thanwater because they take so much more labor input toproduce than does water.

The theory that labor input determines value has,however, some obvious flaws that soon became appar-ent. After all, pieces of heavy rock cut into the shapeof clouds are quite as hard to produce as diamonds,yet we observe that they have a price of near zero (atleast when compared to diamonds) on an open market.But the field would not learn how to resolve thediamond�water paradox for many years. Resolutionsto this paradox were provided only in the middle andlate nineteenth century during the course of what isnow called the Marginalist Revolution.

The conceptual break that led to this revolution issimple, yet profound. The key was to realize that theprice of water in a particular situation should not to bethought of as reflecting the average value of all waterbut rather, that it should be thought of in terms ofwhat is known as its marginal value. Here the wordmarginal refers, in essence, to the first derivative of afunction; taking its meaning from the latin word mean-ing border or edge. To understand the key insight of themarginal revolution, consider a graph of liters-of-wateragainst the total exchange-value of that amount ofwater, as shown in Figure 1.1. To understand the mar-ginalist approach we want to consider the figure asdepicting an empirically observed dataset at the levelof a marketplace. It tells us, based on an imaginaryempirical dataset of the kind gathered during the

4 1. BASIC METHODS FROM NEOCLASSICAL ECONOMICS

NEUROECONOMICS

marginal revolution, how the price of an additionalliter of water depends on how much water the deci-sion maker already possesses. What the marginal revo-lution focused on was the empirical observation thatthe second liter of water one comes to possess isobserved to command a higher price, per liter, thandoes the 10,001st liter of water. And this turned out tobe a broad empirical regularity: these economistsnoted the critical fact that the marginal value of a literof water declined as the total amount of water thedecision maker had increased. Thus, since water is plenti-ful and people already own a large amount of it undermost circumstances, the price of an additional literdoes not have to be high. Of course this also meansthat if water were to become scarce, it would beextremely expensive � much more expensive thandiamonds.

The next question posed by the early economists ofthe marginal revolution, however, is the critical one.Given the kinds of empirically observed price curvesshown in Figure 1.1 these economists asked: whyshould it be that a decision maker who already has10,000 liters of water will only pay a few cents for anadditional liter but a decision maker who has only1 liter of water will pay much more for that sameamount of water? What could account for the fact thatthe same person was willing to pay such a large frac-tion of his wealth for a single liter of water under

some conditions but was only willing to pay a trivialamount of money for the exact same good under otherconditions?

To answer that question they drew a number ofconclusions. First, they reasoned that this could onlybe the case if decision makers derived more happinessor contentment (per liter) from the second liter ofwater they possessed (as they might when in a desertsociety) than they did from the 10,001st liter they pos-sessed (as they might when in a riverine society). Thesepath-breaking economists assumed that what decisionmakers must be doing when they showed this kind of pricingbehavior, was acting so as to maximize their happiness orsatisfaction. They paid more for the second liter ofwater because they wanted it, desired it, and enjoyedit more. In the language of modern economics theyassumed that when subjects decided how much theywere willing to pay, they must be acting to maximizetheir overall utility, that the pattern of observed pricesthus allows us to infer how utility changes as a func-tion of how much water or how many diamonds adecision maker possesses.

Starting with this assumption that people must beutility maximizing (that they pay more for things thatthey want more), they concluded that price curves likethe one observed in Figure 1.1 must mean that thedecision makers driving those price curves experi-enced less satisfaction, or utility, from each subsequentliter of water.

The marginal revolution was critical because itfinally resolved the diamond�water paradox that hadplagued economists at least since the time of AdamSmith (1776). It allowed economists to explain whypricing was related to scarcity by invoking threeabstract notions: (i) by assuming that decision makersmaximize utility by their actions; (ii) that decisionmakers experienced utility from owning or consuminggoods; and that (iii) the amount of utility they experi-enced per unit of most goods was a function “dimin-ishing at the margin.”

Early Decision Theory and UtilityMaximization

As the history of the marginal revolution indicates,early economists took considerable time to formalizetheir theory of choice using the idea of a utility func-tion, a curve relating satisfaction (and hence price) toquantity. A similar formulation was initiated in puremathematics in the early days of probability theoryand also had a broad impact on economics. One suchparticularly important formulation was that of theEnlightenment mathematician Blaise Pascal. He wastrying to understand the logic of gambling decisions.

Small marginal increase in price

Large marginal increase in price

Total liters of water possessed

Exc

hang

e-va

lue

(Pric

e)

} }

1 Literincrement

1 Literincrement

FIGURE 1.1 During the Marginal Revolution it was realized thatthe exchange-value of a single liter of water or a single diamond wasinfluenced both by the intrinsic value of that object and by howmany of those objects the decision maker possessed. Here, that rela-tionship is plotted for water. The graph shows the total exchange-value of any given number of liters of water. Note that as the totalnumber of liters of water possessed increases, the value of an addi-tional liter diminishes. Thus if a person possesses no water at all, asingle liter is of tremendous value. If a person possesses thousandsof liters then the exchange value, to him, of an additional liter wouldbe low.

5RATIONAL CHOICE AND UTILITY THEORY: SOME BEGINNINGS

NEUROECONOMICS

Should an individual prefer to buy a lottery ticket thatyields a 50% chance of winning $200 at a price of $45or should he pass up the ticket and keep his $45? ForPascal the basic technique for answering this questionwas simple and directly numerical. He proposed mul-tiplying the probability of winning by the amount tobe won so as to compute an average, or expected,value for the action. One then selected the option hav-ing the larger of the two expected values. Here Pascalmakes no assumptions about what real people do,instead he simply states what he considers to be thebest way to maximize each decision-maker’s well-being (or welfare):

2003 0:5. 45 ð1:1ÞFrom a scientific perspective, however, the problem

with Pascal’s decision algorithm is that it is both arbi-tary and empirically unrealistic. Consider a poor manwho obtains a lottery ticket that offers a 50% chance ofwinning $20,000. A wealthy woman offers to buy thatticket for $7000. Should the poor man accept the offer?Pascal’s answer is no. But can we be sure that the beg-gar has made a mistake if he accepts this offer? He ischoosing between being certain that he will haveenough money to pay for food and lodging for monthsor facing a 50�50 chance of either continuing in povertyor being roughly three times as wealthy as if he acceptsthe offer. Surely there would be nothing illogical (orirrational) about accepting the $7000 offer under theseconditions? Indeed, no modern economist would con-sider the beggar irrational for accepting the $7000 offer.

To allow for the fact that acceptance of this offeris obviously reasonable, Daniel Bernoulli proposedthat rather than maximizing expected value, whatchoosers should do, is maximize a transformation ofthis quantity called expected utility. For ideal decisions,Bernoulli hypothesized that the value derived from agiven dollar is a non-linear function of how much thatdollar increases a chooser’s total wealth, usually desig-nated ω. In the language of economics we say that: forPascal:

7000. 200003 0:5 ð1:2Þbut for Bernoulli:

uð70001ωÞ , ?.uð200001ωÞ1 uðωÞ

2ð1:3Þ

Here we use the expression u( � ) to indicate that thevalue to each individual is a mathematical function ofthe thing in the parenthesis, which includes a priorendowment of wealth. We refer to that function, notyet fully specified in our discussion, as an utility func-tion or just as u for short. That means that our chooserhas to decide which is bigger, the quantity on the leftor the quantity on the right (which is the average

utility one would get from the two possible outcomesof the lottery), and of course that depends both onwhat the function u looks like and on the wealth of thechooser. Bernoulli went further and proposed a spe-cific functional form for u: he proposed that it was loga-rithmic. Bernoulli’s idea, then, is almost a reflectedform of the ideas of the marginal revolution (which hepreceded). Rather than assuming that observed pricesreflect maximization of some hidden utility function,Bernoulli proposed a fixed utility function andasserted that reasonable choosers should maximizethat function. But despite these differences, one canimmediately see the similarity between Figure 1.1which captures a price theoretic notion of diminishingmarginal value � which implies a diminishing mar-ginal utility in decision makers � and Figure 1.2 thatcaptures a decision theoretic notion of diminishingmarginal utility. The former resting on the assumptionthat, whatever choosers are doing, they must be tryingto maximize their satisfaction, or utility, and the latterspecifying how to maximize utility regardless of whatchoosers actually do.

THE ORDINAL REVOLUTION AND THELOGIC OF CHOICE

Just as the Marginal Revolution seemed to hinge ondiminishing marginal utility, so too did the logarithmicutility function, proposed by Bernoulli. Taken together,these two sets of ideas may be seen as implying thatutility can somehow be measured, and that in any rea-sonable such method of scaling and measuring, therewill be some form of diminishing marginal utility.

Small increase in total utility

Large increase in total utility

$1 Increment $1 IncrementTotal wealth level of the decision maker

Tota

l sat

isfa

ctio

n/ut

ility

poss

esse

d by

dec

isio

n m

aker

FIGURE 1.2 Bernoulli’s logarithmic utility function. Bernoullihypothesized that the value of each additional dollar to a decisionmaker was influenced by the total number of dollars he possessed,and in a logarithmic fashion. Thus the total satisfaction, or utility,conferred upon the decision maker by an additional dollar declinesas a logarithmic function of total wealth.


NEUROECONOMICS

Taking their lead from these insights, the major econ-omists of the nineteenth century began to focus theirenergies on understanding how use-value, costs-of-production and exchange-value were related to the utili-ties experienced by decision makers. Their implicit goalwas to understand how changes in society impacted thenet utilities (also often called the welfare) of citizens in anattempt to design better societies. By the middle of thecentury many of these theories had become quite byzan-tine, with very detailed explanations of how the internalutility-specifying processes of decision makers interactedwith the outside world to yield choices and hence indi-vidual levels of welfare. But critically, all of these theo-ries were brought to a crashing halt at the end of thenineteenth century by the next revolution in the eco-nomic theory of choice, the Ordinal Revolution initiatedby Vilfredo Pareto (1906).

Pareto noted that while contemporary theoriesrooted in diminishing marginal utility relied on thevery precise details of the functions that related thingslike use-value and scarcity to utility, there was noindependent evidence that utility existed, nor wasthere any evidence to support the assumption thatpeople were acting to maximize their utilities. He evenwent so far as to prove mathematically that there was,in principle, no way to directly derive a unique valuefor utility from the choice behavior of a subject � acritical point the economists of the marginal revolutionhad entirely missed. All of these theories were, heproved, houses of cards built on layers of assumptionsthat could not be proven. What he stressed emphati-cally was that the only things traditional economistscould observe and measure were choices and prices.Focusing on anything else was, he argued, placing the-ory before data. Even worse, as he developed this logiche was able to show that the precise numerical scalingof utilities on which these theories rested were almostunconstrained by actual data on choices and pricesdue to this critical flaw in their reasoning.

To understand this issue, consider a decision makerwho is empirically observed to prefer Apples toOranges, Oranges to Grapes and Apples to Grapes. Inthe language of economics we represent thoseobserved preferences in the following way:

Apples g Oranges;

Oranges g Grapes and

Apples g Grapes

ð1:4Þ

In this standard notation, the curly greater than (orless than) sign should be read as meaning “prefers.”With these observed preferences in hand, let us assignfor Jack a utility of 3 to apples, 2 to Oranges and 1 toGrapes. Certainly, with these numbers we can rational-ize the observed pattern of preferences as being based

on a desire for the item offering highest utility � in away much like the pricing curves did for DavidRicardo. Unfortunately, and this is the critical thingthat Pareto recognized, the same pattern could beexplained if we squared all utility numbers, or if wehalved or doubled them. The numbers themselvesseem superfluous to the observed pattern of preference,and indeed as Pareto was the first to realize, they are.

Choice data tells us how subjects rank the objects ofchoice in terms of desirability. We can talk about utili-ties as ways to describe this ranking, but we mustalways remember that utilities are only really good forordering things. Treating utilities as discrete or precisenumbers that can be added or subtracted either forone individual or across individuals goes way too far.

What Pareto went on to stress, to say this anotherway, was that utility functions are only about ordering,not about discrete numerical values described byabstract mathematical functions. Mathematicians referto numerical scales that only provide informationabout ordering as ordinal scales and thus what Paretoargued was that utility must be considered an ordinalquantity. If one good has a utility of 4 and anothergood has a utility of 2 (for a given chooser) then weknow that the first good is better, but we do not reallyknow how much better. This stands in contrast to numer-ical systems in which 4 really is twice the size of 2.These are systems of numbers referred to as cardinal.Pareto thus pointed out that ordinal utility is all that isneeded for the scientific theory of choice.

QUANTITATIVE TESTS OFQUALITATIVE THEORIES: REVEALED

PREFERENCE

To those coming from the natural sciences, it cancome as a shock to discover that economists shy awayfrom assigning cardinal meaning to numerical utilities.Economists look askance at those who would assignany but the most qualitative of meanings to these util-ity numbers. A higher number means no more and noless than that an option is preferred. How much higherone number is than another is seen as essentiallymeaningless, largely thanks to Pareto. This is an abso-lutely central feature of economic thought that must beunderstood by anyone who interacts with economists.

But just because utility theory is ordinal in naturedoes not make it untestable. This was the insight of thenext revolutionary thinker in our story, the greatAmerican economist Paul Samuelson (1938). At a veryyoung age Samuelson posed a question of the utmostprofundity. It is arguably the most important questionever asked concerning how a scientific discipline cantest qualitative theories, and as such may be one of the

7QUANTITATIVE TESTS OF QUALITATIVE THEORIES: REVEALED PREFERENCE

NEUROECONOMICS

most important pointers to how economic reasoningcan best impact neuroscience.

Samuelson’s question was seemingly innocent. Heasked whether or not Pareto had gone far enough inpurging economic theory of unobservable elementslike utility numbers. He noted that Pareto had shownthat the marginal utilities inferred from price curveswere speculative at best, they rested on untestedassumptions, and from this he concluded that theyshould have no foundational role in economic theory.Maybe, Samuelson argued, the same verdict should berendered even on the notion of a preference order �maybe any notion of utility was too speculative andunconstrained to serve as the foundation of a real sci-entific theory. If it is choice data that we are trying toexplain, Samuelson argued, then maybe we shouldtreat choice itself as the fundamental datum aroundwhich our theories should be constructed.

Rather than introduce abstractions such as utilityand orderings and use them to derive implications forchoice, maybe we should start with choice data anduse it to infer the appropriate set of theoretical abstrac-tions from scratch. Samuelson urged, in essence, thateconomists stop thinking of utilities as causing choicesand instead begin to think simply about choices. It waswith this in mind that he pioneered an approach toeconomic theory that places the data itself � thechoices made by decision makers � center stage. Thisis a strategy now known as the revealed preferenceapproach.

Samuelson’s goal in building his approach was tospecify the exact dataset of choices that should be con-structed, and the precise test that would determine thevalidity or the lack of validity of the theory of utilitymaximization. What Samuelson said was: rather thanassume that subjects behave as if they are maximizingutility with their choices, why not figure out how todirectly test the hypothesis that the choices we observeare consistent with utility maximization? In developingthis approach, he started the economic profession onwhat has since become one of its main roads: thecharacterization of broad classes of theory in termsof the properties of an idealized data set designed totest that theory. From a Samuelsonian viewpoint, util-ity maximization is not an item of faith, but rather atestable hypothesis.

The actual process of arriving at the testable implica-tions of utility theory, however, was achieved in stages.Samuelson himself is best seen as having posed ratherthan resolved the decisive question. His own answerwas incomplete, but still critically important.

Samuelson developed his test of utility maximizationin the following way: if we observed that a subjectchose a when b was also available, then any theory thatsought to explain this as resulting from preferences of

any kind would necessarily involve the statement thatthe subject either preferred a to b or was indifferentbetween a and b:

agb or aBb; or equivalently ahb ð1:5Þwhere a and b are choice objects. (We refer to this as a“weak” preference.) It seems clear, Samuelson argued,that this would not be consistent with the individualhaving a strict preference for b over a:

bga ð1:6ÞTo express this in the language of choice: the fact

that the individual cannot strictly prefer b to a can beconveyed by the claim that choice of a over b withoutany money involved contradicts choice of b alone overa plus any positive amount of money. In other words:if a subject is observed to choose a over b, thenSamuelson states as a falsifiable hypothesis that hecannot also strictly prefer b to a.

Now what was important about this was thatSamuelson showed mathematically that anyone whosebehavior violates this rule cannot be described as obeyinga single underlying utility function. Someone who doesnot show this behavior cannot be maximizing a utilityfunction. Obeying Samuelson’s rule is thus a necessarycondition for utility maximization. That was amazingbecause it shifted economics away from very localstatements about particular models towards globaltests of key concepts.

Samuelson’s approach to choice theory provided ahugely important methodological pointer. It intro-duced the idea of testing all models in a key class atone and the same time. As such, it pre-figured lateradvances in physics. For much of its history, physicshas been concerned with making precise predictions.Yet occasionally a dispute occurs of such a fundamen-tal nature that a test of an entire model class seemsappropriate. Such is the case in quantum physics,where Bell’s celebrated inequalities are designed pre-cisely to test all local hidden variable theories. It is lit-tle appreciated that economists had discovered thisapproach to theorizing significantly earlier underSamuelson’s guidance. The reason though, is funda-mentally the same: in situations in which an importantclass of models may be true or false, one wants to per-form as holistic a test as possible before digging inthe weeds. Quantitative physical theories have oftenmatched measurement so well that this approach isentirely superfluous. Such is not the case for econom-ics nor its related discipline neuroeconomics.

To see the importance of the model class approachto theorizing, imagine that an economist had writtendown a specific computational model that employed aparticular class of utility function. That economist thenfitted the choice data with that model and obtained an


NEUROECONOMICS

R2 of 0.6. On these grounds the economist mightbe tempted to declare victory for his model. ButSamuelson’s theory says that if in that economist’sdataset there are choices that violate Samuelson’s rule,then there can be no logical doubt whatsoever that thiseconomist’s model is wrong. Even more strongly, wecan say that there is no model whatsoever, whichemploys a traditional utility function, which couldever account for that choice data.

Samuelson starts with a mathematical statement ofhis assumption: if a!b (if a is chosen over b) then itcannot be that b!a (b chosen over a), a kind of state-ment we refer to as an axiom (the precise modern state-ment has to sort out the annoying case in which thereis indifference between a and b, but that is a sideissue). He then goes on to ask: if that axiom is true,then what type of model can we use to analyze theobserved behavior? The answer is that this behaviorcan likely be modeled as involving maximization ofsome utility function, although we do not know whichparticular such function. And the beauty of thisapproach is that we can either use the axiomatic modelto predict utility-related behaviors, or we can use thebehavior to test the axiomatic model. Samuelson’smathematical theorem, his proof that obeying thisaxiom is a necessary condition for any utility represen-tation, is now widely known as the Weak Axiom ofRevealed Preference, or WARP, and it defined a mini-malist esthetic that dominates much of economics tothis day. How do theoretical assumptions on the driv-ing forces of behavior relate to the observed propertiesof the data of interest? How does refining the assump-tions further restrict observables? How does relaxingthe assumptions expand the data sets that can beobserved? Following Samuelson’s lead, these are thequestions economists ask everyday in their models.

So how would empirical testing of these kinds ofaxiomatic models work? Consider a subject who has$6 to spend on pieces of fruit (either apples or oranges)that cost $1 each. Effectively that subject is asked toselect a point from the shaded portion of the graphshown in Figure 1.3 which describes all of the combi-nations of apples and oranges he can afford. The darkline describes his budget constraint: this is the mostfruit he can afford to buy. If we observe that the sub-ject chooses a particular point a (4 apples and 2oranges) then we can conclude that no point along thedark line can be better than a for our subject or hewould have picked it. We can even go one step fartherif we assume that in this limited world more fruit isbetter than less fruit for this simple example � thatutility functions if they exist are monotonic. (We willshow how to do that, and how to avoid doing that,with an axiom, shortly.) Recall that we know thatevery other point along the line other than a is at least

as good as a for this subject from the fact that the sub-ject picked a. Now notice that every point inside thetriangle is strictly worse than at least one point on theline. From this, and our model, we can hypothesizethat anyone who chooses inside the grey triangle eithercannot be described as having a utility function or can-not be described as having a monotonic utility function.All this is demonstrated with just a few choices and apowerful bit of theory.

It is important to note at this point, however, thatSamuelson and his colleagues also stressed that whilebehavior can be used to infer utilities, it is the choicesand not the utilities that are observed. Utilities andchoices, he argued, are two sides of the same coin, butonly choice is directly observable. All of the majoreconomists of the mid-twentieth century stressed thatmaking statements about utility which are notanchored to choice directly leads to the kind of errorsthat Pareto had identified. This was a point ofimmense importance to Samuelson and his neoclassicalcolleagues � and remains one to economists today.That is hugely important because when a neurobiolo-gist says that “utilities cause choices” or that “neuralfiring rates in spikes per second are utilities,” he parrotsthe language of the nineteenth century economists thatPareto and Samuelson so completely rejected and dis-credited. He confuses a very carefully defined mathe-matical object with empirical measurements that differfrom that definition in important ways.

In any case, the importance of what Samuelson didwas to show that a tiny assumption, or axiomatic state-ment, could be used to make explicitly testable statements

1 2 3 4 5 6 70

1

2

3

4

5

6

7

0

Apples

Ora

nges

WARP

$6 line

a

FIGURE 1.3 A budget-set choice problem. A given chooser isasked how he would like to distribute his purchases of apples andoranges, given that he must spend all of his money. He selects thepoint a. This indicates that point a is at least as good, for him, asevery other point on the line and better than all of the shaded pointswhich strictly offer less.


NEUROECONOMICS

about the relationship between behavior and the class ofmodels that can rationalize that behavior. The importanceof this accomplishment simply cannot be overstated.We can either use the axiomatic model to predict utility-related behaviors, or we can use the behavior to testthe axiomatic model.

GARP

What followed the publication of Samuelson’s WARPpaper were a series of papers by important economiststhat extended his approach, the revealed preferenceapproach, typically by adding additional axioms toSamuelson’s basic model, or changing the basic axiom,and then asking what new predictions these additionalassumptions could support.

The most influential of these additional modelswas the one now known as the Generalized Axiom ofRevealed Preference, or GARP. Recall that obeyingWARP is a necessary condition for a utility representa-tion. But what is a necessary and sufficient condition fora utility representation? It was to answer that questionthat Hendrik Houthakker (1950) developed the GARPaxiom which essentially considered longer strings ofobjects that might be revealed preferred with one cho-sen when the others were available. For simplicity,consider a case with only three options revealed pre-ferred to one another in this manner. Then therewould have to be a relationship among these relationsthat did not cycle:

If ahb and bhc; then ahc ð1:7ÞWhat GARP says is simply this: imagine that we

observe a set of choices: a chosen over b, and b over c.Then a should also be preferred over c. A chooser whoobeys this axiom is one who is transitive in their prefer-ences, and for this reason GARP and a number ofderivative forms of this axiom are often referred to asaxioms of transitivity. What Houthakker showed wasthat obeying GARP was a necessary and sufficient cri-teria for a utility representation. If we see a chooserwho is transitive in their choices then we know, inprinciple, that there is at least one utility functionthat can fully account for their behavior. Houthakkerproved that saying a subject obeys the assumptions ofGARP is exactly the same as saying that he is in factmaximizing an utility function of some kind.

To put that another way: GARP proved that any sub-ject who shows no circularity in her choices behavesexactly as if she was really trying to maximize a utilityfunction. Her behavior in this regard does not tell usmuch about the shape of that utility function but doestell us that for her, a utility function could be describedthat fully accounts for her behavior.

This was an amazing accomplishment because insome sense it resurrected the utility function, but nowon more rigorous grounds and for the first time onentirely testable grounds rather than as a shadowyassumption. Now one could ask: is this subject’sbehavior describable, in principle, with some kind ofutility function? If her behavior obeys the rulesdescribed in the GARP model then the answer is yes.In a very real sense GARP reconciles the kinds of mini-malist arguments that Samuelson introduced with thereduced ordinal notion of utility which Pareto provedwas logically tenable. It made the simple theory ofrational behavior testable.

Understanding Rationality

Imagine for the sake of argument a chooser whoshowed the following preferences and who considers apenny an infinitesimally small amount of money:

apples horangesoranges h pearsBUT: pears h apples

How would such a chooser behave in the real-world? Imagine that this person had a pear andwe offered to sell her an orange for her pear plus 1 cent.She accepts. We then offer her an apple for the orangewe just sold her plus one more cent. She accepts. Then weoffer to sell her back her original pear for the apple plus1 cent. She accepts. At the end of these trades she haslost 3 cents, has her original pear in hand, and consid-ers each trade a good trade. This is a chooser who vio-lates the GARP model, for her the model is falsified.But because obeying the GARP model also means thatyou behave as if you were maximizing a utility func-tion, then we know that this subject’s behavior cannotbe described as an effort to maximize a monotonic util-ity function.

For an economist, such a person’s behavior isreferred to as “irrational,” where the word irrationalhas a very technical meaning. Irrationality here meansthat the subject has inconsistent preferences � that sheviolates GARP. This is a very important point and onethat has generated no end of confusion in the interac-tions of economists with normal people. If our choosertells you that she loves apples over all other fruits,there is nothing irrational about this in the economicsense. If she reveals that she prefers fruit to love, noth-ing irrational (in the economic sense) is implied. If aperson is rational in the economic sense it simplyimplies that one cannot pump money or fruit out ofher as in the above example. It implies that her prefer-ences are internally consistent � and thus by GARP itimplies her behavior can be described with a utility function


NEUROECONOMICS

and thus as a form of utility maximization. It places noconstraints on what she likes, how she feels aboutthose likes or dislikes, or even whether the things shelikes include illicit drugs. To an economist rationality isconsistency and nothing more.

Axiomatic Approaches: Strengths

For someone trying to build a model of a naturalworld phenomena, “models” such as GARP are bothinteresting and frustrating. GARP is a theory aboutrepresentation. Or put even more precisely: GARP is atheory about theories of representation. GARP statesthat if a set of observed choices obeys some very mini-mal constraints, then the chooser is behaving exactlyas if they had internal utility functions. What is frus-trating about this is that GARP does not tell us muchabout what those utility functions look like. It is not atheory that tells us: this chooser has a utility functionof the form:

Utility5 #apples0:5 ð1:8ÞOr

Utility5 12 eð2#apples12r=# applesÞ ð1:9ÞInstead, what it tells us is something that is at the

same time weaker and more powerful. It is weaker inthe sense that it is a theory about the representation ofvalue that places very few constraints on that represen-tation. It is, however, much more powerful when onethinks about it with regard to the standard logic of sci-entific inquiry in which our goal is not to prove theo-ries but rather to falsify them. If we observe that asubject’s choices violate GARP then we can concludethat: there is no model of any kind that rests on a singleutility function that can describe this behavior. It is in thissense that the economic approach is most powerful.

To make this distinction clear consider two scientistsstudying the same choice behavior. The first tries toaccount for the observed choices of a subject by fittingthe data with a power utility function. The best fit is:

Utility5 #apples0:4532 ð1:10ÞBut the scientist also observes that the variance

accounted for by this model is only 20% of the totalvariance in the dataset. So this looks like a bad model.The next step is to try another utility function andrepeat the process, slowly getting better and bettermodel fits. Now consider a scientist that looks at theraw choice data and asks if he can see direct violationsof GARP. That scientist observes that about half of thechoices made by our subject violate the axiomatic basisfor understanding utility functions that GARP pro-vides. What is interesting is that this second scientist

knows something that the first does not. The secondscientist knows that no one will ever succeed in find-ing a utility function for describing this subject’sbehavior because GARP proves that no such functionexists. GARP thus allows us to ask a first order ques-tion in a very global way: is there at least one memberof the class of all possible monotone utility functionsthat can account for the observed data?

Axiomatic Approaches: Weaknesses

Of course, tools like GARP and WARP are also quitelimited. There are many (actually an infinite numberof) utility functions compatible with GARP-like orWARP-like behavior. If, for example, we know that achooser obeys GARP, we know that she cannot behaveintransitively over any of the objects of choice we haveexamined. She cannot, for example, prefer apples tooranges, oranges to pears and also prefer pears toapples. We also know that a utility function exists thatcan account for her behavior. In fact, we know that arelated family of functions exists which can account forher behavior. The most important limitation that wemust keep in mind, however, is that not only are weignorant of the “right” function but we also know thatthere is no unique “right” function, only a family of“right” functions � all equally valid. This was Pareto’spoint when he developed the notion of ordinal utility.

To develop that point, imagine one demonstratesthat a particular chooser obeys GARP in her choicesamongst fruits. Then imagine that we find (perhaps byfitting some function to her choice data) a utility repre-sentation that predicts choices amongst fruits or bun-dles of fruits. The utility of an apple5 3.0, anorange5 2.0, a pear5 1.0. Great. We actually do knowsomething but it is critical to remember that the num-bers we are using are basically made up. Their order isconstrained by the data but the actual magnitude ofthese numbers is not very much constrained, this wasPareto’s point. We know from observed choice that anapple is better than a pear, but in no sense do weknow that an apple is exactly three times better than apear. That is simply going beyond our data � a factthat the GARP approach makes clear because of thevery weakness of the constraints it imposes. If wesquared all of the values in our utility list so thatapple5 9.0, orange5 4.0, and pear5 1.0, our modelwould have exactly the same predictive power as itdid before. Any monotonic transform of these utilitynumbers will preserve choice ordering and preservecompliance with GARP. Of course we also know thatwe cannot take the reciprocal of each of these num-bers. This is a non-monotonic transform that GARPtells us cannot work with these data.


NEUROECONOMICS

To summarize, what the neoclassical approach pro-vides are very minimalistic “models.” These modelsare built out of logical statements about choice calledaxioms. A good model of this type has three importantfeatures. First, the statements are concise, easy tounderstand, and their truth or lack of it is of interest.Second, an underlying mathematical proof relateschoices that obey these axioms to a clear theory ofvalue or utility. Third, the theory of value described bythe axioms cogently defines a large class of importantsub-models and this means that falisifying an axiom-atic model, finding choice behavior at variance withthe axiomatic statement, unambiguously falsifies thisentire class of models. That is the core of this approach.

EXPECTED UTILITY THEORY

One thing that GARP does not really let us do, how-ever, is to understand how the subjective value ofapples grows continuously as their number increases,it does not allow us to examine the form of the utilityfunction in the way the marginal revolutionists hopedthat they could with their graphs of exchange-value. Italso does not allow us to incorporate into our theoryof value, in any sense, the hypothesis that a 50%chance of winning $20,000 is exactly twice as good at a25% chance of winning that same amount. Is there anextension of these original models that can do both ofthese things, place stronger constraints on the shape ofthe utility function and make explicit hypotheses aboutprobabilities � but which can be built up from thisbasic revealed preference approach? The standardanswer to that question is yes, and the axiom grouptypically used to generate that answer forms the coreof Expected Utility Theory.

Expected Utility theory, initially developed by Johnvon Neumann and Oskar Morgenstern (1944), wasdesigned to develop a model of decision making thatwould enable the value of what are called “mixedstrategies” to be computed in the theory of games.That is a point taken up in the next chapter. To thatend, though, they developed a remarkable and explicittheory of value (somewhat like the one Bernoulli hadproposed) using the more rigorous neoclassical eco-nomic approach.

In the cases Samuelson studied, subjects werechoosing between objects such as apples and oranges.To analyze these so-called mixed strategies vonNeumann and Morgenstern had to extend their theoryto cover choices amongst things like lottery tickets.Which does a chooser prefer: a 50% chance of winningan apple or a 28% chance of winning a pear. Of courseWARP and GARP allow you to ask that question, butwith one important limitation. Neither WARP nor

GARP treat a 28% chance of winning a pear and a 29%chance of winning a pear as related. For WARP andGARP, 28.999999% and 29% are no more related thanare apples and oranges. Neither of these super-compact theories of choice give us the tools to treatsimilar probabilistic outcomes as related � as havingrelated utilities. To accomplish the goal of using a neo-classical approach to describe choice under uncertaintyand to describe utility function shape was a naturalendpoint for the neoclassical approach, but one thatthe ordinal axioms did not accomplish. To achieve thatgoal, von Neumann and Morgenstern started withessentially the same axiom that forms the core ofGARP, the transitivity axiom:

If ahb and bhc; then ahc ð1:11Þbut they added three additional pieces.

Defining the Objects of Choice: ProbabilisticOutcomes

von Neumann and Morgenstern’s first step was todefine the kinds of objects people would be asked tochoose between in their formulation in such a way thatuncertain, or probabilistic, events could be studied andrelated. They did this by describing an object of choicethey called a lottery, an object of choice defined by twonumbers: a probability and a value. So, for example,one might choose between a 50% chance of gaining apear and a 25% chance of gaining an apple. Both ofthese are referred to formally as lotteries, and are com-posed of the thing you get, often called a prize and aprobability that you will get that thing.

It is natural to think of these choice objects as beinglike the lottery tickets sold at casinos or newstands,and indeed one often sees that description used, butit is important to stress that von Neumann andMorgenstern meant this to be a general way of talkingabout any kind of decision that had uncertain results.

Continuity Axiom

Given this new way of talking about objects as prob-abilistic events, what we next need is to develop anaxiom which ensures that there are no abrupt jumps inpreference as a tiny change is made in the probabilityof receiving some prize. The goal is to communicatethis idea as a testable minimalist rule, and one thatwill contribute to our overall theory of value in a use-ful way. To achieve this goal von Neumann andMorgenstern settled on the continuity axiom.

If ahbhc, then there exists a unique probability (p)such that:

bB pa1 ð12 pÞc ð1:12Þ


NEUROECONOMICS

Imagine we establish that you prefer apple toorange to pear. The continuity axiom simply says thatthere must be some lottery between an apple (whichyou prefer to oranges) and a pear (which you do notprefer to oranges), that is exactly equal in subjectivevalue to an orange. The continuity axiom rules outirrevocable preferences like: I would rather have noth-ing than $1000 if this amount of money came witheven the smallest possible risk of getting stuck with apear. As such, it goes much of the way to implying theexistence of a continuous utility function.

Independence

There is something quite fundamental about lotter-ies that suggests that preferences over them may havestructure that is missing in the standard case of applesand oranges � and this is the second feature that vonNeumann and Morgenstern attempted to capture. Thisis the fact that a lottery can only have one outcomeregardless of its probabilistic structure. After all, it can-not both rain and not rain at the same time, so that theprize it rains can be thought of as, in principle, inde-pendent from the prize it does not rain. This has aninteresting implication: if you prefer one lottery toanother, and then mix these with a common lotterythat is equally likely in both cases, then the commonlottery may be relatively unimportant in the compari-son. After all, the prize is either the common lottery orit is not. If it is, then there is no difference between thecompound lotteries. If it is not, then we know whichone is preferred. So there should be no change in pref-erence if we add a common prize to a pre-existing pairof lotteries. Here is the corresponding technicalstatement:

If ahb then xa1 ð12 xÞchxb1 ð12 xÞc ð1:13Þ

where c is a third lottery and x is a number between0 and 1.

According to this assumption, a subject whoprefers:

an 80% chance of winning $100 and 20% chance ofwinning nothing overa 40% chance of winning $200 and 60% chance ofwinning nothing

also prefers:

a 40% chance of winning $100, 40% chance ofwinning $10, and 20% chance of winning nothingovera 20% chance of winning $200, 40% chance ofwinning $10, and 40% chance of winning nothing.

This follows because one can regard the two lotter-ies compared in the second case as 50�50% mixturesof the lotteries listed on top with the common lotterythat offers an 80% chance of winning $10 and a 20%chance of winning nothing.

It should be noted that this is the most difficult ofthe expected utility axioms for humans to follow, andthe one that, if explicitly tested, often fails. Kahnemanand Tversky’s (1981) famous observation that peopleover-emphasize low probabilities is a violation of thisaxiom.

The Expected Utility Theorem

So what does all of this mean with regard to a the-ory of value? If we say that we have a chooser whoobeys in her choices the axioms of GARP, that turnsout to be the same as saying that we have a chooserwho behaves as if she had a utility function. What vonNeumann and Morgenstern proved was that if wehave a chooser who obeys GARP plus the continuityand independence axioms, it is the same as sayingthat: (i) she has a utility function (as in GARP); and (ii)she computes the desirability of any lottery by multi-plying the utility of the prizes presented in the lotteryby the probability of those prizes being realized.

To put that another way, an expected utility compli-ant chooser behaves as if she chooses by multiplyingprobability by utility. What should be immediatelyobvious is that von Neumann and Morgenstern havegot us almost back to where Bernoulli left us � but ina much more powerful way. Bernoulli had given us abunch of assumptions about the form of the humanutility function and about what was encoded, whatwas represented, and how those variables interacted.But he had never given us those rules in a waythat allowed us to test the choice behavior of subjectsto see if those rule were correct. Bernoulli’s theorywas essentially unfalsifiable. What von Neumann andMorgenstern did was to give us a few simple rulesthat may describe choice behavior. If you could provethat a chooser obeys those axioms you know quite abit about how she constructs and represents subjectivevalues, at least in a theoretical sense. If you couldprove that the chooser did not obey those choice rulesyou have falsified the theory, at least with regard tothat chooser and that set of choice objects.

A second feature of expected utility theory is that itcan be interpreted as offering hope concerning mea-surement of utility. If we observe a subject whosechoices obey the axioms of expected utility, and weobserve that she prefers apple to orange to pear, andfinds a 50% probability of winning an apple and a 50%chance of winning a pear exactly the same as a 100%

13EXPECTED UTILITY THEORY

NEUROECONOMICS

probability of winning an orange, then there is a well-defined sense in which: she values an apple over anorange exactly as much as she values an orange over a pear.One cannot stress enough what a huge advance thisis for our theory of value. von Neumann andMorgenstern essentially showed how to use probabil-ity as a ruler to measure the relationships between thevalues of different prizes for subjects who obey theiraxioms. Using the von Neumann and Morgenstern, or“vNM,” approach we can get past the ordinal bottle-neck Pareto identified, to some degree.

There are, however, two caveats to the resuscitationof numerical utilities suggested by the vNM approach.First this advance doesn’t get us to a unique numberthat we can assign as the utility of an object. Utilitiesare still only measurable relative to other objects. Thismeans that if we choose to say that an apple has anutility of 4 and a pear has an utility of 2, we can alsojust as accurately say that an apple has an utility of40 and a pear has an utility of 20, but our notion ofutility certainly has more bite than it did underGARP, and we know how to determine when thisadditional bite is warranted by testing the axioms ofthe theory on behavior. Second, there are many otherutility functions on lotteries that can explain the samebehavior, albeit without preserving the expected util-ity property.

Axioms and Axiomatic Reasoning

Much of the economic theory of choice involves plac-ing axioms on choice behavior and developing theoriesof value to which these axioms correspond. The theoriesof value, which incorporate objects like expected utility,are of course conceptual objects. To an economist theyare mathematical tools for predicting choice and noth-ing more. They are not real physical events or thingsthat can be measured directly. But what we will beginto ask in the pages that follow is whether theoreticalconstructs like utility, or the axioms themselves, can berelated directly to the objects that populate the theoreti-cal descriptions of neuroscience and psychology.

The objects that we need to keep most careful trackof for right now are those related to expected utilitytheory. These are the notions that for some chooserswho obey the axioms of expected utility we can treatas if they:

• construct utility functions for things like apples andoranges;

• multiply the utility of each object by the probabilityof obtaining that object to yield an expected utility foreach option in their choice set;

• select the option having the highest expected utility.

However, we also need to acknowledge several pro-blems with expected utility theory in particular andwith our current models of choice. These points aretaken up in more detail in Chapter 3. The greatestpower of these current theories is that they are falisifi-able. Their truly greatest weakness is that we cantherefore prove that most of these theories are false, atleast under some conditions. When the probabilities ofan outcome are small, for example, we know that mosthumans violate the independence axiom. When chil-dren are younger than about 8 years old we know thatthey violate GARP (Harbaugh et al., 2001). These arereal problems and they have generated no end of ten-sion in the economic community. But more immedi-ately, we have to ask whether this kind of axiomaticreasoning can be applied in a neuroeconomic, or neu-roscientific context. Perhaps surprisingly, that turnsout to be the case. As an example, we turn next to anapplication of this kind of rigorous, parameter free the-ory to the neurobiological hypothesis that the dopami-nergic neurons of the midbrain create a rewardprediction error signal appropriate for guiding rein-forcement learning � a problem taken up again inChapters 15, 16, 17, and 18.

USING AXIOMS: THE NEOCLASSICALAPPROACH IN NEUROECONOMICS

The Reward Prediction Error Hypothesis

Dopamine is a neurotransmitter whose release wasoriginally hypothesized to reflect “hedonia,” a conceptclosely related to notions of reward and utility. Morerecent studies have, however, challenged that view.These more recent studies of midbrain dopaminergicneurons (reviewed in Chapters 15�18; Montague et al.1996; Schultz et al., 1997) have suggested that the activ-ity of these neurons, rather than encoding rewardsper se, encodes instead a kind of teaching signal thatcan guide learning about rewards. The basic hypothe-sis developed in this line of research is that the mid-brain dopamine neurons broadcast a “predictionerror” signal precisely of the form needed in reinforce-ment algorithms to drive convergence toward a stan-dard dynamic programming value function (Suttonand Barto, 1998). Dubbed the dopaminergic rewardprediction error (DRPE) hypothesis, this claim hasbeen tested using a variety of neuroscientific techni-ques but has remained controversial. (For an alterna-tive view of these findings see Chapter 18.)

If true, the DRPE hypothesis suggests that there is aprofound bridge between neuroscience and the theoryof choice. Economists are just as interested as neuros-cientists in rewards, in predictions, and in errors. By


NEUROECONOMICS

definition, rewards must be attractive to a decisionmaker, and hence natural objects of choice. Predictionsrelate closely to lotteries, the objects of choice inexpected utility theory. Finally, errors relate to someform of gap between the anticipated lottery and therealized prize in a lottery. It is no wonder that theDRPE hypothesis is so central to Neuroeconomics.

While there was immediate interest among econo-mists in understanding the role of dopamine, the earlywork on the DRPE did not translate well into the lan-guage of economics. In part this was due to deep dif-ferences in the research traditions in neuroscience andin economics, and in part it was a result of even dee-per and more basic conceptual challenges.

One key barrier that early on prevented economistsfrom picking up the ball and running with it was thatthe DRPE hypothesis remains controversial withinneuroscience. While follow-up studies have tended tobroadly confirm the DRPE hypothesis, several of thesestudies have uncovered other factors that may influ-ence the activity of the midbrain dopaminergic neu-rons that are only tangentially related to the DRPE. Forexample, the qualitative fMRI studies of Zink et al.(2003); Delgado et al. (2005); and Knutson and Peterson(2005) suggest that dopamine responses may be modu-lated not only by a reward prediction error but also byless reward-specific properties referred to as “sur-prise,” and/or “salience.”

A second challenge has been that traditional meth-ods of testing the DRPE hypothesis make strong func-tional form assumptions on both utility and thelearning process in order to make quantitative predic-tions (Bayer and Glimcher, 2005; Daw et al., 2006; Liet al., 2006; Montague and Berns, 2002; O’Dohertyet al., 2003, 2004). Formally, these tests combine thegeneral form of the DRPE hypothesis with a series ofassumptions concerning the precise form of the rewardfunction and how strongly past rewards impact learn-ing about likely future rewards. Given these assump-tions, it is possible to estimate the reward predictionerror associated with any sequence of stimulus-prizepairs. Measured fMRI activity in dopaminergic outputareas can then be regressed on this sequence, with astrong positive correlation taken as confirmatory of theDRPE hypothesis.

A third challenge is that those coming from the eco-nomic tradition do not regard the utility and learningassumptions that are tested jointly with the DRPE withparticular favor. While it is true that prizes such asfruit juice and money come in natural units (the vol-ume of juice and the amount of money), and that moreis generally better, the economic tradition indicateshow perilous it is to translate this into a specific quan-titative statement in numerical units. Moreover, econ-omists are uncomfortable hanging the theory of

dopaminergic function on a mechanical theory of rein-forcement learning. Bayesian models may make quitedistinct predictions from reinforcement models inmany environments. For example, good news aboutthe payoff in a particular setting may well be associ-ated with good news about another option in the nextperiod, rather than being a cue to reinforcement ofthe tendency to take the given option. This may bedue to a pattern of change that has previously beenuncovered in the optimal strategy or simply due toan appreciation on the part of the decision maker thatsuch patterns are possible. The DRPE may be true, yetan entirely different mode of learning may be inevidence.

A fourth barrier to communication is that the calcu-lated prediction error is very highly correlated with,and could therefore be similarly explained by, otherrelevant measures such as the magnitude of thereward itself. While one might use statistical methodsto discriminate among models, these will at best pro-duce a ranking of the considered alternatives, ratherthan a global statement on model validity.

A final issue is that, from the viewpoint of economicmethodology, the standard testing protocol from neu-roscience has profound limitations. Even if the DRPEhypothesis is correct in the most literal of senses, andreinforcement learning is taking place as modeled, thecurrent hypothesis test could reject it due to theimplicit assumption that the fMRI signal is related tothe prize according to a simple transformation. AsPareto noted in the case of utility theory, the DRPEhypothesis does not have natural units of measure-ment. In its essence, it is a theory of greater and smal-ler responses, not of exact numerical magnitudes ofthese responses.

So where to turn? One hint lies in the fact that thecontroversies concerning the role of dopamine are quali-tative as well as quantitative in nature. Zink and collea-gues (2003) and Aron and colleagues (2004) suggest thatdopamine output responds in part to “unpredictability”per se, or “salience” rather than just to reward predictionerror. The “incentive salience” hypothesis of Berridgeand Robinson (1998; also covered in Chapter 18) holdsthat dopamine responds to how “wanted” a stimulus is,which is separate from how much a stimulus is “liked.”Redgrave and Gurney (2006) suggest that dopamineplays a role in “switching attention” between differentpossible events.

This suggests that there may be value in steppingback and taking a “model class” approach of the kindfundamental to axiomatic economic analysis. It wasthis that led Caplin and Dean (2008) to propose thatneuroeconomic research follow the methodologicallead of Samuelson and develop model class methodsand apply them to the case of dopamine. This required

15USING AXIOMS: THE NEOCLASSICAL APPROACH IN NEUROECONOMICS

NEUROECONOMICS

a simple restatement of the theory to capture only itsmost essential features and a complementary experi-ment aimed at testing these features.

The DRPE Axioms and the Ideal Data Set

The question that the above controversies raise con-cerns how exactly to get to the heart of the DRPEhypothesis. Is there a way of phrasing the hypothesisthat:

1. makes few if any assumptions about the cardinalfeatures of rewards associated with better actions;

2. makes few if any assumptions about how learningtakes place;

3. is clearly testable in an experimentally feasible dataset;

4. is directly useful in discriminating between theDRPE and alternative hypotheses?

The extent of the challenge is made clear by focusingon the key words that define the hypothesis. Rewardsare hard to treat as objectively measurable in naturalunits, as we know only too well from the review of choicetheory. Predictions themselves are extremely complex,often rely on specific assumptions, and it is only in thevery most trivial of environments that one can hope totreat them as understood. It may be reasonable to modelthe belief that a fair coin will come down heads as 0.5,but how beliefs evolve given typical sequences of out-comes in which learning is taking place is neither theoret-ically nor experimentally well-understood. Errorsrepresent the perceived gap between expectations andoutcomes, and are a fortiori hard to understand in all butthe simplest of settings.

That these challenges can be successfully overcomeis the message conveyed by the axiomatic model of theDRPE of Caplin and Dean (2008) and the correspond-ing experiment implemented by Rutledge et al. (2010).There are three steps involved in formulating and test-ing this hypothesis in its appropriate abstract form:

1. first, one must specify an “Ideal Data Set” on whichto test the theory;

2. second, one must identify the minimal features ofthis data set that would characterize the DRPE;

3. finally, one must implement the theory to establishfeasibility and to conduct the proposed tests.

The first step, that of identifying the ideal data, wasrelatively easy in the historical case of choice theory.Anyone who tried to develop a theory of choice with-out believing that choice data was of the essencewould be revealing clear confusion. The key “observa-bles” are less apparent in the case of the DRPE. Again,a simple review of key words provides important

guidance. Dopaminergic responses clearly must bemeasured in relation to some error in predicting areward. To define a reward, there must be “prizes” ofsome form. To define a prediction, there must bebeliefs as to what these prizes might have been. Todefine an error, one must have a prize as an outcomethat may or may not align with expectations. The sim-plest environment that animates all of these conceptsis one in which subjects face clear uncertainty concern-ing an upcoming prize, in which there are consistentrules that determine the actual prize that will be forth-coming. The paradigmatic example involves one oftwo prizes being realized depending on the flip of afair coin, with the outcome of the flip determining, inan easily understood manner, precisely which prizewill be forthcoming. The only additional flexibilityneeded involves considering cases with more possibleprizes and unequal if still clear prize probabilities.

Suppose now that we observe the resolution of simplelotteries that may result in a well-defined form of sur-prise, and then study the dopaminergic responses. Thisthen raises the second question: what exactly are thedefining features of a neural system responding toexpected and realized rewards, and responding more,the more positively surprising was the realization rela-tive to the expectation? Caplin and Dean argue that threefeatures, or axioms, are necessary if one is to claim thatany system reflects a reward prediction error signal.Each feature is intuitively tied to a particular word defin-ing the hypothesis.

Reward. The system should reflect a Coherent PrizeOrdering: If one prize ever produces a largerdopaminergic response than some other prize for agiven anticipated lottery, it must always produce alarger response regardless of the identity of theanticipated lottery.Prediction. The system should reflect a Coherent LotteryOrdering: If one anticipated lottery ever produces alarger dopaminergic response than some otheranticipated lottery for a particular prize, it must alwaysproduce a larger response regardless of the prize.Error. The system should satisfy No SurpriseEquivalence: If a prize is perfectly anticipated, theprecise dopaminergic response should beindependent of the particular prize.

According to the standard methods of axiomaticmodeling, one simply lists these as minimal desiderata,and then sees whether or not they are mutually consis-tent, and if so whether or not they connect to some moreor less simple technical representation of what exactlythe dopamine system does in transforming lottery beliefsand outcomes into action potentials. It is only at this finalstage that mathematical reasoning is required, since theremay be either real conceptual problems (as when a series


NEUROECONOMICS

of seemingly reasonable axioms are mutually inconsis-tent) or additional technical issues that are not readilyavailable to intuition. Fortunately, Caplin and Dean(2008) found simple and positive answers to these ques-tions, confirming that the three axioms above indeedidentify the essence of the DRPE hypothesis.

In addition to showing the hypothesis in its mostessential characteristics, the axiomatic method hasvalue in guiding experimental design. The axioms sug-gest experimental protocols directed to the centraltenets of the theory, rather than to particular parametri-zations. The simplest possible experiment for modeltesting was developed and implemented in Rutledgeet al. (2010). It produced intriguing findings in a varietyof dimensions that warrant further exploration. Theirwork was interesting because it used the Caplin andDean axioms to generate unambiguous conclusionsabout the DRPE hypothesis that are discussed later inthis volume. But the work is relevant to this chapterbecause it established the relevance and utility of theeconomic approach to purely neuroeconomic issues.

Concluding Remarks

The above provides just a first taste of what maybecome an increasingly important aspect of neuroeco-nomics. The key is the three step process sketched outabove for formulating and testing hypotheses as theyform and as they mature, to which we add a fourthstage aimed at theory development.

STEP 1: Specify an “Ideal Data Set” on which to testthe theory.STEP 2: Identify the minimal features of this dataset that characterize the theory in question.STEP 3: Implement the theory to establish feasibilityand to conduct the proposed tests.STEP 4: Adjust and expand the theory as the firsttests suggest. Return to Step 1.

If this method grows in acceptance, neuroscientifictheory and measurement stand to be enhanced in man-ners that are currently impossible to anticipate. At thesame time, communication with economists will begreatly enhanced. Neuroeconomic theory and mea-surement will at that point provide the perfect bridgebetween these disciplines.

References

Aron, A.R., Shohamy, D., Clark, J., Myers, C., Gluck, M.A., Poldrack,R.A., 2004. Human midbrain sensitivity to cognitive feedback and

uncertainty during classification learning. J. Neurophysiol. 92 (2),1144�1152.

Bayer, H.M., Glimcher, P.W., 2005. Midbrain dopamine neuronsencode a quantitative reward prediction error signal. Neuron.47 (1), 129�141.

Berridge, K.C., Robinson, T.E., 1998. What is the role of dopamine inreward: hedonic impact, reward learning, or incentive salience?Brain Res. Rev. 28 (3), 309�369.

Caplin, A., Dean, M., 2008. Dopamine, reward prediction error, andeconomics. Q. J. Econ. 123 (2), 663�701.

Daw, N.D., Courville, A.C., Touretzky, D.S., 2006. Representationand timing in theories of the dopamine system. Neural Comput.18 (7), 1637�1677.

Delgado, M.R., Miller, M.M., Inati, S., Phelps, E.A., 2005. An fMRIstudy of reward- related probability learning. NeuroImage. 24 (3),862�873.

Harbaugh, W.T., Krause, K., Berry, T., 2001. On the development ofrational choice behavior. Am. Econ. Rev. 91 (5), 1539�1545.

Houthakker, H.S., 1950. Revealed preference and the utility function.Economica. 17 (66), 159�174.

Knutson, B., Peterson, R., 2005. Neurally reconstructing expectedutility. Games Econ. Behav. 52 (2), 305�315.

Li, J., McClure, S.M., King-Casas, B., Montague, P.R., 2006. Policyadjustment in a dynamic economic game. PloS ONE. 1 (1),103�113.

Montague, P.R., Berns, G.S., 2002. Neural economics and the biologi-cal substrates of valuation. Neuron. 36 (2), 265�284.

Montague, P.R., Dayan, P., Sejnowski, T.J., 1996. A framework formesencephalic dopamine systems based on predictive Hebbianlearning. J. Neurosci. 16 (5), 1936�1947.

O’Doherty, J.P., Dayan, P., Friston, K., Critchley, H., Dolan, R.J.,2003. Temporal difference models and reward-related learning inthe human brain. Neuron. 38 (2), 329�337.

O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K.,Dolan, R.J., 2004. Dissociable roles of ventral and dorsal striatumin instrumental conditioning. Science. 304 (5669), 452�454.

Pareto, V., 1906, 1971. Manuel d’Economie Politique (Manual ofPolitical Economy), A. S. Schwier (trans.). Augustus M. Kelley,New York.

Redgrave, P., Gurney, K., 2006. The short-latency dopamine signal: arole in discovering novel actions? Nat. Rev. Neurosci. 7 (12),967�975.

Ricardo, D., 1817. On The Principles of Political Economy andTaxation. John Murray, London.

Rutledge, R.B., Dean, M., Caplin, A., Glimcher, P.W., 2010. Testingthe reward prediction error hypothesis with an axiomatic model.J. Neurosci. 30 (40), 13525�13536.

Samuelson, P.A., 1938. A note on the pure theory of consumer’sbehaviour. Economica. 51 (17), 61�71.

Schultz, W., Dayan, P., Montague, P.R., 1997. A neural substrate ofprediction and reward. Science. 275 (5306), 1593�1599.

Smith, A., 1776. The Wealth of Nations. Strahan and Cadell, London.Sutton, R.S., Barto, A.G., 1998. Reinforcement Learning: An

Introduction. MIT Press, Cambridge, MA.Tversky, A., Kahneman, D., 1981. The framing of decisions and the

psychology of choice. Science. 211, 453�458.von Neumann, J.V., Morgenstern, O., 1944. Theory of Games and

Economic Behavior. Princeton University Press, Princeton, NJ.Zink, C.F., Pagnoni, G., Martin-Skurski, M.E., Mukeshwar, D., Berns,

G.S., 2003. Human striatal response to salient nonrewarding sti-muli. J. Neurosci. 23 (22), 8092�8097.

17REFERENCES

NEUROECONOMICS

http://refhub.elsevier.com/B978-0-12-416008-8.00001-2/sbref1
















































































neuroeconomics || basic methods from neoclassical economics

Documents