experimental economics: short course universidad del desarrollo santiago, chile december 17, 2009

Experimental Economics: Short CourseUniversidad del Desarrollo

Santiago, ChileDecember 17, 2009

Dr. Jonathan E. AlevyDepartment of Economics

University of Alaska [email protected]

Field Experiments

• Field experiments combine elements of lab experiments– Clean counterfactual– Randomization to treatment– Ability to collect auxiliary data (e.g. risk preferences)– Salient incentives (pay based on performance)

• And traditional field studies– Relevance to participants & natural setting

• “Home-grown” values• Salient incentives

Field Experiments in the Empirical Toolkit

AFE_____FFE______NFELab field experiments Uncontrolled Field data

• conventional lab experiment (Lab)– employs a standard subject pool of students, an abstract framing, and an

imposed set of rules• artefactual field experiment (AFE)

– same as a conventional lab experiment but with a non-standard subject pool

• framed field experiment (FFE)– same as an artefactual field experiment but with field context in the

commodity, task, information, stakes, time frame, etc.• natural field experiment (NFE)

– same as a framed field experiment but where the environment is the one that the subjects naturally undertake these tasks, such that the subjects do not know that they are in an experiment

Learning from artefactual experiments• Subject pool effects– Relevance seen in lab as well.

• Students from different schools, different majors etc. • Field Setting: – Selected samples

• Access to “high-level” decisionmakers– Corporate executives play a variant of the trust game (Costa

Rica).» Fehr & List 2004

– Financial market professionals: information processing and cascades.» Alevy, Haigh & List, 2007

– Representative samples• Risk & time preferences in Denmark• Relevant for interpreting welfare effects of policy

– Harrison et al. 2002, 2007

CEO’s, Trust, and Punishment

• Two person trust game:– Player 1 has 10 units of currency– First move: Can pass x {0, 1, …,10} to player 2– Amount passed is tripled.– Second move: Player 2 with 3x can pass any

amount , y {0, 1, …,3x}, back to player 1

CEOs, Trust, and Punishment• Fehr & List (FL) variant

– Player 1: Along with x, requests a specific amount, y*.– Can punish for deviations– Can waive right to punish in advance of transfer

• Waive punishment FL Variant = original trust game • Key findings

– Waiving right to punishment is reciprocated more returned– CEOs significantly more likely to waive right than baseline

(students) and in general more trusting/trustworthy.• Selection into CEO status different profile than students

• Notes:– Empirical strategy: Random effects tobit model (censored data)– Protocol uses neutral language

• “conditional pay cut” not “punish”• “Decision problem” not “trust game”

Further Afield: Framed Experiments

• Introduce context relevant to the subjects– E.g. vary the media in Holt/Laury risk elicitation. • Cash used as a control in each case. • Harrison, List & Towe, 2007 Econometrica

– Coin collecters, treatment varies what is known about quality of coins.

– Coins with low uncertainty similar to cash.

• Alevy, Cristi, & Melo, 2009– Elicit risk preferences using lotteries for water. – Elicited attitudes using water treatment help explain bidding

behavior in separate water auctions.

Chilean Water Market Experiments

• How to sell multiple heterogeneous items?– Estate sales– Sale of condos within a new complex

• Bidder’s choice or ‘Right-to-choose’ (RTC) auctions are a candidate institution– Rather than bid on specific commodity

• Compete for the right to select preferred item from available set

– Idea: If little interest in any particular good• Could competition across goods bolsters prices?

Chilean Water Market Experiments II

• Early evidence on RTC from uncontrolled field data– Ashenfelter Genesove 1992 AER– Nonexperimental evidence on condominium prices– RTC versus bilateral bargaining• Price premium in RTC

• Experimental approach to understanding RTC– A cleaner counterfactual is possible– Good-by-good or sequential (SEQ) auction is benchmark– Observe impact of relevant changes in auction rules

RTC vs. Sequential

• Theory: a two-bidder, two-good model– Theory based on Burguet (2007)– With risk-aversion RTC auctions raise more revenue than

SEQ– Intuition: Possibility that preferred good is chosen in

early rounds risk averse bid more in RTC

• Previous laboratory findings– Goeree et al. 2004– Eliaz et al. 2008– mixed evidence on theory: none control for risk

attitudes

Field Experiments Designs

• Protocol– Salient payoffs, winner pays for good– Second price auction rule– Independently elicit risk attitudes & time preferences

• Chilean Auctions– 1 RTC Auction 20 bidders– 1 SEQ Auction 18 bidders

– Subjects are farm owners and managers in Limari Valley study area

Fase1 del Ejemplo de la Subasta

Resultado de la Subasta

Licitador Precios Producto

ID 1 25,000 3 Silla

ID 2 27,000 1 Bicicleta

ID 3 36,000 3 Silla

ID 4 50,000 2 Computadora






ID 3 36,000 3 Silla



ID 1 25,000 3 Silla





ID 3 36,000 3 Silla



ID 1 25,000 3 Silla


1

2

3




ID 3 36,000 3 Silla



ID 1 25,000 3 Silla


Ganador de la SubastaPrecio

Pagado Producto


Fase 2 del Ejemplo de la Subasta



ID 1 25,000 3 Silla


ID 3 36,000 3 Silla

ID 4 15,000 2 Silla

ID 5 5,000 2 Silla


Ganador de la Subasta Precios Producto

???? ???? ????



ID 3 36,000 3 Silla



ID 1 25,000 3 Silla

ID 4 15,000 2 Silla

ID 5 5,000 2 Silla



ID 3 36,000 3 Silla



ID 1 25,000 3 Silla

ID 4 15,000 2 Silla

ID 5 5,000 2 Silla

Ganador de la Subasta Precios Producto

3 35,000 Silla

Experimental Results - Chile

Auction Type 1 2 3 4

SEQ Revenue 50 60 20 25

Bid 7.54 13.40 7.98 8.30

RTC Revenue 100 100 100 100

Bid 43.54 36.04 27.48 24.83

• Revenues and Bids (Pesos per cubic meter)• Bids in RTC greatly exceed SEQ

Tobit Estimates of bidding behavior

Variable Model 1 Model 2RTC 39.456*** 4.464Pref 2 -7.736 -7.656Good 3 -19.657** -19.425**Good 4 -31.478*** -30.983***RTC x Good 2 0.403 0.322RTC x Good 3 -8.315 -8.595RTC x Good 4 -6.039 -6.465Experience 19.030**Risk -2.414

0.349RTC x Risk 6.170**

0.041_cons 13.359 23.481**N 164 164Log likelihood 582.62 578.97

Ongoing opportunities for framed & natural field experiments

• Sports memorabilia shows, open air markets etc.• Benefits: – Subjects who select into the market place in specific

roles.– Variety of experience levels– Pay their own money

• Some useful protocols for valuation in this setting– nth price auction:

• number of goods sold is determined randomly• Provides low value bidders with incentives for truthful

revelation (List and Shogren, 1998)– BDM mechanism: one person at a time

Interesting Results from Sportscard Studies

• Many behavioral anomalies are mitigated when going from lab to field– Endowment effects eliminated among

superexperienced traders• List 2003, 2004

– Anchoring of values possible only for new market participants

– Alevy, Landry and List, 2007

– Framing effects on valuation mitigated but still strong

– Alevy, List, and Adamowicz, 2006

Gift Exchange in Sportscard Markets• Artefactual, framed, and natural experiments

combined– List 2006 JPE Behavioralist Meets the Market

• Examine whether social preferences influence outcomes in markets.

• Methodology:– 1. Artefactual field experiment• abstract framing, subjects are sports card buyers and

sellers (placed in their usual roles)• Gift exchange exists: Buyers pay high prices,

reciprocated by high quality.

Gift Exchange in Sportscard Markets

Methodology2. Framed Field Experiment– Replace abstract good with sportscards– Gift exchange result

3. Natural Field Experiment– Anonymously visit sellers who had participated in

experiment– Offers of high prices not reciprocated by high quality

• Exception: Local dealers• Explanation reputation not reciprocity

Characterizing the Entrepreneur Using Field Experiments (Elston et al. 2005)

Use entrepreneurs (full- and part-time) vs control groups to test four alleged characteristics of entrepreneurs:

• Entrepreneurs are risk loving?– Test based on Holt and Laury (2002)

• Entrepreneurs have a joy of winning and/or suffer from (optimistic) judgmental errors?– Winners curse in common value auctions

• Holt and Sherman (1994)

• Entrepreneurs are overconfident?– Entry game

• Camerer and Lovallo (1999)

RISK AVERSION – based on Holt and Laury (2002).Figure3 shows raw responses which suggest: • FT Entrepreneurs less risk averse than the PT Entrepreneurs or Others• PT Entrepreneurs may be more risk averse than the nonentrepreneurs,

particularly for the last three problems measuring extreme risk aversion.

….. But figure3 fails to condition on observed differences in samples. Therefore, use Interval Regression Model (results in table 2):

• Full-Time Entrepreneurs (FT) have lower aversion to risk than Part-Time Entrepreneurs (PT). • PT do not have significantly different risk attitudes than Non-Entrepreneurs (default category)

Judgmental Error and Joy of Winning

Common Value Auctions • Judgmental Error: The winners curse– Receive signal on value of good, value is common to all– Those receiving high value win but pay too much• Joy of winning– Another reason for overbidding• Protocol distinguishes between these reasons for

overbidding– But yields no strong results. – No evidence of judgmental errors for any subject pool

Overconfidence and Entry

Camerer and Lovallo (1999) - Protocol.• Subjects in groups of 5 decide whether to enter a market or

not.– Endowed with $10– Pay $10 to enter– The most skilled among entrants receives $35• Subjects that chose not to enter would be allowed to keep

the initial stake. • No subject knew their score on the skill test prior to entry,

so entry was determined in part by the subject’s belief about their skill level in relation to the others that might enter.

• Entry is also affected by risk attitudes• Beliefs about number entering are also elicited

Conclusions

• Experiments an important new tool in economics

• Replication and extension leads to accumulation

• Field experiments an important link – to traditional empirical methods

Replication and Progress through Experiments

• Attitudes to risk are a primitive in economics– Gollier 2001• “It is quite surprising and disappointing to me that

almost 40 years after the establishment of the concept … by Pratt and Arrow, our profession has not yet been able to attain a consensus about the measurement of risk aversion.”

– Since 2001 enormous progress thanks to Holt, Laury, Harrison and others.

Psychology & Economics: Thriving

• Important collaborations/investigations of ideas from psychology– Ideas & methods generated by psychologists• Some withstand economists’ scrutiny

– Preference reversals, framing effects

• Some do not– Methods: Lack of salience & deception– Salience seems to weaken importance some results related to

prospect theory.

Lab & Field Experiments• Large debate on role of lab & field experiments• Scrutiny in lab versus anonymity in field

– Particularly relevant for social preference models

• “The basic strategy underlying laboratory experiments in the physical sciences and economics is similar, but the fact that humans are the object of study in the latter raises special questions about the ability to extrapolate experimental findings beyond the lab, questions that do not arise in the physical sciences.” – Levitt & List, 2007

• Falk & Heckman, 2009 reply to Levitt & List– People are also under observation in field settings. – In laboratory settings the amount and type of scrutiny (observation by

experimenter ) can be a treatment variable.

Summing up: Checklist to run an experiment…

• What is the question of interest?– Why is it interesting (theory, policymaker, fact finding)?

• Stakes sufficient?– Flat-payoff?

• Preferences induced?– What homegrown values remain uncontrolled?

• Appropriate subject pool?• Theory cleanly tested (assumptions, etc.)?• Within vs. between group design?• Instructions appropriate?

– Test these out with colleagues• Need administrative approval?• Pilot?• Run!

Resources for experimentalists• John List’s www.fieldexperiments.com• Charles Holt’s http://people.virginia.edu/~cah2k/research.html • Georgia State University – EconPort http://www.econport.org/• Al Roth’s webpage

http://kuznets.fas.harvard.edu/~aroth/alroth.html • Economics Science Association (ESA) www.economicscience.org • Latin America Field Experiments Network (LAFEN)

http://lafen.uniandes.edu.co/ • University of Alaska Anchorage – Experimental Economics

Laboratory http://econlab.uaa.alaska.edu • Many more resources …. Feel free to email me for further

information and discussion at [email protected]

THANKS!

• Feel free to email me for further information and discussion at [email protected]

experimental economics: short course universidad del desarrollo santiago, chile december 17, 2009

Documents

field context

field setting

trust game costa rica

haigh list

list towe

water treatment help

experimentsfield experiments

different majors