using experimental methods to inform public policy debatesusing experimental methods to inform...
TRANSCRIPT
![Page 1: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/1.jpg)
Using Experimental Methods to Using Experimental Methods to Inform Public Policy DebatesInform Public Policy Debates
Jim MurphyPresentation to ISER
July 13, 2006
![Page 2: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/2.jpg)
Experiments are just one more tool for Experiments are just one more tool for the applied economistthe applied economist
Econometric analysis of non-experimental dataSurveys (quantitative and qualitative)Experimental methods
![Page 3: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/3.jpg)
What is experimental economics?What is experimental economics?
Pioneered by Vernon SmithObserve behavior of human subjects in a controlled settingSetting captures essential elements of an economic problemSubjects paid cash based on their performance
![Page 4: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/4.jpg)
What distinguishes experiments What distinguishes experiments from other empirical research?from other empirical research?
Control– Over rules of exchange (institution)– Over incentives (environment or parameters)– Over individual preferences (cash payments)
Know theoretically predicted outcome• Compare with observed behavior
Replication– Often difficult with field data
![Page 5: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/5.jpg)
Why experiments? (I)Why experiments? (I)Experiments and theoryExperiments and theory
Test a theory– Enforcement of permit markets– Public goods & bargaining experiments
Explore causes of a theory’s failure– Public goods & bargaining experiments
Establish empirical regularities– Existence of hypothetical bias in CV– Efficiency of double auctions
![Page 6: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/6.jpg)
Role of experimental methods in Role of experimental methods in the policy processthe policy process
Incentive-based management gaining popularity– Rights trading for emissions, fishing, water use…– Taxes, tax breaks and subsidies for land use and
agricultural practices (CRP)“If you build it, they will come…”– Economic theory says little about role of institutions– Experiments show that institutions are critical in
determining outcomesCheaply and quickly evaluate economic properties of proposed new programs– Identify problems before implementation
![Page 7: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/7.jpg)
Why experiments? (II)Why experiments? (II)Inform policy decisionsInform policy decisions
Compare institutions (“rules of exchange”)– EPA Acid Rain Program
Evaluate policy proposals– Proposed alternative ITQ market structures– Cotenancy on natural gas pipelines– Management of common-pool resources
Design & test new institutions– RECLAIM combinatorial auctions– Water market design– FCC spectrum auctions
![Page 8: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/8.jpg)
External validity (parallelism)External validity (parallelism)
Economic behavior is governed by basic principles which work in the lab as well as in naturally-occurring environments.– Depends on how well design captures incentives of
naturally-occurring environment– Cash payment aligns subjects’ incentives with those
of agents they representEventually established through repeated successful application to a range of problems
![Page 9: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/9.jpg)
Building an airplane (or new policy)Building an airplane (or new policy)
1. Engineers develop design
2. Test in windtunnel
3. Flight tests4. Use plane
1. Economic theory & econometrics
2. Experimental economics
3. Field pilots4. Implement policy
![Page 10: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/10.jpg)
Enforcement of tradable permitsEnforcement of tradable permits
Test a theory – with policy implications!!
![Page 11: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/11.jpg)
Why run experiments?Why run experiments?
To test a theory about enforcement & compliance in permit markets– Theory has some counterintuitive results– Important policy implications
Theory cannot be tested with existing data– Insufficient variation in enforcement strategies– Test theory in controlled lab setting
Results– Largely support theory– One unanticipated result not predicted by theory
![Page 12: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/12.jpg)
MotivationMotivation
Environmental regulations often involve fixed standards– Technology standards– Emissions standards
Market-based mechanisms can meet environmental goals at lower cost– Programs must be enforced
![Page 13: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/13.jpg)
Enforcing markets different than Enforcing markets different than standardsstandards
Regulators have experience enforcing emissions standards– e.g., Target enforcement based on firm characteristics
Enforcing market-based mechanisms fundamentally different– Permit price is key determinant of compliance
Policy implication: Enforcement strategies that were effective for standards might not be appropriate for markets
![Page 14: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/14.jpg)
Implications for enforcement of Implications for enforcement of permit marketspermit markets
• With markets, violations depend on:– Permit price and expected penalty
Violations are independent of firm-specific characteristics
no justification for targeted enforcement
![Page 15: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/15.jpg)
Direct and indirect price effectsDirect and indirect price effects
Compliance decisions are linked through permit marketEffects of increased enforcement on violations:– Negative direct effect– Countervailing positive price effect
• “Two steps forward, one step back…”Policy implication: Changes in enforcement can induce changes in permit prices—which affect compliance decisions.
Policy implication: Targeted enforcement can induce non-compliance by those who are not targeted.
![Page 16: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/16.jpg)
Common pool resource managementCommon pool resource management
Field experiments
![Page 17: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/17.jpg)
From the lab to the fieldFrom the lab to the field
Recent trend towards more field experiments– Test robustness of lab experiments– Cross-cultural studies– Less artificial environments
Lab and field experiments are complements with +’s and –’s
![Page 18: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/18.jpg)
Common pool resources (Common pool resources (CPRsCPRs))
Easy to access, hard to exclude– Can lead to resource over-extraction
• “Tragedy of the Commons”
– e.g., fisheries, forestsDilemma: individual interest vs. group interests– “A situation in which mutual cooperation is collectively rational
for a group as a whole, but individual cooperation may not be rational for each member” —Bardhan et al., 2000
![Page 19: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/19.jpg)
Why in the field??Why in the field??
Social dilemmas in CPR experiment parallels that in their daily livesExperience in CPR could affect choices in experiment– Familiarity with each other– Social norms about appropriate behavior– May get different results than with students– Observing how people actually affected by CPRs
may be more relevant to understanding resource exploitation in developing world
![Page 20: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/20.jpg)
The sitesThe sites
We conducted the same set of experiments in 3 regions of Colombia Subjects were direct users of natural resources– Majority were
fishermen.
![Page 21: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/21.jpg)
![Page 22: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/22.jpg)
![Page 23: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/23.jpg)
![Page 24: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/24.jpg)
BackgroundBackground
Government intervention could have unintended consequencesRegulations can shape not only behavior, but also preferences– Crowding out
Limits to self-governance
![Page 25: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/25.jpg)
Research questionsResearch questions
What motivates CPR users?– Tested several models of pure Nash strategies
when people motivated by combination of self-interest and alternative motives
– Pure self-interest– Altruism– Reciprocity– Conformity
– Choices consistent with conformity• Leads to more conservative use of resource
![Page 26: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/26.jpg)
Research questionsResearch questions
What are the effects of alternative institutions intended to promote more conservative choices in CPRs? How do these vary among regions?– Possible complementarities between external
(formal) and internal (informal) regulations– Some evidence in support of
complementarities, but not consistent across regions
![Page 27: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/27.jpg)
Hypothetical bias in contingent Hypothetical bias in contingent valuationvaluation
Establish empirical regularitiesMechanism design / calibrationDevelop theory
![Page 28: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/28.jpg)
Contingent valuationContingent valuation
CV surveys elicit values for non-market goods– WTP to preserve ANWR– Value of damages from Exxon Valdez
Hypothetical nature of CV– Payment for the good– Provision of the good
How do you know your CV estimate is accurate?How do you know your CV estimate is accurate?
![Page 29: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/29.jpg)
Basic elements of CV experimentsBasic elements of CV experiments
Subject pool– Students / non-students
Good to be valued– Public vs. private good
Elicitation mechanism
Treatments– Hypothetical
• “Would you pay $X?”
– Actual / Real• “Will you pay $X?”
Key assumptionReal cash payments are unbiased
Key assumptionReal cash payments are unbiased
Hypothetical biasHyp. payments exceed real cash payments
Hypothetical biasHyp. payments exceed real cash payments
![Page 30: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/30.jpg)
Why run experiments? (I)Why run experiments? (I)
Establish empirical regularities– 1980’s to early 90’s– Does hyp. bias exist?– Murphy et al. meta-analysis
• 28 studies, 83 observations• 71% observations CF>1.1• Mean CF = 2.60• Median CF = 1.35
0%
5%
10%
15%
20%
25%
30%
<0.9
0.9
- 1.0
9
1.1
- 1.4
9
1.5
- 1.9
9
2 - 2
.99
3 - 3
.99
4 - 4
.99
5 - 9
.99
10+
Calibration Factor
Perc
ent o
f Obs
erva
tions
(n=8
3)
Hypothetical ValueCF =Actual Value
![Page 31: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/31.jpg)
Why run experiments? (II)Why run experiments? (II)
NOAA panel (early 90’s)– “unfortunate” lack of data to calibrate responses
Develop and test new calibration techniques– Late 90’s to now– Ex ante or instrument calibration
• Try to get unbiased responses– Budget reminders– Cheap talk script
– Ex post or statistical calibration• Control for biased responses
– Statistical bias functions– Respondent uncertainty
![Page 32: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/32.jpg)
Calibration techniques generally Calibration techniques generally effective, but…effective, but…
No theory to explain why they work– Why do people systematically behave
differently in hypothetical & real settings?May not be able to generalize results– Example: Cheap talk
• Assumes that making people aware of bias will eliminate it
• Sensitive to script length, amount asked, subject experience
![Page 33: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/33.jpg)
Why run experiments? (III)Why run experiments? (III)
Test theory & explain anomalies– Ultimatum game modal split roughly equal
• Subsequent experiments sought to explain why– What causes hypothetical bias?
• Surprisingly little experimental work seeks to address this
• Explanations have been offered, but not systematically tested
![Page 34: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/34.jpg)
Some other projectsSome other projects
Mechanism design– California and Peruvian water markets– Australian non-point source trading
Voluntary disclosure (EPA Audit Policy)Voluntary agreementsInternational environmental agreementsEffects of social emotions on behavior in a CPR
![Page 35: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/35.jpg)
ConclusionConclusion: When should I think : When should I think about experiments?about experiments?
When naturally occurring data don’t exist– New institutions, regulations, policies
To complement field data or surveys– All types of empirical studies have their strengths
and weaknessesTo identify how human behavior might affect policy outcomes– Social dilemmas / social norms– Behavioral economics
![Page 36: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/36.jpg)
Other Ongoing UAA Faculty Other Ongoing UAA Faculty Research ExperimentsResearch Experiments
Sharman Haley– “Fairness, Information and Coalition Building in Tax
Policy: An Experimental Approach”Lee Huskey and Wayne Edwards– “Job Search with an External Opportunity”
Gunnar Knapp– “Rent Dissipation in Limited Entry Fisheries with
Aggregate Quotas: An Experimental Analysis”Lance Howe– “Common Pool Resource Management and the
Effect of Heterogeneous Users: an Experimental Investigation”
![Page 37: Using Experimental Methods to Inform Public Policy DebatesUsing Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 ... Role of experimental](https://reader034.vdocuments.site/reader034/viewer/2022042314/5f02fca87e708231d406fb52/html5/thumbnails/37.jpg)
Average Investment in the CommonAverage Investment in the Common--Pool Resource Pool Resource Across Treatments, High and Low Endowment SubjectsAcross Treatments, High and Low Endowment Subjects
Open Access Nash Pareto Optimal