ESA Mentoring Session: Publishing Experimental Economics Papers

Panelists: Daniel Houser, Yan Chen, Tim Cason, Marie Claire Villeval

2015 International ESA at Sydney

Publishing Experimental Economics Papers

Daniel HouserGeorge Mason University

Idea Generation

• A good paper requires a good idea• Read broadly outside of economics, and

narrowly within economics• Start and participate in reading groups• Have many informal discussions with

colleagues• Solicit and respond to advice about your ideas

Project Execution

• A good paper must report data from a well-executed design• Before executing, present specific project plan in a

workshop/lab group meeting• Attend to details: seemingly small design flaws can leave

papers unpublishable• The same hypothesis can be tested many ways; strive to

find simple ways (ideally without deception) to answer your questions

• Usually one wants to avoid experiment designs that require sophisticated econometric analyses to test the hypotheses – there is a simpler way

Paper Writing

• A good paper is only as long as it needs to be• Papers must be written in native English – there

is no shame in asking (or paying) for a paper to be edited by a native speaker

• Hypotheses must be compellingly motivated and, ideally, tested using simple procedures

• Conclusions must be supported by the data; a strong prior with inconclusive data is not generally sufficient (or why run the experiment?)

Journals to Mention

• Journal of Neuroscience, Psychology and Economics

• Journal of Behavioral and Experimental Economics

• Management Science• PLoS ONE• Frontiers in Neuroscience

Publishing Experimental Economics Papers

Yan ChenUniversity of Michigan

Outline of Comments

• Idea selection• Before submission

– Circulating your paper– Giving seminars (internal and external)

• Points regarding some specific journals I am associated with

Idea Selection

• You probably have many good ideas – Every project has its opportunity cost

• Which ideas should you pursue?– Big ideas

• Impact on science and society

– “If everything turns out as expected, will it have the potential to hit the AER or Econometrica?” – E. Fehr

– Find the answer: might need several experiments to get there

– Balance your portfolio

Before submission:Circulating your paper

• After a paper is polished (i.e., copy edited), you should circulate it among experts (potential referees)

• “Send a copy to everyone on your reference list who is still alive.” – B. Allen, 1998

• Reduce the expert’s search cost: “Your paper is cited on page 17”– Error correction– Recognition and awareness– (Occasionally) really good comments

• Acknowledge and incorporate comments

Before submission:Giving seminars

• A good seminar serves the function of a round of referee reports

• Go through the papers published in the top five journals: many seminars acknowledged

• How to get invited?– Invite yourself, e.g., when you circulate your paper– Reduce your host’s cost (combining with conferences)– Organize seminars at your department

• Acknowledge and incorporate comments

Journals to Mention

• Management Science– DE -> AE (anonymous) -> referees -> AE -> DE– When you suggest AEs and referees

• Don’t waste slots• Suggest relevant referees, e.g., on your reference list• Suggested referees should not be too senior

• Games and Economic Behavior– You can’t suggest AEs but might influence who handles

your paper (through citation)• Other journals: should you suggest referees? Yes

Publishing Experimental Economics Papers

Tim CasonPurdue University

Outline of Comments

• Where to submit?• What to submit?• How to resubmit?• (dealing with rejection)• How to referee? (and its importance)• Points regarding some specific journals I am

associated with

Where to Submit?

• Research the journals you are considering, including the editors making decisions– What have the journals published lately?– Get advice from senior (and other) colleagues– Use the editor’s bias to your advantage

• Be ambitious with submissions, but realistic – Failure/rejection is part of the process– Finite set of qualified/appropriate reviewers

• Carefully consider the “fit” of your paper for specific journals (more on that later)

What to Submit?

• Length. Is shorter always better? Usually– Leave something out for the referees to ask for!

• Include appropriate appendices to ensure a thorough review (see old Econometrica guidelines)– such as exact experiment instructions

• and translate them correctly!

• Supplemental materials– Be judicious; don’t show everything you tried in the analysis

that wasn’t interesting enough to include in the paper– Use the supplement to provide details to support concise

statements in the body

How to Resubmit?

• Drop everything so that you can do a thorough revision quickly (especially when your tenure clock is ticking!)

• Response letters should be detailed, explaining how each point is addressed– (you don’t have to do everything the reviewers

request, but you must explain why you disagree!)• Paraphrasing rather than verbatim quotes of

the referees is fine in some circumstances

(Dealing with Rejection)

• Get used to it.• Rejections can be good. Sometimes they

provide the best feedback on your paper• If a referee did not “get” an important aspect

of your paper, it is your fault not the referee’s fault. (Most referees do a careful a job, but everyone has competing demands on their time)

• Best bonus: You can ignore the stupid requests

How to Referee?

• It is important: Refereeing provides another way for you to influence research in your field

• Follow the golden rule and provide timely reports. Karma

• Don’t request more data unless they are needed to address the authors’ own research questions (not your own questions)

• Don’t put your recommendation explicitly in your report (give the editor flexibility)

Journal of Public Economics, Games and Economic Behavior, Experimental Economics

• For JPubE, is the paper of interest to the broader readership of a leading field journal in Public Economics?

• For Experimental Econ, does it have methodological or some other focus that makes it a particularly good “fit” with a journal intended to influence experimental economists specifically?

• For Games: “GEB’s objective is to publish broadly applicable papers that advance the frontiers of game theory and its applications… Broad applicability means that a paper should lead to useful directions in the development of a general theory and its applications.”

Publishing Experimental Economics Papers

Marie Claire VillevalGATE-CNRS, University of Lyon

Three groups of people to convince

- The readers- Put yourself in the shoes of the reader!- Would you like to read this paper in full?- Would you like to cite this paper?

- The editor and co-editors- Evaluate whether the main idea of the paper is interesting enough for a

large audience and whether the paper will be cited

- The associate editor and the reviewers- Usually more specialized- Evaluate if there a sufficient contribution to the previous literature- Check whether the model/design/analysis are valid- Advice on how to improve the presentation of the selling point or the

data analysis

How to convince them

- Make sure that the novelty of your paper is made salient:- New field or new bridge between two fields- New research question- New theoretical model- New experimental task- New dataset- Don’t plagiarize yourself!

- Show clearly and asap your contribution: what is the Big Picture?- Prefer an explicit statement about this contribution- Your main findings should be shown as soon as possible- The literature review should be concise and cite papers published in very

good journals and generating a lot of citations- Each paragraph of the review should be concluded by a sentence

explaining how you contribute to this literature

How to convince them

- Show that your paper is useful in the literature- The paper can be easily found, read and cited (prefer simple titles, spend

time polishing the abstract and the introduction)- Indicate economic and policy implications, if any

Ready to submit?

- Use supplementary materials and on-line appendices for non central but useful analyses

- Do not target only journals specialized in experimental economics! Try to reach a broader audience, even outside economics, and try to anticipate which AE will handle your paper

- A good paper is a well-written paper (write short sentences, repeat words) … but the exposition differs across journals (and disciplines): adjust the writing to the journal (and beware the cost of writing a paper for Nature and Science!)

- For allowing replicability the paper must contain all necessary information

- Don’t submit too early to avoid wasted opportunities But post your working papers as soon as possible to show that you are working on this specific idea

After submission

- Don’t be shocked by a desk rejection! You save time. Try to better target the journals

- Don’t be desperate if you have a good idea, a good model, a good design but “no results”! More difficult to publish, but possible

- Don’t be angry too long against the reviewers who rejected your paper and revise your paper before submitting it again! Referee reports are very rarely useless… and you may get the same referee in the next submission

- If your paper has been invited for a R&R, always explain how you addressed each reviewer’s point to avoid multiple rounds of revisions

• JEBO: general interest journal (more than 1000 submissions/year, 45% desk-rejections, 100 days to first decision)

• Experimental Economics: papers usually include a methodological contribution

• JESA: promotes research on methods which address important economic questions that are difficult to examine using naturally occurring data; short papers (5000 words), replication studies, meta-analyses, …

• JPET: papers using experiments to test an original or already established theory in public economics

• ROBE: encourages a transdisciplinary and pluralistic perspective, embedding behavioral economics in a broader behavioral science

Journals to Mention