QMSS: THEORY AND METHODOLOGY FOR THE SOCIAL SCIENCES Columbia University

QMSS 5010 Syllabus Semet.docx GR5010, Fall 2018 Time: 6:10-8PM, Thursday Place:414 Pupin Instructor: Amy Semet Email: aes2180@columbia.edu; 215-767-0041 Office Hours: Thurs 8-9PM, and by appointment; I can often meet before class most weeks TA: Qiao Wen Email: qw2155@tc.columbia.edu Overview This course is designed as an in-depth introduction to the social sciences and its methodologies. It is intended to give a broad overview so students can intelligently combine ideas in solving real-world problems. We will focus on the logic and design of social research, beginning with some concepts and topics common to research across the social sciences. We will later move on to understanding the principles behind an array of methodologies used in the social sciences: causal inference, experimentation, observational studies, formal models, surveys, and applied machine-learning techniques. We will analyze their applications using cases drawn from the research literature. The focus of this course is not on the techniques themselves — you will have ample opportunity to do that in other courses — but in understanding the logic behind the use of these tools to extract meaningful answers from their applications. Prerequisites: it is assumed that you have had at least one semester of graduate-level statistics involving linear regression and analysis of variance. Some basic mathematics and algebra will also be assumed. Course Materials Three texts are intended to be resources for the crafting of your thesis proposal. Each week I give recommended chapters for review: Royce Singleton, Jr. and Bruce C. Straits. Approaches to Social Research. Oxford University Press, New York, 2010.

(used copies available for reduced cost) Wayne C. Booth, Gregory G. Colomb, and Joseph M. Williams. The Craft of Research. University of Chicago Press,

Chicago, IL, third edition, 2008. Anthony Weston. A Rulebook for Arguments. Hackett Publishing Co, Indianapolis, IN, fourth edition, 2009. For further background, I would also recommend Anthony Salganik, M. J. Bit by Bit Book: Social Research in the Digital Age, 2018. Required Readings are mandatory and should be completed before each class. Article Readings will serve as the foundation to class discussions; they will be available on Canvas. Thesis Readings are intended to help you with the drafting and crafting of the Research Proposal, but will not be discussed in class. Course Dynamics The class will be a combination of lectures and focused discussion. We will devote the first half of the class to a lecture on the topic assigned for the week, and during the second half of the class a panel of students will give a 5-minute presentation of one of the Article readings, followed by a class discussion to analyze in detail these applications of the methods we are studying. I will send a sign-up sheet for you to

pick the weeks you will be “on panel.” When you are on panel, you will distribute a one page summary of the readings the day before class by 6PM. Each panel presentation should not take more than 5 minutes, and should address the following questions: What is the research question? What is the specific hypothesis under investigation? Can it be falsified? Why was a particular method chosen? What inferences can be draw from its results? What are its limitations? Over the course of the semester, you will be required to complete two (2) Assignments where you will apply concepts and methods that you have learned in class. Assignments should be submitted on Canvas by 6PM on the indicated dates. At the end of the course, you will turn in a 12-15 page Thesis Proposal. To ensure that you produce a fully fledged product, you will be required to hand in Thesis Proposal Deliverables at periodic intervals. These will constitute the foundational pieces for your proposal. Feedback will be provided on these deliverables. Deliverables should be should be submitted on Canvas by 6PM on the indicated dates. Course Requirements Attendance is expected and reading assignments are to be completed before each session. All written work must be original and produced exclusively for this class. You are expected to follow the University’s guidelines for the submission of written work and there will be penalties for turning in assignments late. The final grade of the course will be based of your fulfillment of each of the following requirements: Assignments (20%): Students must complete a series of assignments where methods from the course are applied. Assignments are due on October 4 and October 25. Class participation (10%): Students are expected to have read all the Required Reading before class and actively participate in class discussion. Panel presentations (20%): Students will regularly be assigned to present one of the readings. Students will also present a one page analysis of the readings and the grade for this part will be determined by both the quality of the analysis as well as the quality of the oral presentation. Thesis Proposal Deliverables (20%): Students will submit pieces of their proposal at regular intervals. Please submit each one of them by 6PM on the indicated dates on Canvas. Thesis proposal (30%): Throughout the curse, you will work on a research proposal. This 12-15 page paper will be turned in at the end of the semester. ASSIGNMENT SCHEDULE Sept. 20: Thesis Deliverable #1: Description of Topic Oct. 4: Assignment #1 and Thesis Deliverable #2: Hypothesis Oct. 18: Thesis Deliverable #3: Description of Data Oct. 25: Assignment #2 Nov. 1: Thesis Deliverable #4: Literature Review Nov. 15: Thesis Deliverable #5: Research Strategy Nov. 29: Thesis Deliverable #6: Draft Dec. 10: Final Paper Due Course Outline WEEK 1, Sept. 6: INTRODUCTION TO THE COURSE WEEK 2, Sept. 13: THE “SCIENCE” OF SOCIAL SCIENCE AND THE ROLE OF STATISTICAL MODELS; ROLE OF ETHICS

Required Reading: Singleton and Straits, Chapters 1-4 John Elster. Explaining Social Behavior: More Nuts and Bolts for the Social Sciences. Cambridge University Press,

Cambridge, MA, 2007, Ch. 1-2. Lewis A. Coser. 1975. “Two Methods in Search of a Substance.” American Sociological Review 40: 691-700. Thesis Reading: The Craft of Research, Chapter 3 A Rulebook for Arguments, Chapters I-II WEEK 3, Sept. 20: CAUSALITY AND CAUSAL INFERENCE Sept. 20: Thesis Proposal Deliverable #1 due: Description of Topic Required Reading: Paul W. Holland. 1980. “Statistics and Causal Inference.” Journal of the American Statistical Association,

81(396):945–960. Donald B. Rubin. 2005. “Causal inference using potential outcomes: Design, modelling, decisions.” Journal of the

American Statistical Association, 100(469):322–331. Judea Pearl. 1995. Causal Diagrams for Empirical Research. Biometrika82(4): 779-710. Jason Lydall. 2010. Are Co-Ethnics More Effective Counter-Insurgents? Evidence from the Second Chechen

War. American Political Science Review. 104:1 (February 2010): 1-20. Thesis Reading: The Craft of Research, Chapters 4 A Rulebook for Arguments, Chapters VI-VII WEEK 4, Sept. 27: CONCEPTS AND MEASUREMENT Required Reading: Singleton and Straits, Chapter 5 Barbara Geddes. 1990. “How the cases you choose affect the answers you get: Selection bias in comparative

politics.” Political Analysis, 2(1):131–152. Jerry Hausman. 2001. “Mismeasured variables in econometric analysis: Problems from the right and problems from

the left.” Journal of Economic Perspectives, 15(1):57–67. Pamela Paxton. 1999. “Is social capital declining in the United States? A multiple indicator assessment.” American

Journal of Sociology105: 88-127. Thesis Reading: The Craft of Research, Chapters 7-8 Rulebook for Arguments, Chapter VIII WEEK 5, Oct. 4: EXPERIMENTS AND RANDOMIZATION PART 1 October 4: Assignment #1 due and Thesis Deliverable #2: Hypothesis Required Reading: Singleton and Straits, Chapters 7 and 8 Donald Green and Alan Gerber. 2003. “The under provision of experiments in political science.” Annals of the

American Academy of Political and Social Science, 589:94–112. Chattopadhyay, Raghabendra and Esther Duflo. 2004. “Women as Policy Makers: Evidence from a RandomizedPolicy

Experiment in India.”Econometrica, 72(5): 1409-1443.

Thesis Reading: The Craft of Research, Chapters 9-10 WEEK 6, Oct. 11: EXPERIMENTATION AND RANDOMIZATION PART 2 Required Reading: Devah Pager, Bruce Western and Bart Bonikowski. 2009. “Discrimination in a Low-Wage Labor Market: A Field

Experiment” American Sociological Review, 74:777–799. Macaratan Humphreys and Jeremy Weinstein. 2009. “Field Experiments and the Political Economy of

Development.”Annual Review of Political Science 12: 367-378. Alan Gerber, D. Green and C. Larimer. 2008. “Social Pressure and Vote Turnout: Evidence from a Large-Scale Field

Experiment.” American Political Science Review 102(1): 33-48. Thesis Reading: The Craft of Research, Chapters 11 WEEK 7, Oct. 18: OBSERVATIONAL STUDIES AND QUASI-EXPERIMENTATION Oct. 18: Thesis Proposal Deliverable #3: Data Description Required Reading: Singleton and Straits, Chapter 11 Susan Hyde. 2007. “The Observer Effect in International Politics: Evidence from a Natural

Experiment.” World Politics 60(1): 37-63. Robert Erikson and Roc´ıo Titiunik. 2015. “Using regression discontinuity to uncover the personal incumbency

advantage.” Quarterly Journal of Political Science, 10:101–119. John Lott and David B. Mustard. 1997. “Crime, deterrence, and right-to-carry concealed handguns.” The Journal of

Legal Studies26: 1-68. Thesis Reading: The Craft of Research, Chapters 12-13 WEEK 8, Oct. 25: SAMPLING Oct. 25: Assignment #2 Due Required Reading: Singleton and Straits, Chapters 6 Scott Fricker and Roger Tourangeau. 2010. “Examining the relationship between nonresponse propensity and data

quality in two national household surveys.” Public Opinion Quarterly, 74(4):934–955. David S. Yaeger, Jon A. Krosnick, Linchiat Chang, Harold S. Javitz, Matthew S. Levendusky, Alberto Simpser, and Rui

Wang. 2011. “Comparing the accuracy of rdd telephone surveys and internet surveys conducted with probability and non-probability samples.” Public Opinion Quarterly, 75(4):709–747

Marie Hojnacki and David C. Kimball. 1998. “Organized interests and the decision of whom to lobby in Congress.” American Political Science Review92(4): 775-90.

Thesis Reading: The Craft of Research, Chapters 14 WEEK 9, Nov. 1: SURVEY RESEARCH Nov. 1: Thesis Deliverable #4: Literature Review

Required Reading: Singleton and Straits, Chapters 9 and 10 Sebastian, Mikael Gilljam, and Stefan Dahlberg. 2016. “Measuring generalized trust. an examination of question

wording and the number of scale points.” Public Opinion Quarterly, 80(1):26–43. Nora Cate Schaeffer and Stanley Presser. 2003. “The Science of Asking Questions.” Annual Review of Sociology 29:

65-88. Courtney Kennedy, Mark Blumenthal, Scott Clement, Joshua D. Clinton, Claire Durand, Charles Franklin, Kiley

McGeeney, Lee Miringoff, Kristen Olson, Doug Rivers, Lydia Saad, Evans Witt, and Chris Wlezien. 2017. “An evaluation of 2016 election polls in the United States.” URL http://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx. AAPOR

Thesis Reading: The Craft of Research, Chapters 15

Week 10, Nov. 8: RATIONAL CHOICE Required Reading: William H. Riker. 1995. “The political psychology of rational choice theory.” Political Psychology,1(16):23–44. Christopher Achen. 2002. “Parental socialization and rational party identification.” Political Behavior, 24(4):151–170. Keith Krehbiel. Pivotal Politics: A Theory of U.S. Lawmaking. University of Chicago Press, Chicago, IL, 1998, Ch 2 Thesis Reading: The Craft of Research, Chapters 16 Week 11, Nov. 15: IDENTIFICATION Nov. 15: Thesis Deliverable # 5: Research Strategy Required Reading: Singleton and Straits, Chapter 12 Joshua D. Angrist and Pischkem J¨orn-Steffen. Mastering ‘Metrics’: The Path from Cause to Effect.Princeton

University Press, Princeton, NJ, 2015, Ch 3. Edward Miguel, Shanker Satyanath, and Ernest Sergenti. 2004. “Economic shocks and civil conflict: An instrumental

variables approach.” Journal of Political Economy, 112(4): 725–753. Daron Acemoglu, Simon Johnson, and James A. Robinson. 2001. “The colonial origins of comparative development:

An empirical investigation.” The American Economic Review, 91(5):1369–1401. Thesis Reading: The Craft of Research, Chapters 17 ***THANKSGIVING RECESS*** Week 13, Nov. 30: MULTIPLE METHODS AND PAPER DISCUSSSION Nov. 29: Thesis Deliverable #6: Draft of Research Paper Due Required Reading: Singleton and Straits, Chapter 13 Week 14, Dec. 6: TEXT AS DATA Required Reading:

Justin Grimmer and Brandon Stewart. 2013. “Text as data: The promise and pitfalls of automatic content analysis methods for political documents.” Political Analysis,21 (3):267–297.

Michael Laver, Kenneth Benoit, and John Garry. 2003. “Extracting policy positions from political texts using words as data.” American Political Science Review,97(2):311–331.

Pablo Barber. 2015. “Birds of the same feather tweet together. bayesian ideal point estimation using twitter data.” Political Analysis,23(1):76–91.

Monday Dec. 10: FINAL PAPERS DUE: Papers MUST be submitted by 6PM On Monday December 10. Statement on Academic Integrity Columbia’s intellectual community relies on academic integrity and responsibility as the cornerstone of its work. Graduate students are expected to exhibit the highest level of personal and academic honesty as they engage in scholarly discourse and research. In practical terms, you must be responsible for the full and accurate attribution of the ideas of others in all of your research papers and projects; you must be honest when taking your examinations; you must always submit your own work and not that of another student, scholar, or internet source. Graduate students are responsible for knowing and correctly utilizing referencing and bibliographical guidelines. When in doubt, consult your professor. Citation and plagiarism-prevention resources can be found at the GSAS page on Academic Integrity and Responsible Conduct of Research. Failure to observe these rules of conduct will have serious academic consequences, up to and including dismissal from the university. If a faculty member suspects a breach of academic honesty, appropriate investigative and disciplinary action will be taken following Dean’s Discipline procedures. Statement on Disability Accommodations If you have been certified by Disability Services (DS) to receive accommodations, please either bring your accommodation letter from DS to your professor’s office hours to confirm your accommodation needs, or ask your liaison in GSAS to consult with your professor. If you believe that you may have a disability that requires accommodation, please contact Disability Services at 212-854-2388 or disability@columbia.edu. Important: To request and receive an accommodation you must be certified by DS.

Professor Dawn Brancati Email: dmb73@caa.columbia.edu. Prof. Office Hours: 6-7 pm (signup), RIAB509 TA: Renee Li (yoli20@gsb.columbia.edu) TA Office Hours: 12:45-1:45 pm Thursday, IAB 270J

GR4010: Theories and Methods of the Social Sciences This course provides an introduction to research design and quantitative research methods in the social sciences. The goals of the course are: (1) to instruct students in how to critically analyze scholarly articles in terms of research design and (2) to guide students in how to design an original research project. Technical issues related to the implementation of the methods are not the focus of this course. Broader issues regarding the advantages and disadvantages of alternative designs, methods, and techniques are instead.

Reading Assignments:

The readings for the course consist of chapters from the textbook below and a selection of academic articles that address more advanced and specific topics. These readings are available through the library course reserves.

Required:

Dawn Brancati. 2018. Social Scientific Research (London: Sage Publications).

Written Assignments:

The course includes 5 assignments: The main assignment for the course is a project proposal (20-25pp.) that is due at the end of the term. The 4 other assignments (i.e., literature review, research question/puzzle statement, methodology statement, and IRB certification) are designed to aid in the development of this proposal. Detailed instructions regarding these assignments will be provided during the term. All students must submit their assignments on time. Late assignments will be graded down 1/3 of a point for every day late. Late assignments will not be accepted after 7 days. Extensions may be granted but only in the case of serious unforeseen illness and/or family emergencies.

Attendance:

Students are expected to come to each class having read the assignments for that day ahead of time and to participate in the class discussions. Powerpoint slides will not be made available to students. Students may miss 1-2 classes without these absences being reflected in their course grade. Unless you miss more than 2 classes, it is not necessary to email the professor or the teaching assistant to inform us of your absence or the reason for your absence.

Office Hours: To sign up for the professor’s office hours, go to: www.SignUpGenius.com/go/5080F4EAAA62FA3F85-office

Academic Integrity:

Students are expected to do their own work on all assignments for this course and act in accordance with the Faculty Statement on Academic Integrity. Students who plagiarize, use unauthorized materials, or commit any other act of academic dishonesty will reported to the Office of Judicial Affairs and appropriate correctives will be taken.

Disability Accommodations:

To receive a disability accommodation, you must be certified by the Disability Services Office (212-854-2388 or disability@columbia.edu). If you have been certified, please share the accommodation letter with me and your teaching assistant directly or ask your liaison to contact me.

Grades:

Literature Review: 10% (due: 10/02/18)

IRB Certification (P/F): 2% (due: 10/09/2018)

Research Question/Puzzle Statement: 10% (due: 10/16/2018)

Methodology Statement: 10% (due: 11/20/2018)

Proposal (20-25pp): 58% (due: 12/14/2018)

Participation: 10%

I. Introduction (Tues., 09/05/18)

Topics/Questions: What is the scientific method? How scientific are the social sciences?

Brancati, Social Scientific Research, Chapter 1.

Pushkala Prasad. 2005. Crafting Qualitative Research: Working in the Postpositivist Traditions. Abingdon: Routledg, pp.13-17.

II. Research Topic/Question Selection (Tues., 09/11/18) Topics/Questions: What makes a good research topic? What is the difference between a research question and a research puzzle? What strategies are there for identifying a good topic and/or question and puzzle? What are the differences and trade-offs between problem-driven versus method or data-driven approaches?

Brancati, Social Scientific Research, Chapters 3 and 4.

**Note: We will discuss the research question assignment at this class meeting.**

III. Conceptualization (Tues., 09/18/18) Topics/Questions: What is the difference between a concept and a measure? What are the criteria by which concepts are evaluated for quality? What is concept stretching?

Brancati, Social Scientific Research, Chapters 5.

**Note: We will discuss the literature review assignment at this class meeting.**

IV. Causal Arguments (Tues., 09/25/18) Topics/Questions: What is causality? Can we demonstrate it in social scientific research and how? What do the following terms related to causal inference mean: tautology, functionalism, spuriousness, the fundamental problem of causal inference, and endogeneity?

Brancati, Social Scientific Research, Chapter 6.

Paul W. Holland. 1980. Statistics and Causal Inference. Journal of the American Statistical Association, 81(396):945-960.

*** Literature Review Assignment due by 4:00 pm, 10/02/18 ***

V. Method Selection (Tues., 10/02/18) Topics/Questions: How does the method we use shape the answers that we get? Is there a best method? What are the ethical concerns associated with different research methods? What kind of research is permissible?

Brancati, Social Scientific Research, Chapters 2 and 7.

**Note: We will discuss at this course meeting the methodology statement assignment and the IRB certification assignment.**

VI. Laboratory Experiments (Tues., 10/09/18) Topics/Questions: Why types of questions can be analyzed experimentally? How are lab experiments, in particular, used in the social sciences What are the (dis)advantages of experimental versus non-experimental methods? What are their limitations? What are the (dis)advantages of lab experiments, in particular, over non-experimental methods and other types of experimental methods? How do lab-in-the-field experiments differ from lab experiments?

Brancati, Social Scientific Research, Chapter 19.

Uri Gneezy and Alex Imas. 2017. “Lab in the Field: Measuring Preferences in the Wild,” In Handbook of Field Experiments. Amsterdam, edited by Abhijiit Vinjak Baerjee and Esther Duflo, pp. 439-462. Amsterdam: Elsevier BV.

Jacob M. Montgomery, Brendan Nyan, and Michelle Torres.” How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It.” American Journal of Political Science, 62(3):760-775.

**CITI Certification due by 4:00 pm on 10/09/2018 ** https://research.columbia.edu/content/human-subjects-protection-training-program

VII. Field Experiments (Tues., 10/16/18)

Topics/Questions: How do field experiments, including survey experiments, differ from lab and lab-in-the-field experiments? What are their advantages and disadvantages compared to lab experiments? How are field experiments and survey experiments used in the social sciences?

Rachel Glennerster. 2017. “The Practicalities of Running Randomized Evaluations: Partnerships, Measurement, Ethics, and Transparency.” In Handbook of Field Experiments, edited by Abhijiit Vinjak Baerjee and Esther Duflo, pp. 175-239. Amsterdam: Elsevier BV.

Elizabeth Levy Paluck and Eldar Shafir. 2017. “The Psychology of Construal in the Design of Field Experiments.” In Handbook of Field Experiments. Amsterdam, edited by Abhijiit Vinjak Baerjee and Esther Duflo, pp. 245-268. Amsterdam: Elsevier BV.

Recommended:

Gerber, Alan S. and Donald P. Green. 2012. Field Experiments: Design, Analysis, and Interpretation. New York: W.W. Norton.

*** Research Question/Puzzle Statement due by 4:00 pm on 10/16/2018 ***

XIII. Surveys, Part I (Tues., 10/23/18) Topics/Questions: How does the type of information that you collect from surveys differ from that collected from other methods, especially interviews and observational data? What are the key issues to be aware of in survey administration and design in order to maximize the advantages and minimize the shortcomings of surveys vis-à-vis other methods?

Brancati, Social Scientific Research, Chapter 18.

Jaak Billiet and Hideko Matsuo. 2017. “Non-response and Measurement Error.” In Handbook of Survey Methodology for the Social Sciences, edited by Lir Gideon, pp. 149-178. New York: Springer.

Roger Tourangeau, Lance J. Rips, and Kenneth Rasinski. 2000. The Psychology of Survey Response, edited by, pp. 289-312. New York: Cambridge University Press, Chapter 10.

Recommended: Fowler, Floyd J. 1995. Improving Survey Questions: Design and Evaluation. Thousand Oaks: Sage Publications.

IX. Surveys, Part II (Tues., 10/30/18)

Topics/Questions: What are the best strategies related to the administration of surveys and question design for soliciting sensitive information from respondents? What are the strengths and weaknesses of these strategies and how can they be combined, if at all, to offset their weaknesses?

Graeme Blair, Kosuke Imai, and Jason Lyall. 2014. “Comparing and Combining List and Endorsement Experiments: Evidence from Afghanistan.” American Journal of Political Science 58(4): 1043-1063.

Graeme Blair, Kosuke Imai, and Yang-Yang Zhou. 2015, “Design and Analysis of the Randomized Response Technique.” Journal of the American Statistical Association 110(511): 1304–1319.

Will Bullock, Kosuke Imai, and Jacob N. Shapiro. “Statistical Analysis of Endorsement Experiments: Measuring Support for Militant Groups in Pakistan.” Political Analysis 19(4): 363-384.

**Election Day: Tuesday, November 6, 2018. No Classes.**

X. Observational Studies (Tues., 11/13/18) Topics/Questions: What are the advantages of observational studies and when are they appropriate? To what does the term generalizability refer? What are the shortcomings of observational studies and how can they be overcome? What are natural experiments and when are the treatments really exogenous?

Brancati, Social Scientific Research, Chapter 20.

Shenyang Guo and Mark W. Fraser. 2014. Propensity Score Analysis. London: Sage Publications, Chapter 4.

Elizabeth A. Stuart. 2010. “Matching Methods for Causal Inference: A Review and a Look Forward.” Statistical Science 25(1): 1-21.

Joshua D. Angrist, Guido W. Imbens and Donald B. Rubin. 1996. “Identification of Causal Effects Using Instrumental Variables.” Journal of the American Statistical Association 91(434): 444-455.

XI. Case Selection (Tues., 11/20/18) Topics/Questions: What are the best principles to follow when choosing which case studies to study? What are the different types of sampling methods available? What does the term selection bias mean and what problems arise from it? Readings:

Brancati, Social Scientific Research, Chapter 9.

A.K. Sharma. The Textbook of Sampling and Attributes, pp. 1-25. New Delhi: Discovering Publishing House.

Recommended: Levy, Paul S. and Stanley Lemeshow. 2009. Sampling of Populations: Methods and Applications. New York: John Wiley & Sons, Inc.

**Note: We will discuss in today’s class meeting the requirements and expectations for your research proposal.**

*** Methodology Statement due at before 4:00 pm on 11/20/2018 ***

XII. Data and Measurement (Tues., 11/27/18) Topics/Questions: What are criteria for the evaluation of the quality of data and measures? What are the opportunities and pitfalls of ‘big-data’? What are its benefits and shortcomings of content analysis?

Brancati, Social Scientific Research, Chapters 15, 16, 17.

XIII. Alternative Approaches (Tues., 12/04/18) Topics/Questions: How to combine quantitative methods with qualitative methods? When are mixed methods always superior to single method approaches?

Brancati, Social Scientific Research, Chapter 8.

*** Research Proposal due by 4:00 pm on 12/14/2018 ***

1

GR5010: Theory and Methodology in the Social Sciences

Quantitative Methods in the Social Sciences (QMSS)

Columbia University, Fall 2018

Wednesdays, 4:10-6:00 pm

Instructor: Michael Parrott

Office: Int. Affairs 509f, Fridays, 3 to 5 pm and by appt

E-mail: mp3675@columbia.edu (primary contact)

Class Room: 415 Schapiro

TA: DaHee Shon, ds3144@tc.columbia.edu, Int. Affairs 270J, Wednesdays 3 to 4 pm and by appt

Course Description

This course, one of the two foundational courses in the QMSS program, introduces various analytical

approaches that contemporary social scientists use to investigate fundamental questions about social

phenomena. This class is designed to guide students through some of the theoretical, methodological, and

practical issues associated with the development of research design in the social sciences. Students will learn

how to ask empirical questions; how to answer these questions scientifically using appropriate types of

evidence; and how to clearly convey arguments, evidence, and conclusions to others.

We will focus on two related questions throughout the course: how do we, as social scientists, know what we

know? And, how do scholars distinguish between more and less convincing research? The first question

invokes some understanding of philosophy of science but, more importantly for our purposes, the practicalities

of how scholars decide which theories, tools, methods, and approaches to employ in researching specific

phenomena. The second question involves developing a broad understanding of rigor in research design and a

sympathetic skepticism about research conclusions – skepticism because of the challenges all researchers face

in knowledge production, and sympathy because even the best research designs have limitations.

Rather than learning specific statistical tests or tools, we will think about the logic and design of social

research, beginning with some concepts and topics common to research across the social sciences. We will

then consider different methodologies – experimentation, observation, social surveys, and archival research,

using examples drawn from the literatures.

Prerequisites

It is assumed that you have basic knowledge on regression analysis. QMSS offers short tutorials at the

following link if you need review: http://www.ccnmtl.columbia.edu/projects/qmss/

COURSE MATERIALS

Required text:

Royce A. Singleton, Jr. and Bruce C. Straits. 2010. Approaches to Social Research (6th Edition). New

York: Oxford University Press.

Matthew J. Salganik. 2017. Bit by Bit: Social Research in the Digital Age. Princeton, NJ: Princeton

University Press. Open review edition. Available online here.

Recommended readings for thesis preparation (* denotes these optional readings each week):

Wayne C. Booth, Colomb, Gregory, G., & Williams, Joseph M. 2008. The Craft of Research (3rd

Edition). Chicago: University of Chicago Press.

Anthony Weston. 2009. A Rulebook for Arguments (4th Edition). Indianapolis: Hackett Publishing

Company.

The majority of the other course readings are available through the University’s electronic journals. We will

2

have links to these on the course web page.

Plagiarism and Academic Dishonesty:

Students must do all their work within the boundaries of acceptable academic norms. See the Academic

Honesty page of the CU website regarding college policy on plagiarism and other forms of academic

dishonesty - http://www.columbia.edu/cu/history/ugrad/main/handbook/academic_honesty.html. Students

found guilty of plagiarism or academic dishonesty will be subject to appropriate disciplinary action, which

may include reduction of grade, a failure in the course, suspension or expulsion.

COURSE REQUIREMENTS

Attendance is expected and reading assignments are to be completed before each class session.

The class will be a combination of lectures and focused discussion. Before each class, you should have read all

required readings. We will discuss readings under various specific topics to engage applied theories and

methods, primarily by asking a few pertinent questions: What is the research question? Why was a particular

method chosen? What inferences can be draw from its results? What are its limitations?

The final grade for the course will be based on your successful completion of each of the following

requirements:

• Assignments (30%): There will be five short assignments that will help you to build a research proposal

in small chunks. Assignments #1-3 ask you to write a one page response paper that describes a scholarly

article that relates to your research interests. In assignment #4 you will submit a one to two page brief

overview of your research proposal. And in assignment #5 you will write up a five to seven page mini-

literature review that cites at least seven scholarly articles tied to your research question.

• Research Proposal (30%): Throughout the course of the semester, you will work on a research proposal.

This 12 to 15 page paper will be turned in at the end of the semester.

• Proposal Presentation (10%): Each student will present their research proposal to the class and receive

feedback at the end of the semester.

• Presentation of Class Readings (10%): Students will be assigned to present an overview of, and lead

a discussion about, selected readings in class.

• Class Participation (20%): This course is designed to be a mixed lecture/discussion class. Students

are expected to read all readings before each class and actively participate in class discussion.

COURSE OUTLINE AND READINGS

[Readings may be adapted throughout the course. Check regularly for updates]

Singleton and Straits, Approaches to Social Research: Chapters 1-2.

Singleton and Straits, Approaches to Social Research: Chapters 3.

Lewis A. Coser. 1975. “Two methods in search of a substance.” American Sociological Review 40: 691-700.

David A. Freedman. 1991. “Statistical models and shoe leather.” Sociological Methodology 291-313.

Week 1 (September 5): The “Science” of Social Science and the Role of Statistical Models

Introduction. What this course is (and what it is not). Course overview.

Week 2 (September 12): The “Science” of Social Science and the Role of Statistical Models (Cont’d), Research

Ethics

What is so unique about quantitative methods applied to the social sciences? Why is it useful to have statistical models in

the social sciences? Discussion of research ethics.

3

Michael J. LaCour, and Donald P. Green. 2014. “When contact changes minds: An experiment on transmission of

support for gay equality.” Science. Vol. 346, Issue 6215, pp. 1366-1369.

David Broockman and Joshua Kalla. 2015. “Irregularities in LaCour (2014).” Available at

https://stanford.edu/~dbroock/broockman_kalla_aronow_lg_irregularities.pdf

*The Craft of Research. Chapter 3.

Research reading response assignment #1 due:

One page write-up of article relevant to your research interests

Singleton and Straits, Approaches to Social Research: Chapters 4.

Matthew J. Salganik. 2017. Bit by Bit: Social Research in the Digital Age. Preface and Intro

Leslie Kish. 1987. Statistical Design for Research. New York: John Wiley and Sons. Chapter 1, “Representation,

Randomization, and Realism.”

Holland, Paul W. 1980. “Statistics and Causal Inference” Journal of the American Statistical Association 81(396):945-

960.

Arthur Stinchcombe. 1968. “The logic of scientific inference.” pp. 15-37 in Constructing Social Theories. New

York: Harcourt, Brace, and World.

Topic: Interest Group Influence Through Campaign Contributions – Is There a Causal Link?

Welch, William P. "Campaign Contributions and Legislative Voting: Milk money and Dairy Price Supports." Western

Political Quarterly (1982): 478-495.

Hall, Richard L., and Frank W. Wayman. "Buying time: Moneyed Interests and the Mobilization of Bias in Congressional

Committees." American Political Science Review 84.3 (1990): 797-820.

Kalla, J. L. and Broockman, D. E. (2016), Campaign Contributions Facilitate Access to Congressional Officials: A

Randomized Field Experiment. American Journal of Political Science, 60: 545–558.

*The Craft of Research Chapters 9,10, and 11.

*A Rulebook for Arguments pages 31-36.

Research reading response assignment #2 due:

One page write-up of another article relevant to your research interests

Singleton and Straits, Approaches to Social Research: Chapter 5, “Measurement.”

Barbara Geddes,. 1990. “How the Cases You Choose Affect the Answers You Get: Selection Bias in Comparative Politics”

Political Analysis 2:131-52.

Topic: Social Capital

Robert D. Putnam. 1995. “Bowling alone: America's declining social capital.” Journal of Democracy 6: 65-78.

Pamela Paxton. 1999. “Is social capital declining in the United States? A multiple indicator assessment.” American

Journal of Sociology 105: 88-127.

Evan Schofer and Marion Fourcade-Gourinchas. 2001. "The Structural Contexts of Civic Engagement: Voluntary

Association Membership in Comparative Perspective." American Sociological Review. 66:806-828

*The Craft of Research Chapters 4,7 and 8.

Week 4 (September 26): Concepts and Measurement

Week 3 (September 19): Causality and Causal Inference and Elements of Research Design

4

Research reading response assignment #3 due:

One page write-up of another article relevant to your research interests

Singleton and Straits, Approaches to Social Research: Chapters 7 and 8, “Experimentation” and “Experimental

Design.”

Green, Donald and Alan Gerber. 2003. “The underprovision of experiments in political science.” Annals of the

American Academy of Political and Social Science. 589: 94-112.

Morton, Rebecca and Kenneth Williams. 2010. Experimental Political Science and the Study of Causality. Cambridge

University Press. (chapters 1-2, 11-15).

Topic 1: Racial Discrimination

Marti Loring and Brian Powell. 1988. “Gender, race, and DSM-III: A study of the objectivity of psychiatric

behavior.” Journal of Health and Social Behavior 29: 1-22.

Claude M. Steele. 1997. “A threat in the air: How stereotypes shape intellectual identity and performance.”

American Psychologist, 52, 613-629.

Topic 2: Voter Turnout

Gerber, A. and Green, D. 2000. “The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A

Field Experiment.” American Political Science Review 94 (3), 653-663.

Gerber, A., D. Green and C. Larimer. 2008. “Social Pressure and Vote Turnout: Evidence from a Large -Scale Field

Experiment.” American Political Science Review 102(1): 33-48.

IRB for experiments discussion during class (Oct 10)

*The Craft of Research Chapters 5 and 6.

Matthew J. Salganik. 2017. Bit by Bit: Social Research in the Digital Age. Chapter 4, “Running experiments”

Tutorial: Getting great survey results from MTurk and Qualtrics via Amazon Mechanical Turk site. Link to site:

https://blog.mturk.com/tutorial-getting-great-survey-results-from-mturk-and-qualtrics-f5366f0bd880

In class exercise: An introductory demonstration detailing how to use MTurk to generate survey/survey experiement results

Pro-Publica’s guide to Using Mechanical Turk: https://www.propublica.org/article/propublicas-guide-to-mechanicalturk/

Berinsky, Adam J., Gregory A. Huber, and Gabriel S. Lenz. "Evaluating Online Labor Markets for Experimental

Research: Amazon.com's Mechanical Turk." Political Analysis (2012): 351-68.

Buhrmester, Michael, Tracy Kwang, and Samuel D. Gosling. "Amazon's Mechanical Turk: A new source of

inexpensive, yet high-quality, data?." Perspectives on Psychological Science (2011): 3-5.

*The Craft of Research Chapter 12.

*A Rulebook for Arguments Pages 49-72.

Assignment #4 Research Proposal Brief Overview (1-2 pages)

Singleton and Straits, Approaches to Social Research: Chapter 11

Weeks 5 (October 3): Randomization and Experimental Method

Week 7 (October 17): Observational Studies, Using Big Data for Observational Studies

How are they different from experiments? Understanding their limitations for causal inference. Statistical

tools to cope with effects of non-random assignment of treatments. Inference from observational studies.

Weeks 6 (October 10): Innovative Tools used to gather data for Surveys/Survey Experiments

[NOTE: THIS WEEK WILL BE ONLINE ONLY. ]

5

Donald B. Rubin. 2008. “For Objective Casual Inference, Design Trumps Analysis”

Annals of Applied Statistics 2(3):808-840.

Adam Przeworski. 2007. “Is the Science of Comparative Politics Possible?” In Carles Boix & Susan C. Stokes (ed)

Oxford Handbook of Comparative Politics. New York, NY: Oxford University Press.

Matthew J. Salganik. 2017. Bit by Bit: Social Research in the Digital Age. Chapter 4 and Chapter 5, “Observing

Behavior” and data via “Mass collaboration”

*The Craft of Research Chapter 15.

Assignment 5 Due, Minimum of five maximum of seven page literature review citing minimum of seven articles

related to your research question. Due 11/1

Singleton and Straits, Approaches to Social Research: Chapters 6, “Sampling.”

Matthew J. Salganik. 2017. Bit by Bit: Social Research in the Digital Age. Chapter 4, “Asking Questions”

Marie Hojnacki and David C. Kimball. 1998. “Organized interests and the decision of whom to lobby in Congress.”

American Political Science Review 92(4): 775-90.

Douglas Heckathorn and Joan Jeffri. 2001. “Finding the beat: using respondent-driven sampling to study jazz

musicians.” Poetics (28): 307-329

*The Craft of Research Chapters 13, 14, and 16.

Singleton and Straits, Approaches to Social Research: Chapters 9, “Survey Research.”

Claudia Deane et al. 2016. Flashpoints in Polling. Pew Research Center (Mimeo).

David S. Yeager et al. 2011. “Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with

Probability and Non-Probability Samples” Public Opinion Quarterly 75(4):709-747.

Singleton and Straits, Approaches to Social Research: Chapters 10, “Survey Instrumentation.”

Diane Binson and Joseph A. Catania. 1998. “Respondents’ understanding of the words used in sexual behavior

questions.” Public Opinion Quarterly 62: 190-208.

Topic: National Longitudinal Survey of Adolescent Health

Peter S. Bearman, J. Jones, and J. Richard Udry. 1997. The National Longitudinal Study of Adolescent Health:

Research Design (http://www.cpc.unc.edu/projects/addhealth/resdesign/index.htm)

Peter S. Bearman and Hannah Brückner. 2001. “Promising the future: Virginity pledges and the transition to first

intercourse.” American Journal of Sociology 106: 859-912.

Natalie S. The and Penny Gordon-Larsen. 2009. “Entry into Romantic Partnership is Associated with Obesity.”

Obesity.

Week 8 (October 24): Sampling

Weeks 9 (October 31): Survey Research

General features of survey research.

Weeks 10 (November 7): Survey Instrumentation

What is wrong (and what is right) with polls? Respondent Selection Issues. Response Accuracy Issues. Survey

Administration Issues.

6

Fowler, J.H., and Christakis, N.A. 2008. “Dynamic spread of happiness in a large social network: Longitudinal

analysis over 20 years in the Framingham Heart Study.” BMJ 357: 370-379.

Cohen-Cole, E., and Fletcher, J.M. 2008. “Detecting implausible social network effects in acne, height, and

headaches: Longitudinal analysis.” BMJ 357.

Singleton and Straits, Approaches to Social Research: Chapter 12, “Research Using Available Data.”

Roberto Franzosi. 1987. “The press as a source of socio-historical data: Issues in the methodology of data collection

from newspapers.” Historical Methods 20(1): 5-16.

Lee Drutman. and Hopkins, D. J. (2013), The Inside View: Using the Enron E-mail Archive to Understand Corporate

Political Attention. Legislative Studies Quarterly, 38: 5–30. doi:10.1111/lsq.12001

Art Budros. 2004. “Social shocks and slave social mobility: Manumission in Brunswick County, Virginia, 1782 -

1862.” American Journal of Sociology 110: 539-79.

Justin Grimmer & Brandon Stewart. 2013. “Text as Data: The Promise and Pitfalls of Automatic Content Analysis

Methods for Political Documents” Political Analysis 21(3):267-297.

Kenneth Benoit, Michael Laver & Slava Mikhaylov. 2009. “Treating words as data with error: Uncertainty in text

statements of policy positions.” American Journal of Political Science 53(2):495–513.

Drafts of Research Proposals Due 11/21.

Drafts of Research Proposals Returned. Final Revision Due 12/14.

Week 12 (November 21): Happy Thanksgiving! No Class. Wednesday is University Holiday.

Week 13 (November 28): Proposal Presentations

Week 14 (December 5): Proposal Presentations

Week 11 (November 14): Text as Data:

Archival research. Text beyond archives.

THEORIES AND METHODOLOGIESOF THE SOCIAL SCIENCES

Columbia UniversityGR5010, FALL 2018

Thurs, 6:10PM-8:00PMTBD

Instructor: Marco MoralesEmail: marco.morales@columbia.eduOffice: 270 International Affairs BuildingOffice Hours: Thurs 8-9PM, and by appointment

TA: TBDEmail: tbd@columbia.edu

I. Overview

This course — one of the two foundational courses in the QMSS curriculum — is designed

as an in-depth introduction to the social sciences and its methodologies. It is intended

to give a broad overview so students can intelligently combine ideas in solving real-world

problems.

We will focus on the logic and design of social research, beginning with some concepts

and topics common to research across the social sciences. We will later move on to un-

derstanding the principles behind an array of methodologies used in the social sciences:

causal inference, experimentation, observational studies, formal models, surveys, and ap-

plied machine-learning techniques. We will analyze their applications using cases drawn

from the research literature.

The focus of this course is not on the techniques themselves — you will have ample op-

portunity to do that in other courses — but in understanding the logic behind the use of

these tools to extract meaningful answers from their applications.

Prerequisites: it is assumed that you have had at least one semester of graduate-level

statistics involving linear regression and analysis of variance. Some basic mathematics and

algebra will also be assumed.

Updated: May 1, 2018 at 12:39 hrs 1

GR5010, Fall 2018

II. Course Materials

Two texts are intended to be resources for the crafting of your thesis proposal:

• Booth, W. C., Colomb, G. C., and Williams, J. M. (2016). The Craft of Research.University of Chicago Press, Chicago, IL, fourth edition.• Weston, A. (2009). A Rulebook for Arguments. Hackett Publishing Co, Indianapolis,

IN, fourth edition.

All mandatory readings will be available on Canvas. Please note that:

Required Readings are mandatory and should be completed before each class.

Topic Readings are also mandatory as they will serve as the foundation to class discus-

sions.

Thesis Readings are intended to help you with the drafting and crafting of the Research

Proposal, but will not be discussed in class. Read them in the suggested order/dates.

Complementary Readings are intended to serve as further (and future) references if

you ever want to delve deeper on a particular topic, but are not a requirement to this class.

III. Course Dynamics

The class will be a combination of lectures and focused discussion. We will devote the

first half of the class to a lecture on the topic assigned for the week, and the second

half to analyze in detail one or two Topic readings which contain applications of the

method under discussion. To kick off these discussions, students will give 5-minute Topic

Presentations to introduce each reading. We’ll proceed then to discuss the method in

detail. Each presentation should be preceded by a one-page summary to be emailed to

the entire class at 6PM on the day prior to class.

Before each class, you should have read, thought about and be prepared to

discuss all assigned Required and Topic Readings.

Over the course of the semester, you will be required to complete two (2) Assignments

where you will apply concepts and methods that you have learned in class. Make sure to

Updated: May 1, 2018 at 12:39 hrs 2

GR5010, Fall 2018

read the instructions carefully and address all questions. Assignments should be submit-

ted on Canvas by 6PM on the indicated dates.

At the end of the course, you will turn in a 12-15 page Thesis Proposal. To ensure that

you produce a fully fledged product, you will be required to hand in Thesis Proposal

Deliverables every two weeks. These will constitute the foundational pieces for your

proposal. Feedback will be provided on these deliverables where required. Deliverables

should be should be submitted on Canvas by 6PM on the indicated dates.

IV. Course Requirements

Attendance is expected and reading assignments are to be completed before each session.

All written work must be original and produced exclusively for this class. You are expected

to follow the Universitys guidelines for the submission of written work.

The final grade of the course will be based of your fulfillment of each of the following

requirements:

Assignments (20%): Students must complete a series of assignments where methods from

the course are applied. Make sure to submit each one of them by 6PM on the indicated

dates.

Class participation (20%): Students are expected to have read all the required readings

before class and actively participate in class discussion. Note that you will not obtain this

20% unless you actively participate in class.

Topic presentations (10%): Students will regularly be assigned to present an overview

of a “Topics” reading. Make sure to send your one-page summary to the full class by 6PM

on the day prior to the class when you are presenting.

Thesis Proposal Deliverables (20%): Students will submit pieces of their proposal

every two weeks. Make sure to submit each one of them by 6PM on the indicated dates.

Thesis proposal (30%): Throughout the curse, you will work on a research proposal.

This 12-15 page paper will be turned in at the end of the semester.

Late Submission Policy: Course requirements are expected to be submitted on the due

date. For every day after the submission date, 10% of the maximum grade will be deducted

from the score.

Updated: May 1, 2018 at 12:39 hrs 3

GR5010, Fall 2018

V. Course Requirements

WEEK 1: INTRODUCTION TO THE COURSE

What this course is (and what it is not). Course overview. What is so unique aboutquantitative methods applied to the social sciences? Why do we need models to understandthe world? Why is it useful to have statistical models in the social sciences?

WEEK 2: THE “SCIENCE” OF SOCIAL SCIENCE

What is so scientific about social sciences? A look at “the method”. Inductive v deduc-tive perspectives. Theories, hypothesis and falsifiability. Links to quantitative methods.Mechanisms.

Required Readings:

• Gelman, A. (2011). Induction and deduction in Bayesian data analysis. Rationality,Markets and Morals, 2:67–78.• Elster, J. (2007). Explaining Social Behavior: More Nuts and Bolts for the Social

Sciences. Cambridge University Press, Cambridge, MA. [Ch 1-2]

Thesis Readings:

• Booth, W. C., Colomb, G. C., and Williams, J. M. (2016). The Craft of Research.University of Chicago Press, Chicago, IL, fourth edition. [Ch 3]• Weston, A. (2009). A Rulebook for Arguments. Hackett Publishing Co, Indianapolis,

IN, fourth edition. [Ch I-II]

Complementary Readings:

• Popper, K. (2002[1935]). The Logic of Scientific Discovery. Routledge, New York,NY.• Kuhn, T. S. (2012). The Structure of Scientific Revolution. University of Chicago

Press, Chicago, IL, fourth edition.• King, G., Keohane, R. O., and Verba, S. (1995). Designing Social Inquiry: Scientific

Inference in Qualitative Research. Princeton University Press, Princeton, NJ.• Gelman, A. and Shalizi, C. R. (2013). Philosophy and the practice of Bayesian

statistics. British Journal of Mathematical and Statistical Psychology, 66(1):8–38.• Meehl, P. E. (1967). Theory-testing in psychology and physics: A methodological

paradox. Philosophy of Science, 34:103–115.

Updated: May 1, 2018 at 12:39 hrs 4

GR5010, Fall 2018

WEEKS 3 | 4: CAUSALITY AND CAUSAL INFERENCE (I & II)

Causes of effects or effects of causes? The search for causes: from Aristotle to Fisher. Thefundamental problem of causal inference. The Neyman-Rubin model.

Required Readings:

• Holland, P. W. (1980). Statistics and causal inference. Journal of the AmericanStatistical Association, 81(396):945–960.• Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modelling,

decisions. Journal of the American Statistical Association, 100(469):322–331.

Thesis Readings:

• Booth, W. C., Colomb, G. C., and Williams, J. M. (2016). The Craft of Research.University of Chicago Press, Chicago, IL, fourth edition. [Ch 4]• Weston, A. (2009). A Rulebook for Arguments. Hackett Publishing Co, Indianapolis,

IN, fourth edition. [Ch VI-VII]

Complementary Readings:

• Imbens, G. and Rubin, D. B. (2015). Causal Inference for Statistics, Social andBiomedical Sciences: An Introduction. Cambridge University Press, Cambridge, MA.• Pearl, J. (2009). Causality: Models, Reasoning and Inference. Cambridge University

Press, Cambridge, MA, second edition.• Sekhon, J. S. (2004). Quality meets quantity: Case studies, conditional probability

and counterfactuals. Perspectives on Politics, 2(2):281–293.• Dawid, A. (2000). Causal inference without counterfactuals. Journal of the American

Statistical Association, 95(450):407–448.• Page, S. (2006). Path dependence. Quarterly Journal of Political Science, 1(1):87–

115.

[MM/DD] - Thesis Proposal Deliverable #1 (research topic) due.[MM/DD] - Students will receive Assignment #1.

Updated: May 1, 2018 at 12:39 hrs 5

GR5010, Fall 2018

WEEK 5: EXPERIMENTS AND RANDOMIZATION

Theoretical Foundations of Experiments. Statistical foundations of experiments. Taxonomyof randomized experiments. Randomized experiments as the golden standard for causalinference. Inference from randomized experiments.

Required Readings:

• Green, D. and Gerber, A. (2003). The under-provision of experiments in PoliticalScience. Annals of the American Academy of Political Science, 589:94–112.• Angrist, J. D. and Pischkem, J.-S. (2015). Mastering ‘Metrics’: The Path from Cause

to Effect. Princeton University Press, Princeton, NJ.

Topic: Voter Turnout

• De la O, A. (2013). Do conditional cash transfers affect electoral behavior? Evidencefrom a randomized experiments in Mexico. American Journal of Political Science,57(1):1–14.• Wantchekon, L. (2003). Clientelism and voting behavior: Evidence from a field ex-

periment in Benin. World Politics, 55(3):399–422.

Thesis Readings:

• Booth, W. C., Colomb, G. C., and Williams, J. M. (2016). The Craft of Research.University of Chicago Press, Chicago, IL, fourth edition. [Ch 15]

Complementary Readings:

• Martel Garcıa, F. and Wantchekon, L. (2010). Theory, external validity, and ex-perimental inference: Some conjectures. Annals of the American Academy of PoliticalScience, 628(132-147).• Angrist, J. D. and Pischkem, J.-S. (2009). Mostly Harmless Econometrics: An Em-

piricst’s Companion. Princeton University Press, Princeton, NJ.• Imbens, G. and Rubin, D. B. (2015). Causal Inference for Statistics, Social and

Biomedical Sciences: An Introduction. Cambridge University Press, Cambridge, MA.• Morton, R. and WIlliams, K. (2010). Experimental Political Science and the Study

of Causality. Cambridge University Press, Cambridge, MA.

[MM/DD] - Thesis Proposal Deliverable #2 (hypotheses) due.[MM/DD] - Assignment #1 due.

Updated: May 1, 2018 at 12:39 hrs 6

GR5010, Fall 2018

WEEK 6: OBSERVATIONAL STUDIES

How are they different from experiments? Understanding their limitations for causal infer-ence. Statistical tools to cope with non-random assignment of treatments. Inference fromobservational studies.

Required Readings:

• Rubin, D. B. (2008). For objective causal inference, design trumps analysis. TheAnnals of Applied Statistics, 2(3):808–840.• Przeworski, A. (2009). Is the science of Comparative Politics possible? In Boix, C. and

Stokes, S. C., editors, Oxford Handbook of Comparative Politics. Oxford University,Press, New York, NY.

Topic: Selected statistical applications to observational studies

• Abadie, A., Diamond, A., and Hainmueller, J. (2010). Synthetic control methodsfor comparative case studies: Estimating the effect of California’s tobacco controlprogram. Journal of the American Statistical Association, 105(490):493–505.• Erikson, R. S. and Titiunik, R. (2015). Using regression discontinuity to uncover the

personal incumbency advantage. Quarterly Journal of Political Science, 10(1):101–119.

Complementary Readings:

• Cochran, W. G. (2015[1972]). Observational studies. Observational Studies, 1(1):126–136.• Rubin, D. B. (2006). Matched sampling for causal effects. Cambridge University

Press, Cambridge, MA.• Rosenbaum, P. R. (2002). Observational Studies. Springer, New York, NY.• Imai, K., King, G., and Stuart, E. A. (2008). Misunderstandings between experimen-

talists and observationalists about causal inference. Journal of the royal statisticalsociety: series A (statistics in society), 171(2):481–502.• Winship, C. and Morgan, S. L. (1999). The estimation of causal effects from obser-

vational data. Annual Review of Sociology, 25(1):659–706.• Sekhon, J. S. and Titiunik, R. (2012). When natural experiments are neither natural

nor experiments. American Political Science Review, 106(1):35–57.

Updated: May 1, 2018 at 12:39 hrs 7

GR5010, Fall 2018

WEEK 7: CONCEPTS, MEASUREMENT, AND MEASUREMENT ERROR

Research design and the research question. Measurements as a function of concepts. Theo-retical consequences of measurement error. Statistical consequences of measurement error.

Required Readings:

• Geddes, B. (1990). How the cases you choose affect the answers you get: Selectionbias in comparative politics. Political Analysis, 2(1):131–150.• Hausman, J. (2001). Mismeasured variables in econometric analysis: problems from

the right and problems from the left. Journal of Economic Perspectives, 15(4):57–67.

Topic: Economic Perceptions

• Michelitch, K., Morales, M. A., Owen, A., and Tucker, J. A. (2012). Looking to thefuture: Prospective economic voting in 2008 presidential elections. Electoral Studies,31(4):838–851.

Thesis Readings:

• Booth, W. C., Colomb, G. C., and Williams, J. M. (2016). The Craft of Research.University of Chicago Press, Chicago, IL, fourth edition. [Ch 7-9, 12]

Complementary Readings:

• Alwin, D. F. (2007). Margins of Error: A Study of Reliability in Survey Measurement.John Wiley & Sons, Hoboken, NJ.• Goertz, G. (2008). Concepts, theories, and numbers: A checklist for constructing, eval-

uating, and using concepts or quantitative measures. In Box-Steffensmeier, J., Brady,H. E., Collier, D., and Goertz, G., editors, Oxford Handbook of Political Methodology.Oxford University Press, New York, NY.• Jackman, S. (2008). Measurement. In Box-Steffensmeier, J., Brady, H. E., Collier, D.,

and Goertz, G., editors, Oxford Handbook of Political Methodology. Oxford UniversityPress, New York, NY.

[MM/DD] - Thesis Proposal Deliverable #3 (data description) due.

Updated: May 1, 2018 at 12:39 hrs 8

GR5010, Fall 2018

WEEK 8: IDENTIFICATION

Graph models and identification. Recursive causation. The empirical - and theoretical- problems of endogeneity. Quantitative methods to address identification. Instrumentalvariables.

Required Readings:

• Angrist, J. D. and Pischkem, J.-S. (2015). Mastering ‘Metrics’: The Path from Causeto Effect. Princeton University Press, Princeton, NJ. [Ch 3]• Morgan, S. L. and Winship, C. (2014). Counterfactuals and Causal Inference: Meth-

ods and Principles for Social Research. Cambridge University Press, Cambridge, MA,second edition. [Ch 3]

Topic: Selected statistical applications to address identification

• Miguel, E., Satyanath, S., and Sergenti, E. (2004). Economic shocks and civil conflict:An instrumental variables approach. Journal of Political Economy, 112(4):725–753.• Acemoglu, D., Johnson, S., and Robinson, J. A. (2001). The colonial origins of

comparative development: An empirical investigation. American Economic Review,91(5):1369–1401.

Complementary Readings:

• Manski, C. F. (1999). Identification Problems in the Social Sciences. Harvard Uni-versity Press, Cambridge, MA.• Angrist, J. D. and Krueger, A. B. (2001). Instrumental variables and the search for

identification: From supply and demand to natural experiments. Journal of Economicperspectives, 15(4):69–85.• Antonakis, J., Bendahan, S., Jacquart, P., and Lalive, R. (2014). Causality and

endogeneity: Problems and solutions. In Oxford Handbook of Leadership and Orga-nizations. Oxford University Press, New York, NY.• Page, S. (2006). Path dependence. Quarterly Journal of Political Science, 1(1):87–

115.

Updated: May 1, 2018 at 12:39 hrs 9

GR5010, Fall 2018

WEEKS 9 | 10: SURVEY RESEARCH AND SURVEY METHODOLOGY (I& II)

What is wrong (and what is right) with polls? The Total Survey Error Paradigm. Respon-dent Selection Issues. Response Accuracy Issues. Survey Administration Issues.

Required Readings:

• Bautista, R. (2012). An overlooked approach in survey research: Total Survey Error.In Handbook of Survey Methodology for the Social Sciences. Springer, New York, NY.• Kennedy, C., Blumenthal, M., Clement, S., Clinton, J. D., Durand, C., Franklin, C.,

McGeeney, K., Miringoff, L., Olson, K., Rivers, D., Saad, L., Witt, E., and Wlezien,C. (2017). An evaluation of 2016 election polls in the United States. AAPOR AdHoc Committee on 2016 Election Polling.

Topic: Question Wording and Response Scales

• Lundmark, S., Gilljam, M., and Dahlberg, S. (2015). Measuring generalized trust:An examination of question wording and the number of scale points. Public OpinionQuarterly, 80(1):26–43.

Topic: Non-response and Data Quality

• Fricker, S. and Tourangeau, R. (2010). Examining the relationship between nonre-sponse propensity and data quality in two national household surveys. Public OpinionQuarterly, 74(5):934–955.

Topic: Mode of Data Collection

• Sakshaug, J. W., Yan, T., and Tourangeau, R. (2010). Nonresponse error, measure-ment error, and mode of data collection: Tradeoffs in a multi-mode survey of sensitiveand non-sensitive items. Public Opinion Quarterly, 74(5):907–933.

Topic: Probabilty v Non-probability Samples

• Yeager, D. S., Krosnick, J. A., Chang, L., Javitz, H. S., Levendusky, M. S., Simpser,A., and Wang, R. (2011). Comparing the accuracy of rdd telephone surveys andinternet surveys conducted with probability and non-probability samples. PublicOpinion Quarterly, 75(4):709–747.

Thesis Readings:

• Booth, W. C., Colomb, G. C., and Williams, J. M. (2016). The Craft of Research.University of Chicago Press, Chicago, IL, fourth edition. [Ch 13-14, 16-17]• Weston, A. (2009). A Rulebook for Arguments. Hackett Publishing Co, Indianapolis,

IN, fourth edition. [Ch VIII]

Updated: May 1, 2018 at 12:39 hrs 10

GR5010, Fall 2018

Complementary Readings:

• Weisberg, H. F. (2009). The Total Survey Error Approach: A guide to the new scienceof survey research. University of Chicago Press, Chicago, IL.• Tourangeau, R., Rips, L. J., and Rasinski, K. (2000). The Psychology of Survey

Response. Cambridge University Press, Cambridge, MA.• Foster, I., Ghani, R., Jarmin, R. S., Kreuter, F., and Lane, J. (2016). Big Data and

Social Science: A practical guide to methods and tools. CRC Press, New York, NY.• De Leeuw, E. D., Hox, J., and Dillman, D. (2012). International Handbook of Survey

Methodology. Routledge, New York, NY.

[MM/DD] - Thesis Proposal Deliverable #4 (literature review) due.[MM/DD] - Students will receive Assignment #2.

WEEKS 11 | 13: RATIONAL CHOICE AND FORMAL MODELS (I & II)

Formal models of social behavior. The rational choice paradigm. What is rationality? Whatare its limitations? Can rational choice help explain collective action?

Required Readings:

• Riker, W. H. (1995). The political psychology of rational choice theory. PoliticalPsychology, 16(1):23–44.• Arrow, K. J. (1994). Methodological individualism and social knowledge. The Amer-

ican Economic Review, 84(2):1–9.

Topic: Institutionalism

• Przeworski, A. (2005). Democracy as an equilibrium. Public Choice, 123(3-4):253–273.

Topic: Partisanship and the transmission of partisan attachments

• Achen, C. H. (2002). Parental socialization and rational party identification. PoliticalBehavior, 24(2):151–170.

Topic: Legislative Behavior

• Krehbiel, K. (1998). Pivotal politics: A theory of US lawmaking. University of ChicagoPress, Chicago, IL. [Ch 2]

Updated: May 1, 2018 at 12:39 hrs 11

GR5010, Fall 2018

Topic: Rational Choice and Its Critics

• Kahneman, D. and Tversky, A. (1979). Prospect theory: An analysis of decisionunder risk. Econometrica, 47(2):263–292.

Complementary Readings:

• Friedman, M. (2008). The methodology of positive economics. In Hausman, D. J.,editor, The Philosophy of Economics. Cambridge University Press, Cambridge, MA.• Becker, G. S. (2013). The economic approach to human behavior. University of

Chicago Press, Chicago, IL.• Sen, A. (1999). The possibility of social choice. American Economic Review, 89(3):349–

378.• Morton, R. B. (1999). Methods and models: A guide to the empirical analysis of

formal models in political science. Cambridge University Press, Cambridge, MA.• Osborne, M. J. (2004). An Introduction to Game Theory. Oxford University Press,

New York, NY.• Kahneman, D. (2011). Thinking Fast and Slow. Farrar, Strauss and Groux, New

York, NY.• Chwe, M. S.-Y. (2014). Jane Austen: Game Theorist. Princeton University Press,

Princeton, NJ.

[MM/DD] - Thesis Proposal Deliverable #5 (research strategy) due.[MM/DD] - Assignment #2 due.[MM/DD] - Thesis Proposal Deliverable #6 (research proposal draft) due.

WEEK 12: ACADEMIC HOLIDAY

WEEK 14: TEXT AS DATA

Text analysis beyond Natural Language Processing. Statistical and algorithmic analysis of(text) data. Big Data, Machine Learning and Causal Inference.

Required Readings:

• Grimmer, J. and Stewart, B. M. (2013). Text as data: The promise and pitfalls ofautomatic content analysis methods for political texts. Political Analysis, 21(3):267–297.

Updated: May 1, 2018 at 12:39 hrs 12

GR5010, Fall 2018

Topic: Estimating Ideology from Texts

• Laver, M., Benoit, K., and Garry, J. (2003). Extracting policy positions from politicaltexts using words as data. American Political Science Review, 97(2):311–331.• Barbera, P. (2014). Birds of the same feather tweet together: Bayesian ideal point

estimation using twitter data. Political Analysis, 23(1):76–91.

Complementary Readings:

• Manning, C., Raghavan, P., and Schutze, H. (2009). An Introduction to InformationRetrieval. Cambridge University Press, Cambridge, MA.

[MM/DD] - FINAL PAPER DUE.

Updated: May 1, 2018 at 12:39 hrs 13

GR5010, Fall 2018

Statement on Academic Integrity

Columbia’s intellectual community relies on academic integrity and responsibility as the

cornerstone of its work. Graduate students are expected to exhibit the highest level of

personal and academic honesty as they engage in scholarly discourse and research. In

practical terms, you must be responsible for the full and accurate attribution of the ideas

of others in all of your research papers and projects; you must be honest when taking your

examinations; you must always submit your own work and not that of another student,

scholar, or internet source. Graduate students are responsible for knowing and correctly

utilizing referencing and bibliographical guidelines. When in doubt, consult your professor.

Citation and plagiarism-prevention resources can be found at the GSAS page on Academic

Integrity and Responsible Conduct of Research.

Failure to observe these rules of conduct will have serious academic consequences, up to

and including dismissal from the university. If a faculty member suspects a breach of

academic honesty, appropriate investigative and disciplinary action will be taken following

the Dean’s Discipline procedures.

Statement on Disability Accommodations

If you have been certified by Disability Services (DS) to receive accommodations, please

either bring your accommodation letter from DS to your professor’s office hours to confirm

your accommodation needs, or ask your liaison in GSAS to consult with your professor.

If you believe that you may have a disability that requires accommodation, please contact

Disability Services at 212-854-2388 or disability@columbia.edu.

Important: To request and receive an accommodation you must be certified by DS.

Updated: May 1, 2018 at 12:39 hrs 14