Harvard Summer School, 2013Graduate Research Methods and Scholarly Writing in the Social Sciences: Government and History

Harvard Summer School: SSCI S-100b Section 2 (32761)

Joe BondClass 5

July 8, 2013

Agenda

• Announcements• Research Related Presentation• Facilitation (Richard)• In-Class 5

• Volunteers to facilitate next week?– July 15th Midterm Exam– July 17th

3

Logic of Nomothetic Explanation• Using a few factors (independent variables) to

account for many variations in a given phenomenon

• Unlike idiographic explanation, nomothetic explanation is probabilistic or usually incomplete

• Factors that might cause attitudes about the legalization of marijuana

• Political orientation cause attitudes toward legalization of marijuana even though not all liberals approve nor all conservatives disapprove

4

Criteria for Nomothetic Causality

1. The variables must be correlated2. The cause takes place before the effect3. The variables are non-spurious

Some times referred to as internal validity

5

Necessary and Sufficient Causes• A necessary cause represents a condition that

must be present for the effect to follow• It is necessary to take college courses in order

to get a degree• But simply taking courses is not a sufficient

cause of getting a degree• You need to take the right ones and pass• Being female is a necessary condition of

becoming pregnant, but it is not a sufficient cause

6

• Unfortunately, we almost never discover single causes that are absolutely necessary and absolutely sufficient when analyzing nomothetic relationships

• It is not uncommon, however, to find causal factors that are either 100% necessary (you must be female to become pregnant) or 100% sufficient (skipping every exam will inevitably cause you to fail it)

7

Units of Analysis

• Nomothetic research often looks at large collections of people or things, or aggregates

• It is important to distinguish between the unit of analysis and the aggregates that we generalize about

• We may want to study individuals as actors or groups of individuals as actors

8

Typical Units of Analysis

• Individuals• Groups• Organizations• Social Artifacts (e.g. books, poems,

automobiles)• Countries

9

Ecological Fallacy

• Ecological refers to groups or sets or systems: something larger than individuals

• The ecological fallacy is the assumption that something learned about an ecological unit says something about the individuals making up that unit.

10

Example• We want to know something about the nature of

electoral support received by a female candidate• We have the vote tally for each precinct so we

know which precincts gave her the most support• We also know that most of her support came

from precincts with relatively young voters• Conclusion: younger voters gave the most

support – an ecological fallacy has been committed

11

Reductionism• A second type of potentially faulty reasoning related to

units of analysis is reductionism• Reductionism means seeing and explaining complex

phenomena in terms of a single, narrow concept or set of concepts

• What caused the American Revolution?• A shared commitment to the value of individual

liberty?• The economic plight of the colonies in relation to

Britain?• The megalomania of the founders?

12

The Dimension of Time• Cross-Sectional Study – A study based on

observations representing a single point in time

• Longitudinal Study – A study design involving the collection of data at different points in time

• Trend Study – A type of longitudinal study that examines changes within a population over time (comparison of U.S. Censuses over a period of decades)

13

The Dimension of Time, Continued

• Cohort Study – the study of specific subpopulations, or cohorts, as they change over time (e.g. series of surveys conducted over the years to study attitudes of the cohort born during WW II toward U.S. involvement in foreign affairs)

• Panel Studies – like a cohort study but they examine the same set of people each time

14

Questions to Ask

• Does the study seem to be exploring, describing, or explaining (or some combination of these)?

• What are the sources of data?• Can you identify the unit of analysis?• Is the dimension of time relevant? If so, how is

it handled?• Group exercize

15

Basic Elements of a Research Proposal

• Problem or Objective– What exactly do you want to study?– Why is it worth studying?– Does the proposed study have practical

significance?– Does it contribute to the construction of social

theories?

16

Basic Elements of a Research Proposal

• Literature Review– What have others said about this topic?– What theories address it and what do they say?– What previous research exists?– Are there consistent findings, or do past studies

disagree?– Are there flaws in the body of existing literature

that you think you can remedy?

17

Basic Elements of a Research Proposal

• Subjects for Study– Whom or what will you study in order to collect

data?– Identify the subjects in general, theoretical terms.– In specific, more concrete terms, identify who is

available for study and how you’ll reach them.– Will it be appropriate to select a sample? If so,

how will you do that?– Is there any possibility that your research will

affect those you study, how will you ensure you don’t harm them? 18

Basic Elements of a Research Proposal

• Measurement– What are the key variables in your study?– How will you define and measure them?– Do your definitions and measurement methods

duplicate or differ from those of previous research on this topic?

– Have you have already developed your measurement device (e.g. a questionnaire) or will you use something previously developed by others?

19

Basic Elements of a Research Proposal

• Data-Collection– How will you actually collect your data?– Will you conduct an experiment or a survey?– Will you undertake field research or will you focus

on the reanalysis of statistics already created by others?

– Perhaps you will use more than one method.

20

Basic Elements of a Research Proposal

• Analysis– Indicate the kind of analysis you plan to conduct.– Spell out the purpose and logic of your analysis.– Are you interested in precise description?– Do you intend to explain why things are the way

they are?– Do you account for variations in some quality: for

example, why some students are more liberal than others?

– How will you know if you have explained variations adequately? 21

Measurement• Can we measure anything?• Most variables we want to study don’t actually

exist in the way a rock exists• Political party affiliation

– A) look at voter registration lists– B) ask voters which party they identify with– A & B reflect different definitions of political party

affilitation– They may even produce different results

22

Indicators and Dimensions

• Indicator: An observation that we choose to consider as a reflection of a variable we wish to study. For example, attending church might be considered an indicator of religiosity.

• Dimension: A specifiable aspect of concept.

23

Reliability• That quality of measurement method that

suggests that the same data would have been collected each time in repeated observations of the same phenomenon.

• In the context of a survey, we would expect that the question “Did you attend church last week?” would have higher reliability than the question “About how many times have you attended church in your life?

• Do not confuse this with validity.24

Validity• A term describing a measure that accurately

reflects the concept it is intended to measure.• IQ would seem a more valid measure of you

intelligence than would the number of hours you spend in the library

• Though the ultimate validity of a measure can never be proven, we may agree to its relative validity on the basis of face validity, criterion validty, content validity, construct validity, internal and external validities

25

Face Validity

• The quality of an indicator that makes it seem like a reasonable measure of some variable

• That the frequency of church attendance is some indicator of a person’s religiosity seems to make sense without a lot of explanation

• It has face validity

26

Criterion Validity

• The degree to which a measure relates to some external criterion

• For example, the College Board is shown in their ability to predict the college success of students

• Also called predictive validity

27

Construct Validity

• The degree to which a measure relates to other variables as expected within a system of theoretical relationships

• You want to measure marital happiness• Assume couples who are faithful tend to be

happier than those who are not• If you measure relates to fidelity, that

constitutes evidence of your measure’s construct validity

28

Content Validity

• The degree to which a measure covers a range of meanings included within the concept

• A test of mathematical ability can’t be limited to addition alone

• If we are measuring prejudice, do our measurements reflect all types of prejudice, including prejudice against racial and ethnic groups, religious minorities, women, the elderly, and so on?

29

Index

• A type of composite measure that summarizes and rank-orders several specific observations and represents some more general dimension

• We might construct an index by simply accumulating scores assigned to individual attributes

• We might measure prejudice, for example, by adding up the number of prejudiced statements each respondent agreed with

30

Scale

• A type of composite measure composed of several items that have logical or empirical structure among them

• Examples include Guttman, Likert, and Thurstone scales

31

U.S. News and World Report College Rankings

• Rankings reflect an index created from items like educational expenditures/student, graduation rates, selectivity, SAT scores of 1st year students, etc.

• 1999 Cal Tech moved from 9th to 1st

• Expenditures/student were considered as rankings (i.e. 1, 2, 3) and not the actual amount it spent relative to other schools

32

Scale Construction• 1995 UN set out to examine the status of

women in the world• Created 2 indexes reflecting 2 different

dimensions– Gender-related Development Index (GDI)

• Life expectancy, education, income– Gender Empowerment Measure (GEM)

• Proportion of parliamentary seats held by women, proportion of administrative, managerial, professional, and technical positions held by women

33

How Would you Rate Colleges and Universities

• What criteria would you use?

34

Non-probability Sampling

• Any technique in which samples are selected in some way not suggested by probability theory

• Examples include reliance on available subjects as well as purposive (judgmental), quota, and snowball sampling

35

Purposive (judgmental) Sampling

• A type of non-probability sampling in which you select the units to be observed on the basis of your own judgment about which ones will be the most useful

• Want to compare left-wing and right-wing leaning students

• Sample the memberships of right and left wing groups

36

Snowball Sampling• A non-probability sampling method often

employed in field research whereby each person interviewed may be asked to suggest additional people for interviewing

• Wish to learn a community organization’s pattern of recruitment over time

• Might begin by interviewing fairly recent recruits, asking them who introduced them to the group

• You might then interview the people named37

Quota Sampling

• A type of non-probability sampling in which units are selected into a sample on the basis of pre-specified characteristics, so that the total sample will have the same distribution of characteristics assumed to exist in the population being studied

38

Simple Random Sample

• A type of probability sampling in which the units composing a population are assigned numbers.

• A set of random numbers is then generated, and the units having those numbers are included in the sample

39

Systematic Sampling

• A type of probability sampling in which every kth unit in a list is selected for inclusion in the sample

• Compute k by dividing the size of the population by the desired number of the desired sample size

40

Stratified Sampling

• The grouping of units composing a population into homogeneous groups before sampling

• Improves the representativeness of a sample• Stratified sample of university students

– First organize your population by college class and then draw appropriate numbers of freshmen, sophomores, Juniors, seniors

41

5th In-Class Writing Exercise• You are interested in researching some behavioral

phenomenon exhibited by a group of individuals – say you want to learn about the type of person who still does not wear seatbelts even though virtually every state in the country requires their use. Alternatively, you might want to learn about the type of person who still smokes cigarettes… after decades of research advising people that smoking kills.

• Using one of these two examples, and on this side of this sheet only, please state a research question, the null hypothesis for it, and identify the type of research you will use and explain how you intend to execute your study.

42