case selection and sampling in social science methodology

7/29/2019 Case Selection and Sampling in Social Science Methodology

1/13

J A Y C O L B U R N

R Y A N L A M B

S H A H N O Z A N O Z I M O V A

Case Selection and Sampling


2/13

Case Selection and Sampling

I . W H A T I S A C A S E ?

I I . S E L E C T I O N B I A S

I I I . D E A L I N G W I T H S E L E C T I O N B I A S


3/13

I. What is a case?

The term case is a methodological construct thathas become distorted corrupted over time

(Ragin, p. 3).

Individual social scientists answer the questionWhat is a case? in remarkably different ways

and answers to this question affect the conductand results of research (Ragin, p. 8).


4/13

I. Conceptual Map for answers to What is a case?

Understanding ofcases

Case Conceptions

Specific (Qualitative) General (Quantitative)

As empirical units(relist)

1. Cases are found(Berkley School; Collieret al.*)

2. Cases are objects(Geddes; Collier et al.**)

As theoretical constructs(nominalist)

3. Cases are made(Collier and Mahoney;Small)

4. Cases areconventions

Source: Ragin, p. 9

* Within-case analysis** Full data-set analysis


5/13

I. Questions

1. Do you agree with the presented typology of theliterature?

2. Becker (in Ragins presentation) wants theresearchers not to have strong preconceptionsabout their cases. The less sure researchers areof their answers [to the question What is this acase of?], the better their research may be. Raginseems to be troubled by this approach. Where do

you stand on this debate?


6/13

II. Selection Bias

Definitions

Bias is systematic error that is expected

Selection bias occurs when the process in the designresults in systematic error Self-selection of an explanatory variable

Values of explanatory variable are affected by the values of the

DV Selection of cases that have extreme values along the DV


7/13

II. Selection Bias

Selection Bias is easily understood in the context ofquantitative analysis

Selection Bias also occurs in Qualitative analysis

In short, selecting cases on the dependent variable

entails a high probability of getting the wronganswer (Geddes, p. 149).


8/13

II. Selection Bias

Illustration of selection bias resulting from truncation

Source: Collier et. al., p. 91


9/13

II. Questions

1. If selecting cases along the DV can be dangerous, isit still valuable?

2. When should cases be selected along the DV?When should cases not be selected along the DV?

3. In what ways do our understanding of cases informwhat constitutes a selection bias?


10/13

III. Dealing with Selection Bias

Addressing the selection bias problem in quantitative and qualitativeresearch

Cross-case The simple relationship that seems to exist when the analyst examines only the most

rapidly growing countries disappears when a more representative sample is examined(Geddes, p.140).

It is unrealistic to expect qualitative researchers, in their effort to avoid selection bias, tomake comparisons across contexts that may reasonably be thought to encompassheterogeneous causal relations. Given the tools that they have for causal inference, it may bemore appropriate for them to focus on a more homogenous set of cases, even at the cost ofnarrowing the comparison in a way that may introduce problems of selection bias (Collierand Mahoney, pp. 68-69).

Single case

Given the finding that the regression slope for the truncated sample underestimates themain relationship, Collier et. al. find that for within-case analysis, selection bias is not aproblembecause within-case analysis does not involve intuitive regression (Collier et. al.,p. 95)

Crucial leverage in testing explanations comes from within-case analysis, and this leverageis valuable irrespective of whether these cases are embedded in a full-variance design or ano-variance design (Collier et. al., p.101).


11/13

III. Dealing with Selection Bias

Techniques from Small

Nonrandom sampling Sampling for a range Snow-ball sampling

Sequential interviewing Sampling logic vs. case study logic Sampling logic is superior when asking descriptive questions about a

population; case study logic is probably more effective when asking howor why questions about unknown processes (Small, p.25).

Extended case method Whether Bill selected an ostensibly typical or an apparently atypicalcase would be immaterial, since the hypothesis would be based on a)logical inference and b) the internal conditions of the case (Small, p.24).


12/13

III. Questions

Some of these techniques to deal with selection bias leadto significant changes in many aspects of researchdesign. Do these techniques also have implications onepistemology?

How can altering our conceptions of terms other thancase (e.g., cause, control, bias) affect certain aspects ofa research design?

Do these strategies/techniques for avoiding selectionbias assume or require deductive or inductive reasoning?


13/13

Works Cited

Collier, David and James Mahoney. 1996. Insights and Pitfalls: SelectionBias in Qualitative Research. World Politics 49(1): 56-91.

Collier, David, James Mahoney, and Jason Seawright. 2004. Claiming TooMuch: Warnings about Selection Bias. Chapter 6 inRethinking Social

Inquiry: Diverse Tools, Shared Standards. Rowman and Littlefield

Publishers.

Geddes, Barbara. 1990. How the Cases You Choose Affect the Answers YouGet: Selection Bias in Comparative Politics.Political Analysis 2: 131150.

Ragin, Charles. 1992. Introduction: Cases of What is a Case? Chapter 1 in

What is a Case?Cambridge University Press.

Small, Mario L. 2009. How Many Cases Do I Need? On Science and theLogic of Case Selection in Field Based Research.Ethnography 10(1): 538.

case selection and sampling in social science methodology

Documents