Multiple Indicator Cluster SurveysSurvey Design Workshop

Sampling: Overview

MICS Survey Design Workshop

Introduction • MICS :

–Household survey program implemented across various countries and at multiple point in time within a country

–Data are consistent and comparable across countries and over time

Contents • Importance of a correct sample design• Major steps in designing MICS sample• Key principles of MICS sample• Sampling options• Sampling tools available for countries • Group activity: sample size calculation

Sample Design

• Sample design involves determining:– Sample size: number of units of analysis to be

selected for the survey– Sampling structure: how those units are to be

selected– Estimation procedures: how the results from the

sample are to be used to draw inferences about the entire population of interest from which the sample was selected

Importance of a correct sample design

• Sample size and structure affect:–Validity of inferences about the entire

population–Magnitude of:

• Sampling error•Non sampling error

Importance of a correct sample design

• Sampling error: due to sampling of a small number of units from the population instead of complete enumeration

• Non sampling error: due to problems during data collection and data processing (e.g., failure to locate and interview the correct sample household, misunderstanding of the questions, data entry errors)

Importance of a correct sample design

• Link with other aspects of the survey– Dispersion of sample affects travel cost and

time– Sample size affects the number of teams,

interviewer workload, and cost of household listing and interviewing

– Sample size affects timeliness of results– Large sample size can affect data quality

Major steps in designing MICS sample

• Define objectives

• Identify a suitable pre-existing sampling frame

• Determine sample size and allocation

Major steps in designing MICS sample

• Define objectives: –Key indicators–Reporting domains–Desired level of precision for survey

results Critical for sample size determination

Major steps in designing MICS sample• Identify a suitable pre-existing sampling

frame:–Most recent census of population and

housing–Master sample or sample for another

survey conducted recently which is large enough to support the MICS sample design

Major steps in designing MICS sample

• Identify a suitable pre-existing sampling frame:–The availability of a suitable sampling

frame is a major determinant of the feasibility of conducting a MICS survey.

–This issue should be addressed in the earliest stages of planning for a survey.

Major steps in designing MICS sample

• Identify a suitable pre-existing sampling frame:– Regardless of source, evaluate the quality of the

frame before drawing the sample• Characteristics of a good sampling frame

– Complete coverage of the target population – No duplicates– Up to date

Major steps in designing MICS sample

• Characteristics of a good sampling frame – Area units: Boundaries well defined and

good maps are available– Identification codes – Measure of size (household or population)– Auxiliary information available for

stratification

Major steps in designing MICS sample

• Determine sample size and allocation– Survey objectives (key indicators, desired

level of precision and need for sub-national results)

– Sampling parameters from previous MICS or DHS (e.g., response rates, design effect)

– Survey budget and resource constraints– Distribution of target population

Key principles in MICS sampling

• Probability sample at every stage of selection (units are selected randomly with known and nonzero probabilities)

• Latest census as sampling frame when available

• Adequate sample size

Key principles in MICS sampling

• Simple design • Sampling in two or three stages • Separate household listing• Clusters of moderate size: 20-25

households• No replacement of primary sampling

units or households

Key principles in MICS sampling

• Implement the sample exactly as designed

• Proper sampling weights– Extrapolate survey results to the population– Used in all analyses to prevent biased results– Calculation depends on the exact sample design– Weights: households, women, men and children

Key principles in MICS sampling

• Sampling error calculation– Possible only when probability sampling is used

• Good sample documentation

Key principles in MICS sampling

• Report on sample design describes:– Sampling frame– Sampling methodology– Sample size calculation and sample

allocation– Survey domains and stratification– Probabilities of selection at each stage

MICS Sampling Option 1 – new sample with household listing

• Design new MICS sample • Two stages with census as frame• Selection of census EAs with PPS at

first stage• Carry out household listing in

selected EAs/segments

MICS Sampling Option 1 – new sample with household listing• Select households systematically

from listing• Interview selected households, no

replacement will be allowed

Sampling Option 1 - continued

• Advantages of option 1- simple design- probability-based

Sampling Option 1 - continued

• Limitations of option 1- expense of listing households- time necessary to list households

[Example, sample size of 5000 households may require 25000 to 50000 households to be listed]

MICS Sampling Option 2 – use an existing sample

• Design MICS as a rider to another survey if timely and feasible

• Use sample from a previous survey and re-interview households for MICS

• Use old survey sample EAs and construct new listing of households to select for MICS

MICS Sampling Option 2 – use an existing sample

• Old sample must be probability-based, national in scope

• Possibilities – DHS, other national health survey, recent labour force survey

• Important: design parameters must be known (such as selection probability, stratification, etc.)

Sampling option 2 - continued

• Use of existing master sampling frame• Some countries use master sample design for

intercensal national household surveys• Master samples generally sufficiently large for

MICS; subsample of PSUs can be selected• Advantage – updated maps may be available

for master sample of PSUs, and perhaps updated listing

Sampling option 2 - continued

• Advantages of using previous sample- cost savings- maps available for interviewers- appropriate sampling plan available- simplicity

Sampling option 2 - continued• Limitations of using old sample

- burden on respondents- sample design may need modification

* sample size* sub-national coverage* number of PSUs or clusters

• Balance between loss and gain

Sampling strategy for low fertility countries

• In MICS 4 and 5, some low fertility countries are using second-stage stratification of listing by households with and without children under 5

• Higher sampling rate used for households with children

• Increases number of households with children in MICS sample, and therefore number of sample children

Sampling strategy for low fertility countries (continued)

• Improves the reliability of the child indicators without increasing the sample size to a very high level

• This procedure also increases the variability in the weights and the design effects for the overall sample

• Important to avoid very large variability in the weights for households with and without children– Differential weights between households with and without

children generally should not exceed a factor of about 4

MICS Sampling Tools

• Household listing manual and listing forms

• Template for sample size calculation

• Template for calculation of weights

• Template for household selection

• SPSS program for sampling error estimation

SAMPLE SIZE DETERMINATION

Selection of key indicators

• Choose an important indicator that will yield the largest sample size

• Step 1: Select 2 or 3 target populations representing each a small percentage of the total population (pb); typically – Children 12-23 months: 2-4% or – Children under 5 years: 7%-20%

Selection of key indicators

• Step 2: Review important indicators for these target groups but ignore indicators with very low or very high prevalence (less 10% or over 40%, respectively)

• Do not choose from the desirably low coverage indicators an indicator that is already acceptably low

• Do no choose childhood and maternal mortality ratios

• n is the required sample size (number of households)

• 4 is a factor to achieve the 95 percent level of confidence

• r is the predicted or estimated value of the indicator in target population

• deff is the design effect

• RR is the response rate• pb is the proportion of the target

subpopulation in total population (upon which the indicator, r, is based)

• AveSize is the average household size (that is, average number of persons per household)

• e is the margin of error to be tolerated at the 95% level of confidence

• Currently, note that e = 0.12r [defined as 12% of r, in this case the relative standard error of r is 6% because e = 2 standard error (r)]

Previously in MICS2

• 2 different values for margin of error – Margin of error was 5 percentage points for high

values of r (over 25%)– Margin of error was 3 percentage points for low

values of r (25% or less) • Difficulty for users in deciding on the sample

size for their surveys.

MICS template for sample size calculation - EXCEL FILE