1

Fieldwork logistics and data quality control procedures

Kathleen BeegleWorkshop 17, Session 2Designing and Implementing Household SurveysMarch 31, 2009

2

What makes a good survey? Relevance

Answer the policy questions of import Quality

Methodology Accuracy and reliability Adherence to international statistical guidelines

Timeliness Punctuality Production time Comparability over time

Sustainability Public dissemination Documentation

3

Overview Overview of the main actors Quick description of broad activities of an

LSMS Options for organization of field work &

quality control aspects

4

Who does what Core team

Principal Investigators who understand the purpose of the survey

Local team Will implement the survey and process data For LSMS is usually the national statistics office For smaller/specialized surveys, usually local firms Need to identify who in the local team is the project

manager Technical assistance

Short-term consultants for specialized expertise such as sample design and data entry system development

Long-term assistance: possible project head if lacking in local team

5

Activities Broad activities involved with an LSMS

Management/logistics Questionnaire development Sampling Field staff: recruitment, training, payments Field work Data management Data analysis and final documentation

Some activities are sequential (training after questionnaire finalization), but lots of overlap in timing (sampling & questionnaire development; logistics is on-going).

6

Activity groups 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

1 Management and logistics2 Questionnaire development3 Sampling4 Staffing and training5 Data management6 Field work7 Data analysis and documentation

Design

Data collection

Analysis

Management: Prepare a timeline…

7

….and make it detailed, with deadlines!

4 Staffing and training 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

4.01 Select and train field test staff4.02 Prepare supervision procedures & manuals4.03 Prepare interviewer manual4.04 Select interviewers4.05 Train interviewers

1 Management and logistics 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

1.01 Agreements and institutional environment1.02 Appoint core staff1.03 Core staff logistics1.04 Acquire anthropometric equipment1.05 Acquire computers1.06 Acquire survey materials1.07 Mobility startegy and vehicle acquisition1.08 Publicity and hh motivation strategy

8

Questionnaire development

Pilot/pre-test questionnaires in various settings as relevant (urban, rural, etc)

Careful review of translation (if any). Back translation is ideal.

Finalize questionnaire before any training. Prepare Manuals with specific instructions for

enumerators and supervisors, other staff, to accompany the questionnaire. This is also part of the final documentation to assist data users.

9

Field staff

Recruit & train more enumerators than you need Enables a selection process during training –

competition among candidates Provides reserves in case field staff leave early

Training in two parts (if large field staff) Training of Trainers (TOT): train field supervisors Training of enumerators: main training with active

participation of supervisors as facilitators

10

Field staff Training for an LSMS – a long, multi-topic

questionnaire – 3-4 weeks. Includes: in-class study of questionnaires in-class mock interviews/exercises tests field practice and feedback more if the survey will use special instruments

(anthropometric, GPS, etc.) Common training for all team members if feasible

Payments Usually not paid per questionnaire, instead paid daily rate Bonuses paid at end of field work can reduce staff attrition

during field work (important for longer surveys) Performance bonuses (if they can be objectively determined)

11

Field work There are many options for field work structure.

Your selection will depend on the specifics of the survey, timeline, and the budget: Mobile teams with integrated DE. Improves quality of data

because teams get immediate feedback on inconsistencies in the questionnaire.

Mobile teams without DE. DE in office. Enumerator in the village. DE in office. Difficult to supervise

well! May not be cost effective (with exception of surveys over several moths with ~20 HHs per village).

Local office from which teams operate. Feasible for a survey focused in one region/small geographic area.

No excuse not to do concurrent DE! Don’t wait until end of field work to start entering questionnaires. This is often the reason for lags in data availability.

Field work: Composition of a mobile field team

Supervisor Interviewers Anthropo-

metrist

Data entry

operator.maybe.

13

14

15

Other factors of survey quality Management Team for the survey

Composed of Project Manager Fieldwork Manager Data Manger

Dedicated to the project in all three phases Design and Preparation Implementation Dataset documentation and initial tabulations

16

Other factors of survey quality Actions of the field team supervisors

Visual scrutiny of completed questionnaires Less critical if DE is integrated with field work.

Computers do it better Visual observation (direct observation) of interviews

Some of this is needed, but not too much Check-up visits

Critical, and can only be implemented by human supervisors

Need to be random

Supervisors must be supervised too!!

17

Other factors of survey quality Review data files as they come in – done by

Core staff (i.e. in Stata, look for inconsistencies across modules of the questionnaire which is normally not done in DE programs, compare with baseline data if a panel, etc…)

Send Addendum notes/instructions to field staff as issues arise (for things missing from the Manuals)

Keep detailed documentation, needed for analysis and relevant for panel surveys

18

Aspects for evaluation work If the evaluation entails panel data (baseline and follow-up

surveys), plan surveys accordingly. At baseline, think about how to minimize attrition and match

people correctly get adequate address/map/phone information to locate

households again Record complete names of all household members on

questionnaire and enter in DE Other possibilities: GPS, photos of respondents

At follow-up pre-print some baseline data (Roster: names, age, sex, roster

IDs) and also baseline information relevant to the evaluation for measuring changes (assets?)

collect some minimal information on respondents/households who drop out or move (in case you are unable to find them for a re-interview). For example: did they move, when, why…