2016 aapor (not presented) rahija mtui aapor
TRANSCRIPT
Report on Tablet Assisted Personal Interview (TAPI) Implementation by the Ministry of Livestock and Fisheries Development (MLFD) in Tanzania
Michael Rahija, Research Officer, GSARS Dr. Niwael Mtui, Principle Veterinarian, MLFD
1
Overview
• Background: Study and justification of using
TAPI
• Survey Implementation
• Monitoring reports and communication with
field staff
• Overview of costs
• Challenges encountered in the field
• Lessons Learned and Conclusion
2
Background
3
Background – Study Rationale
FAO, the Tanzania Ministry of Agriculture,
Livestock and Fisheries and the Tanzania National
Bureau of Statistics have been collaborating since
2011 to:
- Improve the livestock component of the
agricultural statistics system
- Facilitate the use of official statistics / statistics
methods for formulating / implementing
evidence-based policies
4
Background – Study Rationale
- Tanzania currently has one of the largest datasets
on livestock at household (farm) level throughout
Africa via the National Panel Survey)
- Ministry of Agriculture analysed the data and found
that many farmers relied on livestock, but did not
access extension services.
- Accordingly, a policy increasing access to
extension services would benefit farmers and
enhance food security.
5
Background – Study Rationale
MALF agreed to invest resources to improve the
system of livestock extension.
HOW?
6
Background – Study Rationale
MALF agreed to invest resources to improve the
system of livestock extension.
1. A survey of extension officers (change agents)
2. An experiment to test alternative options to
improve their on the job performance
7
Background – Study Rationale
1. A survey of extension officers (change agents)
2. An experiment to test alternative options to
improve their on the job performance (Jan 16)
TAPI
8
• Timelineness and cost
• Data Quality
• Capacity development
• Choice of Survey Solutions
Background – TAPI Rationale
9
Implementation
10
• Headquarters at MALF and PI from
London School of Economics (LSE)
–Survey design - Translation
–Survey management
• Enumerators and field staff
–Basic tablet operations
–Questionnaire training directly on tablet
–Use of quizzes
Implementation - training
11
• 5 teams of 3 enumerators and 1 supervisor
• Allocation of enumerators – Based on quiz performance
– Tech savviness
• Assignment of Field Supervisors
– Based on understanding of questionnaire
– Tech savviness
• Headquarters – MALF, Global Strategy, LSE
Implementation – Teams
12
• Target population: Village Ward livestock
officers, and District Livestock Officers
• Goal: complete enumeration in Morogoro,
Iringa, and Dodoma
• Final sample: 63 DLOs and 415 VLOs
Implementation – Sample
13
• Questionnaire (~150 questions for VLOs): – Personal info
– Livestock service provision and transport
– Farmer interaction
– Hierarchy
– Other income sources
– Personality measures
– Knowledge of rule, regulations, and policies
– Incentives
– The game
Implementation – Q
14
Implementation – Field work
Interview locations
15
Implementation – Field work
Progress by day
16
Implementation – Field work
Duration
17
Monitoring
& Communication
18
• Daily export of paradata
• Rmarkdown for automated tabulations – Interviews by team and enumerator
– Duration of interviews
– Average interviews per enumerator
– Assignments
• Daily export of microdata and automated
checks
Monitoring
19
• After report generation, mgmt at the MALF
called Supervisors
• HQ called tablet of enumerators re:
specific interviews
• Comments inside of questionnaire used to
alert enumerators of various problems
Communication
20
• Information management:
– Email, 2 sim cards, IDs through CAPI software
Communication
internal.id devicename email interviewname enumerator AIRTEL card tigo MinistryLF1 MinistryLF1 [email protected] lss_inter1 ### 0687333122 0675343970 MinistryLF2 MinistryLF2 [email protected] lss_inter2 ### 0687332727 0675343971 MinistryLF3 MinistryLF3 [email protected] lss_inter3 ### 0687327582 0675433985
21
Data analysis
• September 2015 data
cleaning (limited)
• October 2015: MALF
report available
• Nov 9th 2015: report
presented to the
Permanent Secretary
22
Overview
of Costs
23
Overview of costs of using TAPI
Item Units Total (TZS) Total (USD 17/9/15)
Samsung Galaxy 4 22 20,592,000 9,950.00
Battery chargers 22 50,000 512.00
Tablet covers 22 20,000 204.00
1 month data Tigo 22 25,000 256
1 month data Airtel 22 25,000 256
Server (Amazon – cloud) 0 0 0
Programming cost 0 0 0
Total - 23,232,000 10,820
24
Challenges encountered
in the field
25
• In some remote areas, it was difficult to
find a signal.
• Solution 1: Enumerators waited to
synchronize until a signal was found.
• Solution 2: Back-up paper questionnares
were provided.
Challenge - Connectivity
26
• Seemingly at random, SIM cards would
become de-configured and the tablets
wouldn’t synchronize.
• Solution 1: The second SIM card was
used.
• Solution 2: Field Supervisors were trained
and provided instructions for manual re-
configuration.
Challenge – De-configuration
27
• During the first week of data collection,
enumerators recommended some
improvements to questionnaire.
• Solution 1: HQ team made changes and
conducted trouble-shooting at night. Then
field supervisors were instructed to tell
enumerators to re-synchronize. Not
Recommended!
Challenge – Questionnaire
28
• The initial delivery of tablets was missing 6
tablets.
• Solution 1: Anyway, four tablets were
meant to be back-ups.
• Solution 2: Paper questionnaires provided
to some enumerators
• Solution 3: NBS loaned the project 2
tablets.
Challenge – Procurement
29
Lessons Learned,
Software recommendations,
and Conclusion
30
• Cost – justified in terms of data quality,
time b/t collection and analysis, and cost.
• Monitoring – Daily monitoring reports were
valuable to enhance team performance.
• Connectivity – Giving enumerators 2 SIM
cards minimized these problems.
• Training – Supervisors should be trained
on basic tablet maintenance.
Lessons Learned
31
• Training – One should train enumerators
on the questionnaire directly in the tablet.
• Training – Survey Solutions can be used
by people with limited or no programming
experience.
Lessons Learned
32
• Offline survey designer
• Feature to toggle between languages
• Automated generation of PAPI
questionnaire needs to be greatly
improved
• More documentation on enabling and
validation conditions.
Recommendations
33
• MALF consider implementation of CAPI to
be a success.
• Preliminary results were made available
and circulated within 1.5 months.
• The expected benefits of using Survey
Solutions were realized.
• Experience to be replicated / improved
through other FAO’s policy-assistance
initiatives
Conclusion
34
Thank You • For more questions,
email:
• Michael Rahija
• Dr. Niwael Mtui
35