rapid mobile phone based surveys (scott chaplowe, ifrc)

www.ifrc.orgSaving lives, changing minds.

Rapid Mobile Phone-based Surveys (RAMP) for Evidence-based Emergency Response

ALNAP 28th Annual Meeting,5-7 March 2013, Washington, D.C.

Scott Chaplowe, Senior M&E Officer, IFRCRose Donna, Director, Datadyne.org

Jason Peat, Senior Officer Public Health, IFRCAmanda Mcclelland, Emergency Health Officer, IFRCJoel Selanikio, CEO DataDyne GroupMac Otten


Presentation Overview Application of mobile technology (RAMP) to address specific challenges in data collection during emergency operations.

1) Introduce RAMP

2) How RAMP works

3) Emergency contexts

4) Key considerations


What is RAMP?

RAMP (Rapid Mobile Phone-based Surveys) is a survey methodology utilizing mobile phones to help RCRC National Societies, governments, NGOs and other partners efficiently conduct quality surveys that:

Reduced time

Reduced cost

Improved quality assurance

Limited external technical assistance


RAMP Background (www.ifrc.org/ramp)

1. Developed by IFRC in partnership with WHO, CDC, and other partners.

2. Initial focus = malaria program household surveys Four pilots in Africa 2011-2012 (Kenya, Namibia and Nigeria),

3. Refine and developed trio of user guides: 1. Designing a RAMP survey 2. Implementing a RAMP survey3. Training a RAMP survey team

4. Scale-up to other program areas – increase survey functionality – use of SMS

http://www.ifrc.org/ramp


RAMP takes advantage of 2 technologies1. Mobile phone to collect data(Low-cost, standard mobile phones, as well as Android, Symbian, Blackberry, SMS, and iPhone)

2. Web-based software application Enables mobile phones to become a data collection platform


How does RAMP work?

5. Data Reports

2. Data collection on phone

1. Develop survey on website

3. Transmit data

4. Collate/analyze data on computer


Connectivity

InternetRequired

• Create/edit surveys

• View/export data

• Create reports

InternetNot Required• Collect data

Can be cellular, wifi, cable


Data monitoring and analysis

Preliminary analysis available before data collection is complete


Survey bulletins/updates Full survey reports

Timely Reporting


Digital Data Collection – Changing the way we work

Paper questionnaires filled out in the field

Data entered into a computer at a central location

Data analysis and reporting often takes months to complete

Local capacity is often under-utilized and there is a dependence on external experts

Mobile and internet-based technologies used to reduce time for data collection to reporting

Enables rapid reporting of results, decision making, and action

Empowers local ownership of evaluation and research

The “old” The “new”


Vaccination coverage

Surveillance

Supply chain management

Household surveys

Clinic surveys

Supervisory checklists

Anything that can be put on a form


RAMP Potential in Emergencies?

Beginning to explore the potential of RAMP in emergency context: Site assessment – needs, damage

Community assessment – needs, damage

Beneficiary registration

Distribution of emergency (and non-emergency) items

Baseline/endline data collection (monitoring and impact study)

Repeated surveys to track time trends for key indicators

Beneficiary communication – (broadcast Terra)

Beneficiary/community monitoring

Disaster preparedness – EWS monitoring


SMS Disease Surveillance Systems Piloting in community based disease surveillance

Sierra Leone – 400 community volunteers distributing ORS.

Referred only 5% of cases of AWD they saw in community = only 5% of cases were potentially recorded in normal MoH system.

RAMP allows real time communication and data gathering suitable for this context.

Problems with integration and harmonization of data between community and MoH.

But SMS proved real time information to assist program prioritization in outbreak scenarios.


SMS Considerations Simplified questions rather than full surveys

Coding syntax with 2 to 7 key variables as best practice

Quantity of messages handled depend on networks, and whether staggered or simultaneous reporting.

Paper form can be used to facilitate data entry to SMS

Quality assurance auto feedback

Reminder SMS to field person to report data at a set time

Thank you SMS to confirm receipt of data.

Ability to send airtime to the mobile account if someone reports from a common central account.

amanda.mcclelland

you cant do any of the last three points with RAMP


Benefits?


Benefits – decision making

Data rapidly available for decision-making

Maintain data control

Scalable for studies of varying sizes

Shared, electronic database to compare across contexts and with partners to build a body of evidence related to impact


Benefits - management Cost effective

Do not have to reinvent the wheel – Adaptable RAMP toolkit

Consultants not required

No software licensing or subscriptions

Multiple languages (depending on program)

Export data for custom analysis using any statistical analysis package

Additional SMART phone features


Benefits - management

Online library of survey forms

Collect and aggregate data form multiple areas and partners

Ease of creating and changing analyses/reports

Efficient reporting and dissemination


Benefits - Fieldworkers

• Build local capacity for M&E

• Standard and familiar mobile phones

• No more paper to collect, transport or return

• Automated data submission (assuming network)


Benefits - Quality Assurance

Remote QA: Enables monitoring of survey team work rate, productivity and quality Monitor times/location of data collection (time/date data stamps) Provide feedback remotely

Efficient data management reduces “paper” mistakes Easier to back-up forms/data Reduced error of repetitive data entry and re-entry Easier to change and update forms

Immediate QA: Real-time error analysis and field

correction Utilize skip patterns, custom logic and

validation


Reality Check!

Not suitable for very long questionnaires

No “magic bullet” –work is still in the details!

Things to improve – i.e. offline form generation

Technology is a moving target – (hardware and software)

Challenges resource development/training

(But also means improvements and reduced costs)


Questions to Consider

What applications do you see for mobile data collection in the humanitarian sector?

What has worked well?

What hasn’t worked well?


www.ifrc.org/ramp

Package of field-friendly User Guides:1. Volume 1: Designing a RAMP survey: technical considerations2. Volume 2: Implementing a RAMP survey: practical field guide3. Volume 3: Training a RAMP survey team: guide for trainers

Living archive of additional resources: Example database and STATA files for data cleaning and analysis of a

sample malaria survey Latest up-to-date malaria questionnaires and STATA files for data

cleaning and analysis Country reports and results bulletins, information, useful links

http://www.ifrc.org/ramp


The following slides are extra and can be referred to if needed, (but unlikely).


Cost of a IFRC RAMP HH survey for Malaria programming (average)

Description Cost (US $)

Training (4 or 5 days) including two facilitators 10,623

Field survey, including transportation, daily allowances and accommodation 12,415

Mobile phones, accessories and air time 3,806

Survey administration 2,243

Total in-country expenditure (US $) US $ 29,087


When might a RAMP survey be suitable? FlexibilityItems that can be adjusted Comments

Adjust precision ±10%, 5%, 3%, etc.

Adjust indicator type (denominator of indicator) including mixtures of indicator types

- Person all ages- Children <5 years old- Pregnant women- Households- Schools

Adjust number of domains - 1 domain with 30 clusters- 2 domains with 30 clusters to

compare statistically - 10 domains with 30 clusters each to

compareAdjust overall sample size - 200 to 5000 households


How is the RAMP method different from MIS/DHS?MIS/DHS RAMP

Complex design, uses external consultants to design survey

Simple design, external consultants not needed

Listing of all households is done in all clusters; expensive, often taking several days in each cluster

Divides clusters into manageable-sized segments using standard survey methods; takes <1/2 day

Simple random sampling of households (from the cluster list)

Simple random sampling of households (from the final segment list)

Real-time data cleaning not possible Real-time data cleaning during the survey

Real-time data analysis not possible; results take several months

Real-time data analysis and results/draft report finished within 3 days of last interview

Data analysis done by third-party consultants

Organization performs analysis, building capacity and maintaining control of data


Mobile application

Record field data, even without network coverage


RAMP

Based on standard survey sampling methodology

Web-based platform for survey design, data storage, analysis, reporting and data export.

Field-based data entry through mobile phone application.

Questionnaires downloaded to standard mobile phones

Web-based dataset that can export “real-time” for rapid analysis and reporting

rapid mobile phone based surveys (scott chaplowe, ifrc)

Documents

orgsaving lives

data viewexport data

data monitoring

data entry

data gatheringsuitable

data reportsdata

receipt of data

data collection platformwww