General overview of operations and implementation research

Emily A. Bobrow, PhD, MPHSenior Research Officer

IAS Rome20 July 2011

Overview

• Definitions

– Research

– Monitoring and Evaluation (M&E) and Quality Improvement (QI) vs. Research

– Operations Research (OR) & Implementation Research (IR)

• Examples of OR/IR studies

• Criteria for prioritizing OR/IR questions

• OR/IR common study designs

• Research ethics

Technical definition of research

A systematic process of discovering new knowledge, involving application of the scientific method to make generalizable statements based upon specific inquiries

Making a Distinction: Research vs. M&E/QI

Research M&E/QI

Purpose Test or generate a hypothesis Assess a program/process

Starting Point Prospectively designed, formal written hypothesis

An established set of standards

Risks/Burdens May put subjects at risk No risk, with the exception of confidentiality concerns

End point Answer a research question Improve the program/process

Analysis Determine validity of hypothesis

Compare the program/process to established guidelines

Intended result Conclusions may be generalized beyond a specific program/process, contribute to overall scientific literature

Conclusions apply only to specific program/process, and shared with those related to program/process

Research and service provision feed each other!

Using M&E Data for Research Purposes

• M&E data is used to generate research questions

• Routine data to conduct a research study

• For application for research funding

• For advocacy to Ministry of Health and implementing partners

• Using research findings to inform M&E strategy

OR/IR definitions and audience

• Definitions– Operations research: • Focus on operational issues of specific health

programs

– Implementation research:• Focus on strategies for specific products or

services

Remme JHF, Adam T, Becerra-Posada F, D’Arcangues C, Devlin M, et al. (2010) Defining Research to Improve Health Systems. PLoS Med 7(11): e1001000. doi:10.1371/journal.pmed.1001000

Definitions of OR/IRPrimary CharacteristicsFocus of the research

Users of the Research Outputs

Utility of the Research Outputs

Research DomainOperational/Operations

Operational issues of specific healthprograms

Health care providers and programmanagers

Local

Implementation Implementation strategies for specificproducts or services

Program managers, R&D managers

Local/broad

Health System Issues affecting some or all of thebuilding blocks of a health system

Health system managers, policy makers

Broad

Examples of OR and IR questions

OR question: • Which locations should be targeted for delivering HIV

prevention services in Kabale district, Uganda? • Remember the definition of OR:

– Focus on the operational issues of specific health programs

IR question• How do we effectively implement option A to HIV-

exposed infants through ANC clinics in Western Kenya?• Remember the definition of IR:

– Focus on strategies for specific products or services

Terminology

• Operations Research = Operational Research = Programmatic Research = Implementation Research = Implementation Science = Translational Research

How is OR/IR different from other types of research?

• All types of health research try to improve health

• All types of research can use similar methodologies– Quantitative, qualitative, observational studies,

experimental studies

• Difference = focus and goal– Focus within the routine health system– Goal is to improve programs and to inform policy

Intervention: TBA Log in MalawiDrawings used for illiterate TBAs

Has started pushing at her home

The mother’s condition is good

The mother has taken NVP

The mother is on ARVs

The mother has died

The baby’s condition is good

The baby has taken NVP

The baby has died

Has been referred to hospital

Delivery during the day

Delivery at night

DATE

NAME OF CLIENT PMTCT#

NAME OF TBA

_______________

MONTH

Hosseinipour MC et al. Improving uptake of PMTCT services in Lilongwe, Malawi. EGPAF OR Think Tank, Oct 2009

The Early Infant Diagnosis (EID) Cascade in Tanzania, Q4 2009

Infants tested results back pos PCR pos. results parents

referred enrolled treated started ART0

200

400

600

800

1000

1200

#

W. Schimana, G. Kasori, J. Songoro, G. Mbita, A. Mwangomale, C. Kimario, J. Van’t Pad Bosch. Systematic analysis of the early infant diagnosis (EID) cascade and implementation of specific interventions: Lessons from a program to increase identification and treatment of infants with HIV in Tanzania. IAS Rome 2011.

Information - Intervention

• WHY?• Long turnaround time due to:

- Inconsistent sample transportation- Inconsistent result collection and distribution- Under staffing of the lab

- Lab support- Outsourcing of transport- Linking EID to loss to follow-up

(LTFU)

Change due to introduction of courier service

Q4 09 Q1 10 Q2 10 Q3 100

200

400

600

800

1000

1200

1400

1600

DBS takenresult back

Courier service introduced

The EID Cascade Q4 2009

pos PCR pos. results parents

referred enrolled treated started ART0

10

20

30

40

50

60

Turnaround time reduced through courier service and lab support

Staff sensitization and introduction of LTFU tracking system

Q4 09 Q1 10 Q2 10 Q3 10 Q4 100

20

40

60

80

100

120

140

pos. DBSParents informed

#

The EID Cascade Q4 2009

pos PCR pos. results parents

referred enrolled started ART0

10

20

30

40

50

60

%

Q4 09 Q1 10 Q2 10 Q3 100

10

20

30

40

50

60

70

80

90

infants enrolledinfantes initiated on ART%

Clinical mentoring

Confidence building of

mentors

Initiation of HIV-infected infants on ART

Existing national PMTCT program based on sdNVP

Draft scale up plans developed(Jan ’08)

Formation of national working group(Dec ’07)

Collaborative operational research on changing ARV prophylaxis regimen in PMTCT (June ’06-Dec ’07)

Resource mobilisation(Feb – May ’08)

Adaptation of national tools(July ’08)

WHO guidelines, August 2006

Regional exchange visit (May 2008)

Directive from MOHCW leadershipScale up (July ‘08)

National, Provincial and District sensitisation and planning on prelim results (May – onwards ’08)

Results from operational

research

Keatinge J et al. Using operational research to influence policy and programme developments: lessons learnt from Zimbabwe. EGPAF OR Think Tank, Oct 2009

Criteria for prioritizing OR/IR questions

• Relevance• Avoidance of duplication• Timeliness (urgency of need for data)• Political acceptability• Feasibility• Applicability of possible results or

recommendations• Ethically acceptable

Intervention

Group

Result

Result

Comparison Group

compare

Study Population

Common OR/IR designQuasi-experimental

NO Random Assignment

Another common OR/IR designPre-post test design

Time

Pre-test Post-test

Study Population

Intervention

Compare

Important to remember . . .

• There is no best study design

• Match the design to your research question • Then consider . . .

– Feasibility• Time• Money• Other resources

• Make sure you have a multi-disciplinary team assembled at the start of your planning

Also important . . . Research Ethics

Definition of Human Subjects Research • “Human subject” are individuals whose physiologic or

behavioral characteristics and responses are the object of study in research. Under the federal regulations, human subjects are defines as: living individuals(s) about whom an investigator conducting research obtains:– Data through intervention or interaction with the individual, or– Identifiable private information

Federal Policy, 45 CFR 46.102

• Human subjects research requires approval from an Institutional Review Board (IRB) or Research Ethics Committee (REC/RC)

Thank you!