Mixed methods in medical research

Robert PoolCentre for International Health, Hospital Clinic, University of Barcelona

London School of Hygiene & Tropical Medicine

THE BACKGROUND

Social science & medicine

• Increasing demand for qualitative social science in medical research programmes due to:

– Realisation that illness is social as well as biological (adherence, acceptability, treatment seeking); the need to understand beliefs in order to change behaviour, fashionable

– Better interdisciplinary communication

• Consultation, collaboration, integration

• Cooperation depends on suppressing/ignoring underlying differences

Why mix methods?

1. To triangulate (get closer to the truth)

2. To develop (sequential use)

3. To complement (examine different aspects)

4. To broaden (discover wider context, new perspectives)

THE PROJECT

Microbicides Development Programme

Objective: To determine the efficacy and safety of two concentrations of “PRO 2000/5 Gel” compared to placebo in preventing vaginally acquired HIV infection

Primary outcome: acquisition of HIV infection

Sample: 10,000 women

Process: Feasibility Pilot Phase III

Study sites

Quantitative trial data

• Women followed up for 12 months

• Regular clinic visits (every 4 weeks) for:– condom & gel distribution– applicator returns– clinical exams– lab tests (STDs, HIV & pregnancy)– clinical & behavioural interviews (CRF)

Issues

Interpreting the trial result depends on the accuracy of sexual behaviour and adherence data

But self-reporting is unreliable

Key questions

• How do you know that participants understand the categories and questions in the way you intend?

• How do you ask sexual behaviour questions across different cultures & vernaculars in a standardised way that enables meaningful comparison?

• How do you know that your sexual behaviour data are accurate?

• How do you know whether women have used the product (properly)?

The social science

• Qualitative social science integrated to address these issues

• Random subsample (100 women at each site)• In-depth interviews with women & partners• Coital diaries• Focus group discussions• Ethnography

Explore & clarify standard categories & questions

• Almost all the central concepts in sexual behaviour research are ambiguous: – marriage, regular partner, casual partner– sex acts, condom use, etc.

• Get emic meanings

THE TRIANGULATION PROCESS

Women

Case Record Form

Coital Diary4 weeks before CRF

Comparison Form

1. Interview2. Comparison &

probing inconsistencies

IDI T&T Nvivo

Summaries

Summaries

Nvivo

T&TIDI

1. Interview2. Compariso

n with women

Male partners

Women, community

Summaries

T&T NvivoFGDs

In-depth interview guide

Consistency of CRF data

• In 53% (254/482) some inconsistency

• Mainly under-reporting of sex acts and over-reporting of gel and condom use on CRF

sex acts gel condom

IDI

CRF

Two main sets of reasons for the inaccuracies in the CRF:– The participant forgetting or not understanding. – The interviewer not asking, not listening, or not

understanding

• Confirmed by recording of CRF interviews

• Training, memory aid, short recall periods, clarification of terminology

Getting closer to the truth

• You can get more accurate data on sexual behaviour (get closer to the truth) if you:

• Clarify key concepts & categories

• Need to identify and solve inconsistencies during the study

• Ensure continuous feedback between different teams, methods and data sets

• Engage in dialogue with participants

THE ISSUES

1. How do you deal with inaccuracies once you’ve found them?

• Unresolved inconsistencies (uncertainty)?

• Resolved inconsistencies: – change the database?– extrapolate?

2. What do you do with key categories that remain ambiguous?

Sex acts/rounds/days

3. How do you take into account the influence of the research process on the data?

• On behaviour (condom use, practices)

• On symptom perception (itching)

• On how people categorise their experience (definition of sex acts)

4. How do you deal with “ethics”?

The misplaced application of clinical trial ethics to social science research:

• Flexibility of questions and “instruments” (open interviews, CD)

• Flexibility of procedures (recording CRF interview, follow up women)

• Observation

Conclusion

• Mixed models rather than mixed methods

• Combine emic and etic approaches

• Follow up and solve inconsistencies during the study

• Reform “ethics” procedures

How do we decide which data are most accurate?Case Record Form

2 sex acts in the last week

Coital Diary Ticked “sex” 7 times

In-depth Interview

Reported that she’d had sex 7 times

IDI probing When asked about the inconsistency with the CRF she explained that in the CRF interview she had been asked about sex acts and not rounds.

Partner IDI Her partner reported having had sex 7 times in the same period

CONCLUSION 7 sex acts; CRF underreporting

Opposing theoretical paradigms

Positivism Constructivism

Reality Single, objective Multiple, constructed

Knowledge Neutral & objective Influenced by knower

Values No role Influence facts

Generalisation

Time & context free Time & context bound

Different methods

QUANT QUAL

Data type Numerical Narrative

Sample size Large Small

Instruments Closed Open

Aim Explain Explore, understand

Process Fixed Interative

Also:

• Continuation of the trial depends on informed consent

• Success of product depends on acceptability