www.monash.edu.au

Interpreting interventional studies

Dr Paul JenningsDepartment of Community Emergency Health and Paramedic PracticeMonash University

Interpreting an interventional study

• Interventional study– Investigators assign an exposure

Types of Quantitative

study designs

Experimental / Interventional

Randomised Controlled Trial (RCT)

Non / Quasi-randomised

Controlled Trial

Observational

Cohort (Longitudinal)

Case-control

Cross-sectional

Ecological (population-

based)

Dissemination of findings?

• Abstract• Introduction• Methods• Results• Discussion/Conclusion• References• Figures/Tables

Components of a paper?

• Introduction• Methods• Results• and• Discussion/Conclusion

IMRaD

• Title: Does it convey a message?

• Authors: Are they experts in the field/study type?

• Keywords: Appropriate, MeSH (medical subject headings).

• Abstract: Succinct overview of the paper, reflects manuscript?

• Publication Date: Recent? Still Relevant?

Preliminary Information

• The informative preview section of the paper.• What is the papers purpose?• Why was it done?• How was it done? Setting/design type/methods• What were the main findings?• What do these findings mean and why is it

important?• Structured or unstructured• Usually up to 250 words.

Abstract

Introduction

• Provides the context of the paper.• Describes what is known about the topic.• Describes gaps and limitations in the

literature.• Describes and introduces the

question/problem to be addressed.• States the rationale/purpose of the paper.

• Setting:– Where did the study occur?

• Participants:– Number, characteristics (homogeneity of

groups), selection/randomisation process.– Drop-out or exclusions

• Procedures:– Step-by-step process– Can the study be replicated?– Operational definitions provided

Methods & Design

• Are the methods used reliable?• Are the methods used valid?

– Randomisation? Process used (True vs psuedo)

– Was allocation concealed?– Analysis of groups (ITT vs per-protocol)– Blinding?– Group treatment (same?)– Representativeness

Methods & Design

• What was measured?• Does it report descriptive or inferential

statistics?• Are the statistical methods appropriate?• Is the significance level set and accurately

reported?• Are confidence intervals reported?• Does study power need to be reported?

Data Analysis

• Are the results clearly presented (text, tables, figures)?

• Does each result have a method?• Is there no discussion of results?• Is the precision of the results presented?

– 95% Confidence intervals

• Table 1

Resultsblah blah

blah blah blah blah blah blah blah blah blah blah blah blah

blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

blah blah blah blah blah blah blah blah blah blah blah blah blah

blah

blah blah blah blah blah blah blah blah blah blah

• Revision of the research question.• Presentation of the important results.• No new results presented.• Discussion of the results in the context of the

available literature.• Are there strengths/limitations discussed?• What is the generalizability of the results?• Recommendations for future research.• Can these findings be applied to patient care?

Discussion

• Are the conclusions supported by the results and available evidence?

• Are they justified?• Are they proportionate to the methods,

sample and design used? (Consider the NHMRC levels of evidence).

Conclusion

• Are they published accurately with full details• Does the list contain both contemporary and

seminal literature?• Any key papers missing?

References

• Many developed.• Example: Downs & Black. J Epidemiol

Community Health. 1998; 52:377-384.• Five domains:

– Reporting: 10 questions– External validity: 3 questions– Bias: 7 questions– Confounding: 6 questions– Power: 1 question (power to detect a clinically

important difference)

Review Checklists

Reporting

External Validity

Bias

Confounding

• CONSORT (CONsolidated Standards of Reporting Trials):

– Useful guide for reviewing RCTs– http://www.consort-statement.org/

• STROBE (Strengthening the Reporting of Observational studies in Epidemiology):

– Useful guide for reviewing observational studies.– cohort, case-control, and cross-sectional studies – http://www.strobe-statement.org/

Paper Presentation Guides

• There are few, if any, “perfect” studies/papers.• All papers have been through peer review.• Does the paper have a clear aim?• Is the method appropriate to answer the aim?• Does each result have a method?• Are the results adequately discussed in context?• Are the study limitations addressed?• Are the conclusions appropriate?

Summary

• Coughlan M, Cronin P, Ryan F. Step-by-step guide to critiquing research. Part 1: quantitative research. Br J Nurs. 2007; 16(11):658-663.

• Stockhausen L, Conrick M. Making sense of research: a guide for critiquing a paper. Contemp Nurse. 2002; 14(1):38-45.

Useful References

Thank-you for your attentionQuestions?