1critical appraisal

8/10/2019 1Critical Appraisal

1/45

Critical Appraisal of theLiterature

Sarah-Jo Sinnott MPharm MPSI

Dept of Epidemiology and Public Health [email protected]


2/45

Overview

Critical Appraisal as part of evidence baseddentistry

Why do critical appraisal? How to do critical appraisal

First steps in deciding whether to read a paperproperly or not

Steps when reading the paper properly Tools to help you


3/45

Scenario

You are working as a dentist/hygienist in a busyprivate practice

You are visited by a sales rep for a dental supplycompany who urges you to buy their latest state -of- the art technology for treating dental caries HealOzone


4/45

No pain, no drilling, no side -effects.Didn't you always wish you could saythese three catchphrases to yourpatients before their treatment?

This is now possible with HealOzone,a new therapy concept for cariesremoval


5/45

Ozone (O 3) is a naturally occurring compound and a

powerful oxidising agent Toxic to living systems in high concentrations (damages cellmembranes)

Applying ozone to carious lesions reduces the numberof cariogenic bacteria Arrests or reverses lesions and prevents the need to restore

the tooth surface

A very attractive alternative to conventional drillingand filling

The rep advises you to check their website for fulldetails and promises to return in a week to discussfurther

How does it work?


6/45

Do you buy a HealOzone machine foryour dental practice?

What further information do youneed to help to make your decision?


7/45

Critical Appraisal as part of EvidenceBased Medicine/Dentistry

Definition of EBM

Evidence based Medicine (dentistry) is theenhancement of a clinicians traditional skills indiagnosis, treatment, prevention and relayedareas through the systematic framing of relevant

and answerable questions and the use ofmathematical estimates of probability and risk Greenhalgh 2001


8/45

Evidence-basedDentistry

The needs and preferencesof patients

The best availableevidence

The professionals expertise, skills and

judgement

The three elements of evidence-based dentistry (EBD)


9/45

Critical Appraisal as part of Evidencebased medicine/dentistry

1. Convert information needs into ananswerable question

2. Track down, EFFICENTLY, the best

evidence to answer question

3. Appraise the evidence critically, assessvalidity and usefulness

4. Implement results into clinical practice

5. Evaluate performance


10/45

Convert your question into ananswerable question: PICO format

P Population

I Intervention C Comparison O Outcome

(T) Time


11/45

Convert information needs into ananswerable question

PICO format

For an adult population with caries , does Ozone therapycompared to conventional therapy for caries result in a

more effective treatment, as measured by the reversal of,or arrest of, caries ?

For a patient with tooth discoloration due to coffee andtobacco , will Crest Whitestrips , as compared to customtrays for use with an at-home whitening/bleachingsystem , be as effective in whitening his teeth within 1week


12/45

Critical Appraisal as part of Evidencebased medicine


2. Track down, EFFICENTLY, the bestevidence to answer question





13/45

Track down, EFFICENTLY, the bestevidence to answer question

Key word


14/45

How to search

Has anyone else searched before you andsummarised the evidence? The Cochrane Library

Pubmed, Embase, Web of Science, CINAHL Google Scholar

Use PICO tooth discolouration AND coffee ORtobacco AND whitening


15/45


16/45

How to search

Has anyone else searched before you andsummarised the evidence? The Cochrane Library

Pubmed, Embase, Web of Science, CINAHL Google Scholar

Use PICO tooth discolouration AND coffee ORtobacco AND whitening


17/45


18/45


19/45

Critical Appraisal as part of Evidencebased medicine


2. Track down, EFFICENTLY, the bestevidence to answer question





20/45

What is Critical Appraisal

Balanced assessment of strengths of researchagainst its weaknesses

Assessment of research process and results To be undertaken by all health professionals as

part of their work Both dentist and hygienist


21/45

What Critical Appraisal isnt

Negative dismissal of research Assessment of results alone

P


22/45

Why do Critical Appraisal

Published research is not always reliable

Only an estimated 2% is judged clinicallyrelevant

Patients come pre-read


23/45

Critical Appraisal on first glance

Why was the study done and what hypothesiswere the authors testing? Bottom of intro Beginning of methods Discussion


24/45


What type of study was done?Secondary Research(also Economic analyses,decision analyses,

guidelines)

PrimaryResearch


25/45


Was this design appropriate to the question addressed?

Choosing the best research design for each question Sackett, 1997, BMJ

ResearchConcern

Description Preferred design

Therapy Testing efficacy of method,

approach, equipment

RCT

Diagnosis Is a test valid (can we trust it) orreliable (? same result if repeated)

Cross-sectional survey ofnew test and gold standard

Screening Demonstrating the value of tests topick up a defect at a pre-

symptomatic stage

Cross-sectional survey

Prognosis What happens to someone whosillness is picked up at an earlystage

Longitudinal cohort study

Causation Does harmful agent lead to aparticular outcome

Cohort or case-control


26/45


27/45

Going through the methods..

1. Was the study original?2. Who was it about?

3. Was it well designed?4. Was systematic bias avoided?5. Statistically sound?


28/45


29/45

Who was it about?

How were subjects recruited?

Who was included in the study? If inclusion criteria very narrow, results may not be

generalisable

Who was excluded from the study Focus on moderate/severe forms of disease could lead to

false conclusion about treatment of mild disease

Were subjects studies in real life circumstances? Applicability of the results to your practice


30/45

Was it well designed?

Identify Intervention Comparison

What was the outcome and how was it measured? Valid (e.g. Index) Reliable calibrated examiners and measure of reliability

(Kappa) Surrogate outcomes

-We measured how often dentists ask patients aboutfamily history of oral cancer

-We looked in dental records and counted how manypeople had family hx recorded

-Assumption that dental records are 100% accurate


31/45

Was systematic bias avoided?

Any error in the design or conduct of a studythat results in a conclusion that is differentfrom the truth

Groups being compared should be as similaras possible except for the difference beingexamined


32/45

Bias

Selection Bias Systematic errors are introduced by the selection of

study participants or allocation of individuals todifferent study groups

Comparison groups no longer comparable RANDOMISATION

Measurement Bias Inaccurate measurement of classification of an

outcome or exposure BLINDING


33/45

Statistically Sound

1. Size of sampleo Powero Clinical difference

2. Duration of follow upo Survivalo Painkillers

3. Completeness of follow upo Death?o Non-adherent patients?o Withdrawal by clinician?o Adverse reactions?o INTENTION TO TREAT

Ignoring these people will bias

results in favour of intervention


34/45

Statistically Sound

Tests Categorical vs continuous Are the data normal?

Parametric vs non-parametric Repeated measures on patients?

Paired tests Tests of difference? Correlation? Causalitybeware associations

E.g., coffee drinkers found to be associated with higherrates of cancer.

Categorical:Caries; Yes or No

Continuous:Height: ~30cm to ~200cm


35/45

Four possible outcomes from any study

1. Difference is clinically and statisticallysignificant i.e. important and real.

2. Of clinical significance but not statisticallyso. ?sample size too small.

3. Statistically significance but not clinicallyi.e. not clinically meaningful.

4. Neither clinically nor statisticallysignificant.


36/45

Checklists

Help with assessing quality of research can befound at

http://www.casp-uk.net/

And

http://www.cebm.net/index.aspx?o=1157
http://www.casp-uk.net/http://www.cebm.net/index.aspx?o=1157http://www.cebm.net/index.aspx?o=1157http://www.casp-uk.net/http://www.casp-uk.net/http://www.casp-uk.net/


37/45


38/45

Checklists GRADE (google scholar thisto get pdf)


39/45

Do you buy a HealOzone machine foryour dental practice?

What further information do youneed to help to make your decision?


40/45


41/45


42/45


43/45


44/45

In summary what did we cover?

How critical appraisal is essential in EBD How to formulate your research question How to go about searching for evidence How to decide whether to read something or

not..time is precious

How to read it properly Tools to help you read the paper and make a

decision on whether you should take itseriously or not.


45/45

References

Greenhalgh, T. (2001). How to read apaper: Assessing the methodologicalquality of published papers. London:BMJ Publishing Group

Users guides to the medicalliterature 1990s, JAMA (about 20articles)

Tip

USE THE GUIDELINES!

1critical appraisal

Documents