critical appraisal

CARA EFEKTIF BELAJAR SEPANJANG HAYAT

DR. ARIANI RD. ,SPM. FK UNISMA MALANG. 2011.

Critical Appraisal

The Problem

• Vast and expanding literature.

• Limited time to read.

• Different reasons to read : – Keeping up to date.– Answering specific clinical questions.– Pursuing a research interest.

mean different strategies

2

1

Stages

Clarify your reasons for reading.

Specify your information need.

Identify relevant literature.

Critically appraise what you read.

2

Clarify Your Reasons for Reading

Keeping up to date. Skimming the main journals and summary bulletins.

Answering specific clinical questions. Finding good quality literature on subject.

Pursuing a research interest. Extensive literature searching.

3

Specify Your Information Need

What kind of reports do I want?

How much detail do I need?

How far back should I search?

The answers to these questions should flow from the reasons for reading.

4

Kind of Reports

...Specify Your Information Need

What causes this disease?

Randomized controlled clinical trial (RCT), Prospective Cohort Study, Case Control Study (Rare Diseases, probably retrospective)

How good is a diagnostic test?

Prospective cohort study with good quality validation against “Gold Standard”

Does this treatment work? RCT

How is the prognosis? Cohort Study, Cross-sectional Survey, Qualitative Study

Study Types for Question Types

5

.Identify Relevant Literature

There are many ways of finding literature.

Selectivity is the key to successful critical appraisal.

6

Ways of Finding Literature

Source of Medical Information Colleagues Conferences Drug Reps Textbooks Journals Internet/Patients

...Identify Relevant Literature

7

Keeping Up to Date

Daily InfoPOEMs http://www.infopoems.com/ bmjupdates+

http://bmjupdates.mcmaster.ca/index.asp Tables of contents of journals

(e.g. JAMA) From journal itself My NCBI from PUBMED


8

http://www.infopoems.com/

http://bmjupdates.mcmaster.ca/index.asp

Clinical Question - Research

EBM Databases MEDLINE

Free online access thru PubMed www.ncbi.nlm.nih.gov/entrez/ Search by MESH terms or free text EBM Filters


9

http://www.ncbi.nlm.nih.gov/entrez/

Selectivity


Filters for Medline Search Diagnosis

Sensitivity and Specificity Treatment

Randomized controlled trial Blind or Double Blind

10

Critically Appraisal What You Read

E B M

11

What is EBM ?

Why do it ?

Evidence-Based Medicine

...Critically Appraisal What You Read

12

"...the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.“

from Sackett, DL, et al. "Evidence based medicine: What it is and what it isn't." (BMJ 1996; 312: 71-2)

What is EBM ?


13

What is EBM ?

An approach to clinical decision-making

in which one searches the literature,

critically evaluates the research evidence, and

then chooses the most appropriate intervention

or course of action to take.


14

APPLICATION TO PRACTICE

Increase patient trust

Provide best possible care

Reduce liability risk

Why do EBM ?


15

What a Physician can be Faced With on a Daily Basis?

~1 - 2 questions per patient (clinics)15/patient/day (wards)

40% easy to answer30% tough to answer30% cannot answer

Source: Dawes M, Sampson U. Knowledge management in clinical practice: a systematic review of information seeking behaviour in physicians. Int J Med Inform. 2003 Aug; 71(1):9-15. Review. PMID: 12909153


16

Critical Appraisal of Literature

Intended to enhance the clinician’s skill todetermine whether the results reported inan article were likely to be . . . .

. . . true . . . important . . . applicable to their patients!


17

Why Should we Critically Appraise?

Published research is not always reliable

Published research is not always relevant

To improve clinical effectiveness, we need a systematic framework to interpret research


18

Difficulties with Critical Appraisal

o Can be time consuming initially

o Doesn’t provide an “easy” answer

o It could show a lack of good evidence in a particular topic


19

How can we do Critical Appraisal?

Use common sense

Use simple checklists

Use different checklists depending on the different types of studies (i.e., RCTS, systematic reviews etc)

Checklists help you focus on the important parts of the article


21

Checklists

Quick and easy

3 stages: Basic questions. Essential appraisal. Detailed appraisal.

22...Critically Appraisal What You Read

22

1st. Basic Questions.


How to Read a Research Article? – First Glance.

1. Purpose of reading the paper.2. Do not read the abstract.3. Read the title, find out who the authors are and where they work,

look for sources of funding and conflicts of interest4. Look at the tables and figures5. Is there a diagram to show the flow of participants through each

stage of the study 6. Why was the study done and what hypothesis were the authors

testing?7. What is broadly the topic of research?8. Therapy, Diagnosis, Screening, Prognosis, Causation?9. What type of study was done?10. Is this a primary (experimental, clinical trial, survey) or a

secondary paper (review, meta-analysis, guideline)?

...1st. Basic Questions.

24

Patient or problemInterventionComparisonOutcome

PICO Structure

...1st. Basic Questions.

25

2nd. Essential Appraisal.


What are the results?

Are the results valid?

Will the results help me in patient care?

EBM “simplified” approach

...2nd. Essential Appraisal.

27

KEY QUALITY PARAMETERS

VALIDITY

RELIABILITY

IMPORTANCE


28

VALIDITY

INTERNALIs the study designed in such a way that I can trust the findings?

EXTERNALIs the study designed in such a way that Ican generalize the findings?


29

RELIABILITY

If the study was conducted again,would the results be the same?

Usually interpreted as the accuracy of measurement.


30

Validity and Reliability

A test is valid when it measures what it’s supposed to.

If a test is reliable, it yields consistent results.

A test can be both reliable and valid, one or the other, or neither.


31

Reliable, but Not Valid!


Not Reliable, Not Valid!


Reliable and Valid


IMPORTANCE

What was the effect sizeor magnitude of effect?

Clinical vs. statistical significance.


35

3rd. Detailed Appraisal.


Dasar-Dasar Metodologi

Penelitian KlinisEdisi ke-2

Prof. DR. Dr. Sudigdo Sastroasmoro, Sp.A (K)

Prof. Dr. Sofyan Ismael, Sp.A (K)

Jakarta 2002

Deskripsi Umum

Desain apakah yang digunakanManakah populasi target, populasi terjangkau,

sampelBagaimanakah cara pemilihan sampelManakah variabel bebasManakah variabel tergantungApakah hasil utama penelitian

Sastroasmoro, Sudigdo (2002)

Tabel 21-2. Telaah Kritis Makalah Kedokteran : Hal-hal yang dinilai pada studi hubungan sebab-akibat.

38

Apakah hasil dipengaruhi biasApakah hasil dipengaruhi faktor peluangApakah observasi dipengaruhi perancu


Validitas Interna, hubungan non-kausal


39

Validitas Interna, hubungan kausal

Apakah hubungan waktu benarApakah asosiasi kuatApakah ada hubungan dosisApakah hasil konsisten dalam penelitian iniApakah hubungan bersifat spesifikApakah ada koherensiApakah hasil biologically plausible



40

Validitas Eksterna

Apakah hasil dapat diterapkan pada subjek terpilih

Apakah hasil dapat diterapkan pada populasi terjangkau

Apakah hasil dapat diterapkan pada populasi yang lebih luas



41

Desain Penelitian

Deskripsi Umum

Diagnosis Prospective cohort study.

Therapy Randomized controlled clinical trial (RCT)

Etiology/Harm

RCT.Cohort.Case-control study.

Prognosis Prospective cohort study

42

Desain Penelitian

TARGETPenerapan Hasil Penelitian. Batas : klinis, demografis

TERJANGKAU/SUMBER

Populasi Target dibatasi tempat dan waktu.

SAMPEL

Populasi

Deskripsi Umum

48

Populasi

Deskripsi Umum

Populasi TARGET

Populasi Terjangkau / SUMBERSAMPEL49

Sampel

Karakteristik Subjek Penelitian

= pasien kita ?

Dapat kita terapkan ?

Deskripsi Umum

50

Cara Pemilihan Sampel

Probability Sampling Simple Random Sampling Systematic Sampling Stratified Random Sampling Cluster Sampling

Non Probability Sampling Consecutive Sampling Convenient Sampling Judgmental / purposive sampling

Deskripsi Umum

51

Variabel

BEBAS: faktor resiko, perlakuan (pemeriksaan /

terapi medikamentosa / tindakan)

TERGANTUNG: dipengaruhi oleh variabel bebasFaktor resiko Kejadian penyakitPemeriksaan TerdiagnosisObat / Tindakan Kesembuhan

Deskripsi Umum

52

Bias Seleksi Subjek

1. Neyman’s Bias Bias prevalens / insidenPenyakit dengan mortalitas tinggi di awal

2. Berkson’s fallacyAdmission rate biasCase-control : perbedaan indikasi rawat kontrol

Validitas Interna, non-kausal

53

Bias Seleksi Subjek

3. Bias non-respons / bias relawanLatar belakang kesediaan / penolakan sebagai relawan

4. Membership biasMultifaktorial

5. Procedure selection biasNon randomized tidak setara


54

Bias Pengukuran

1. Bias prosedur Perlakuan tidak setara antar kelompokKetersamaran (blind / double-blind)

2. Recall biasCase-control : kelompok kontrol < ingat

3. Insensitive measurement biasAlat ukur < sensitif


55

Bias Pengukuran

4. Detection bias Deteksi > dini (alat ukur > sensitif ?) masa sakit >>

5. Compliance biasCase-control : frekuensi pemakaian obat berbeda


56

Perancu

( Confounding factors )


Minum Kopi

Merokok

Penyakit Jantung Koroner

Variabel bebas

Variabel tergantung

Perancu

57

Peluang

Kemungkinan hasil tersebut disebabkan peluang bila hipotesis nol (tidak ada/tidak)

P valueNilai P ≤ 0,05

Confidence intervalCI ≥ 95%


58

Hubungan Waktu

Variabel bebas (sebab, faktor resiko, perlakuan)

HARUS MENDAHULUI

Variabel tergantung(akibat, penyakit, kesembuhan)


59

Asosiasi

Kemungkinan Sebab A menghasilkan Akibat B

lebih besar bila

Atau

Nilai P (peluang / probability) KecilInterval kepercayaan (CI : Confidence Interval) Sempit


BESARRR (Relative Risk)Odds ratioPrevalens ratio

60

hubungan lain

Hubungan DosisDose dependent Studi menetapkan dosis optimal

KonsistenBeberapa kelompok subjek multicenter

KoherensiFenomena / kenyataan dalam masyarakat


61

…hubungan lain

Biological plausibilityHipotetis – spekulatif

Perbandingan dengan hasil penelitian lainBeda Mengapa ?


62

Validitas Eksterna

pada Sampel Terpilih Drop out : ≤ 15%

pada Populasi Terjangkau Random sampling / Consecutive sampling

pada Populasi TargetLogika dalam bab DiskusiKarakteristik subjek ~ pasien sehari-hari

Validitas Eksterna

63

Stay up to date with the current literature

A clinician needs to read 17 peer reviewed articles per day, every day of the year, to stay current (Haynes 1993).

Haynes, R. (1993) Where's the meat in clinical journals? ACP Journal Club, 119: A23-4.

There are 20 million pieces of 'evidence' of varying quality and sometime of contradictory conclusions.

64

Basic elements of clinical decision making

Haynes RB. Loose connections between peer-reviewed clinical journals and clinical practice. Ann Intern Med 1990;113:724-8.

"Without clinical expertise, practice risks becoming tyrannized by external evidence,

for even excellent external evidence may be inapplicable to or inappropriate for an

individual patient. Without current best external evidence, practice risks becoming rapidly out of date, to

the detriment of patients."

Clinical Expertise & Critical Appraisal

Haynes RB. Loose connections between peer-reviewed clinical journals and clinical practice. Ann Intern Med 1990;113:724-8.

67

TERIMA KASIH

Critical Appraisal. Cara Efektif Belajar Sepanjang Hayat. Dr. Ariani RD.,SpM. FK Unisma Malang. 2011.

critical appraisal

Documents