Department of Pharmacology and Toxicology

Gadjah Mada University

QUASI EXPERIMENTAL

Nonrandom assignment in experiments

Perlu strategi/metode untuk mengantisipasi tidakdapat dilakukannya random assignment.

unethical

Impractical

Very expensive

Interest in „intact groups‰

Alasan:

Pendahuluan

Pada penelitian sosial

fenomena tidakharus dibandingkan

randomisasi sulit dilakukan

tidak harusada kontrol

Pendahuluan:

• pengamatan efek tidak selalu harusdibandingkan (preliminary)

• keterbatasan scope yang diteliti

• randomisasi sulit dilakukan

• aklimatisasi diasumsikan telahmengendalikan confounder

• pengamatan efek tidak selalu harusdibandingkan (preliminary)

• keterbatasan scope yang diteliti

• randomisasi sulit dilakukan

• aklimatisasi diasumsikan telahmengendalikan confounder

Penelitian di laboratorium

• Randomisasi?• Blinding?• Inclusion criteria?• Heterogeneity?• Homogeneity?

• Randomisasi?• Blinding?• Inclusion criteria?• Heterogeneity?• Homogeneity?

External validity?Ekstrapolasi?

Keterbatasan Quasi Experimental

• Tidak dapat digunakan untuk mengujihipotesis causal

• Sulit untuk ekstrapolasi ataugeneralisabilty

• Tidak menggambarkan true value

• Tidak dapat menyingkirkan confounder

Notasi

• X = treatment

• O = observasi

• --------= non randomised

Design quasi experimental yang tidak memungkinkancausal inferences

1. One group posttest only design

3. One group pretest-posttest design

2. Posttest only designwith non-equivalent group

• Tanpa pretest observation• Tanpa kelompok kontrol

1. One group posttest only design

Design: X O

1. Pemberian pengobatan2. Evaluasi hasil pendidikan3. Environmental toxicology

Kapan dilakukan?

• Tanpa pretest observation• Efek treatment vs. selection effect• non ekuivalen

Design:

progress suatu treatment tidak bersamaan

Kapan dilakukan?

X O

O

2. Posttest only design with non-equivalent group

Contoh:

X 12 O

X 9 O

X 6 O

X 0 O

Mengamati hasil counseling 12 bulan, 9 bulan,6 bulan, dan tanpa counseling

• Pada satu kelompok• Disertai pretest observation

Design:

Apa kelemahannya ?

3. The One-Group Pretest-Posttest design

Contoh: efek supervisi

O1 X O2

Kelemahan

• History (kenaikan gaji, perubahan policy)

• Regresi (produktivitas kerja)

• Spurious

• Efek maturasi (lebih skillful)

The One-Group Pretest-Posttest design

O1 X O2

1. The untreated control group designwith pretest & posttest

O1 O2

Design:

• Selection maturation• Instrumentation• differential statistical regression• interaction of selection and history

Masalah:

O1 X O2

Populasi-2

Proportion blood Proportion blood Proportion blood Proportion blood pressure and /or lipid pressure and /or lipid pressure and /or lipid pressure and /or lipid lowering drug uselowering drug uselowering drug uselowering drug use

Populasi-1

Proportion blood Proportion blood Proportion blood Proportion blood pressure and / or lipid pressure and / or lipid pressure and / or lipid pressure and / or lipid lowering drug uselowering drug uselowering drug uselowering drug use

Proportion blood Proportion blood Proportion blood Proportion blood pressure and /or lipid pressure and /or lipid pressure and /or lipid pressure and /or lipid lowering drug uselowering drug uselowering drug uselowering drug use

Proportion blood pressure Proportion blood pressure Proportion blood pressure Proportion blood pressure and /or lipid lowering and /or lipid lowering and /or lipid lowering and /or lipid lowering

drug usedrug usedrug usedrug use

intervention No interventiointervention No interventiointervention No interventiointervention No interventionnnn

Atthobari, et al. Br. J. Atthobari, et al. Br. J. ClinClin. . PharmPharm. 2003; 57(3):328. 2003; 57(3):328--336336

OUTCOME 1

Pretest Posttest

treatment

control

O1 O2

O1 X O2

Atthobari, et al. Br. J. Atthobari, et al. Br. J. ClinClin. . PharmPharm. 2004; 57(3):328. 2004; 57(3):328--336336

ControlExperiment

OUTCOME 2

Pretest Posttest

treatment

control

Terdapat pola selection-maturation interaction

Contoh: memberikan pendidikan ekstra pada anak yg. > pandai

O1 O2

O1 X O2

OUTCOME 3

Pretest Posttest

treatment

control

O1 O2

O1 X O2

OUTCOME 4

Pretest Posttest

treatment

control

Contoh: insentif extra untuk meningkatkan performance

O1 O2

O1 X O2

OUTCOME 5

Pretest Posttest

treatment

control

O1 O2

Contoh: intervensi pada paramedik vs. dokter

O1 X O2

Threats to internal validity⁄

1. history

2. maturation

3. testing

4. instrumentation

5. statistical regression

6. differential selection of participants

7. mortality

8. selection-maturation interaction

⁄the occurrence of events that are not part of the experimental treatment but that occur during the study and affect the dependent variable

1. history

History: another study

Peningkatan LLD pada pasien yang di screening

7.06.05.04.03.02.01.00.0

Follow up (years)

0.3

0.2

0.1

0.0

Cu

mu

lati

ve

in

cid

en

ce

Screened enriched cohort

Screened aselect cohort

Unscreened cohort

Lipid Lowering Drugs (LLD)

Atthobari, et al. Br. J. Atthobari, et al. Br. J. ClinClin. . PharmPharm. 2007, in press. 2007, in press

2. maturation2. maturation

⁄the physical, intellectual, and emotional changes that occur naturally in a studyÊs participants over a period of time

Maturation threat:Clinical Skills improves as physician naturally mature over the year.Can you conclude that the training was effective?

Apakah training mampu meningkatkan clinical skilssdokter baru?

0

10

20

30

40

50

60

70

80

90

Pre Post

Mean adherence

3. testing3. testing

⁄refers to improved scores on a posttest as a result of having taken a pretest

Testing threat:If similar problems are used in the pretest and posttest, faster problem solving may be due to familiarity with the diagnosisCan we conclude that teaching the new technique improves diagnostic skill?

0

2

4

6

8

10

12

14

pre post

Minutes (M

ean)

Apakah training mempengaruhi kecepatan dokterdalam menegakkan diagnosis?

4. instrumentation4. instrumentation

⁄the unreliability or lack of consistency in measuring instruments that can result in an invalid assessment of performance

Instrumentation threat:Pada saat yang bersamaan, perekonomian sangatmemburuk, daya beli masyarakat rendah, kemampuanekonomi masyarakat rendah

Apakah program KB menurunkan angka kelahiran bayi?

0

5

10

15

20

25

30

35

40

45

50

1 2 3 4 X 5 6 7 8

Month

Reports of Rape

5. statistical regression5. statistical regression

⁄the tendency of participants who score highest on a test to score lower on a second, similar test and vice versa

Regression threat:Alat evaluasi post test tidak reliabel untuk mengukur penurunantekana darah, akibatnya terjadi regress to the meanCan we conclude the drug ÂXÊ is effective?

Terapi obat ÂXÊ terhadap tekanan darah

0102030405060708090

100

Pre Post

Tes

t Sc

ores

(M

ean)

6. differential selection of participants6. differential selection of participants

⁄the outcome when already formed groups are compared raising the possibility that the groups were different before a study even begins

0000

10101010

20202020

30303030

40404040

50505050

60606060

70707070

%%%%

Terapi ATerapi ATerapi ATerapi A Terapi BTerapi BTerapi BTerapi B

Tidak sembuhTidak sembuhTidak sembuhTidak sembuh SembuhSembuhSembuhSembuh

Hasil berbeda lebih disebabkan oleh perbedaankarakteristik pada 2 kelompok

7. mortality7. mortality

⁄the case in which participants drop out of a study which changes the characteristics of the groups and may significantly affect the studyÊs results

0000

10101010

20202020

30303030

40404040

50505050

60606060

70707070

%%%%

Terapi ATerapi ATerapi ATerapi A Terapi BTerapi BTerapi BTerapi B

Tidak sembuhTidak sembuhTidak sembuhTidak sembuh SembuhSembuhSembuhSembuh

Hasil berbeda karena angka drop out pada salahsatu kelompok lebih tinggi drpd kelompok yang lain

8. selection8. selection--maturation interactionmaturation interaction

⁄if already-formed groups are used in a study, one group may profit more (or less) from a treatment or have an initial advantage because of maturation, history, or testing factors

Proxy Pretest Design

• pretest based on recollection or archiveddata

• useful when you werenÊt able to get a pretest but wanted to address gain

N O1 X O2

N O1 O2

Contoh: Tujuan studi: mengetahui efek suatuprogram terhadap peningkatan kepuasan

pasien di UGD

• Program sudah berjalan• Kita bisa mendapatkan nilai post test,

tetapi tidak memiliki nilai pre test. • Kita perlu mencari proxy variable yang

kira-kira dapat digunakan untuk estimasipre test.

• Sebagai contoh: gunakan nilai pretest pasien di kamar bersalin sebagai proxy pretest.

Separate Pre-Post Samples

• Groups with the same subscript come from the same context

• here, N1 might be people who were in the program at Agency 1 last year, with those in N2 at Agency 2 last year

• This is like having a proxy pretest on a different group

N1 ON1 X ON2 ON2 O

Separate Pre-Post Samples

• take random samples at two times of people at two nonequivalent agencies

• useful when we routinely measure with surveys

• can assume that the pre and post samples are „equivalent‰ but the two agencies may not be

R1 OR1 X OR2 OR2 O

NN

Contoh: tujuan penelitian adalah meningkatkankepuasan pasien di 2 rumah sakit

• Pasien berganti-ganti dari waktu ke waktu, kitatidak bisa mengukur nilai pretest untuk pasienyang sama.

• Lakukan pengukuran kepuasan pada satu saat, kemudian implementasikan program selamakurun waktu tertentu.

• Desain ini tidak cukup kuat, karena mengukurorang yang berbeda

• Oleh sebab itu yang dilihat hanya perubahanantara sebelum program dan setelah program.

Double Pretest DesignDouble Pretest DesignDouble Pretest DesignDouble Pretest Design

• strong in internal validity

• helps address selection-maturation

• how does this affect selection-testing?

N O O X ON O O O

Switching Replications

• strong design for both internal and external validity

• strong against social threats to internal validity

• strong ethically

NNNN OOOO XXXX OOOO OOOONNNN OOOO OOOO XXXX OOOO