Ins and OUTs

Types

Level of evidence

Randamization

Designs

Statistical tests used

Advanced statistical tests

Critical appraisal

Critical appraisal tools

Clinical trial design issues

Types

Types of trials

Randomised clinical trials (RCTs)

Non- Randomised clinical trials-Allocation to the different treatment groups of a trial is not done through proper randomization process

Level of evidenceFive levels of evidence

Level Research Design Description

Level 1

Randomized controlled

trial (RCT)

Randomized controlled trial, PEDro score 6. Includes within

subjects comparison with randomized conditions and cross-over

designs

Level 2

RCT Randomized controlled trial, PEDro score < 6.

Prospective controlled trial Prospective controlled trial (not randomized)

Cohort

Prospective longitudinal study using at least 2 similar groups with

one exposed to a particular condition.

Level 3 Case control

A retrospective study comparing conditions, including historical

controls

Level 4

Pre-post

A prospective trial with a baseline measure, intervention, and a

post-test using a single group of subjects.

Post-test

A prospective post-test with two or more groups (intervention

followed by post-test and no re-test or baseline measurement) using

a single group of subjects.

Case Series

A retrospective study usually collecting variables from a chart

review.

Level 5

Observational Study using cross-sectional analysis to interpret relations.

Clinical Consensus

Expert opinion without explicit critical appraisal, or based on

physiology, biomechanics or "first principles"

Case Report Pre-post or case series involving one subject

Editors: Riegelman, Richard K.Title: Studying a Study & Testing a Test: How to

Read the Medical Evidence, 5th Edition, 2005, Lippincott Williams & Wilkins

Randamization Methods of Randomization

1. Fixed Randomization Methods

Simple randomization-tossing coin, random number tables or computer generated random numbers

Block randomization

Stratified randomization

Randomized consent method

2.Adaptive Randomization Methods

Play the winner

Minimization

Non randomization Alternate allocation

Geographical distribution

Cluster allocation

Odd/even number allocation

Based on date of birth/surnames /alphabetical orders

Designs

RCT designs

1. Factorial design

2. Cross- over design

3. Cluster random design

4. n-of-1 trials

5. Open trials

Statistical tests used

Parametric and non parametric tests

ANOVA

Statistical tests usedNonparametric Tests Parametric Tests

Chi-Square test- For frequency data Fisher’s exact test.

For N x N design and very small sample size Fisher's exact test should be applied

McNemar testMcNemar test can be used with two

dichotomous measures on the same subjects (repeated measurements). It is used to measure change. Mann-Whitney test -Ordinal data

independent groups.

Wilcoxon signed rank test- Ordinal data dependent (paired samples)groups

Kruskal-Wallis test -Ordinal data independent groups (more than 2 groups)

Friedman test- Ordinal data repeated measures.

Measures of association.Spearman correlation coefficient- for

correlation of non -parametric data.

One group t-test - Comparison of sample mean with a population mean.

Independent groups t-test- Comparison of means from two unrelated groups

Dependent or paired t-test- Comparison of means from two related samples

k unrelated group, one -way ANOVA test, two way ANOVA test- Comparison of means from k unrelated groups(more than 2 groups)

Repeated measures ANOVA- Interval or ratio data repeated measures (Comparison of means from k related groups)

Measures of Association.Pearson correlation coefficient- for

correlation of parametric data

Parametric tests and analogous nonparametric procedures

Type of analysis Example Parametric tests Nonparametric tests

Compare means between two independent groups

Is the mean Quality of life score of incontinence patients (at baseline)

for patients assigned to placebo different from the mean for

patients assigned to the treatment (Interferential therapy) group?

Two-sample t-test

Wilcoxon rank-sum test

Compare two quantitative repeated measurements

taken from the same sample

Was there a significant change in Quality of life score of incontinence patients between baseline and the

Four-month follow-up measurement in the treatment

group?

Paired t-test Wilcoxon signed-rank test

Compare means between three or more

independent groups

If the study have three groups (e.g., placebo, Interferential therapy,

kegal’s exercises),to know the mean difference among the three groups?

Analysis of variance

(ANOVA)

Kruskal-Wallis test

Association between two quantitative variables

Is Quality of life score associated with the patient’s age?

Pearson coefficient of correlation

Spearman’s rank correlation

Advanced statistical tests

Relative risk

Relative risk reduction

Relative risk increase

Absolute risk reduction

Absolute risk increase

Number needed to treat

Number needed to harm

Risk-benefit analysis

Critical appraisal

Strength and weakness/issues of RCT designs

Sample size calculation stated in the study

Selection methods- randomization in allocation -handles the issue of confounding factors of the study

Selection bias –inclusion and exclusion criteria

Homogenous groups are compared-no distribution bias

Blinding – reduces the bias of experimenter observation

Performance bias- groups treated equally apart from the treatments.

Standardized procedures followed in administering the treatment protocols

Reliability and validity of outcome measures used.

Duration of the study – adequate follow up logically needed to find a difference.

Intention to treat analysis performed- whether subjects received or completed the treatment.

Drop outs management.Worst case scenario method.Last data carried forward method

Appropriate statistical tests

Critical appraisal tools

Appraisal of a meta-analysis or systematic review

Appraisal of a controlled study

Appraisal of a cohort or panel study

Appraisal of a case control study

Appraisal of a cross-sectional study (survey)

Appraisal of a qualitative study

Appraisal of a case study

Checklist for Analytical Cross Sectional Studies

Checklist for Quasi-Experimental Studies (non-randomized experimental studies)

Checklist for Prevalence Studies

Critical appraisal tools AGREE - http://www.agreecollaboration.org/ Alberta University Evidence Based Medicine Toolkit - http://www.ebm.med.ualberta.ca/ CASP - http://www.sph.nhs.uk/what-we-do/public-health-workforce/resources/critical-

appraisals-skills-programme CATwalk - http://www.library.ualberta.ca/subject/healthsciences/catwalk/index.cfm CEBMH - http://cebmh.warne.ox.ac.uk/cebmh/education_critical_appraisal.htm Centre for Evidence Based Medicine - http://www.cebm.net CLIST Resources for Critical Appraisal - http://www.londonlinks.nhs.uk/groups/clinical-

librarians-informationskills-trainers-group/trainers-toolkit/resources-for-critical-appraisal

Scottish Intercollegiate Guidelines Network (SIGN): • http://www.sign.ac.uk/ AMSTAR • Shea, B.J., Grimshaw, J.M., Wells, G.A., Boers, M., Andersson, N., Hamel, C., &

Bouter, L.M. (2007). Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology, 7(10). doi: 10.1186/1471-2288-7-10 • Article link: http://www.biomedcentral. com/1471-2288/7/10

Clinical trial design issues

Appropriate statistical tests (e.g.;-pre-post design analysis ‘T’ test Vs Anova)

Surrogate end points –Information bias

Hawthorne effect( handled by keeping a silent group)

Interim analyses( to avoid abundant sample coverage)

Drop outs management.

Worst case scenario method.

Last data carried forward method