9/4/2014

1

Biostatistics

Kevin M. Sowinski, Pharm.D., FCCP

Pharmacotherapy Webinar Review Course

Tuesday, September 3, 2014

I have no conflict of interest to disclose

Conflicts of Interest

Objectives

By the end of this presentation, participants

should be able to:

Describe the appropriate application and

interpretation of statistical tests commonly

encountered in the pharmacotherapy literature.

Differentiate observational and controlled trial

designs

Detect common errors in the presentation and

interpretation the of clinical study designs.

Descriptive statistics: Numerical methods

Measures of Central Tendency

Mean

Used only for continuous and normally distributed data

Median (a.k.a 50th percentile)

Midpoint of the values when placed in order from highest to lowest. Half above and below.

Used for ordinal or continuous data (especially for skewed populations)

Mode

Most common value in a distribution

Used for nominal, ordinal, or continuous data

Data may have > one mode (bimodal, trimodal)

SD: measure of the variability about the mean,

applied to normally distributed, continuous data

Empirical rule: 68% within ±1 SD, 95% within

±2 SD, and 99% within ±3 SD

CV relates the mean to the SD

(SD/mean×100%)

Variance = SD2

Others: Range and Percentiles (IQR)

Measures of Data Spread and Variability

Null hypothesis (H0): No difference between comparator groups (Tx A = Tx B)

Alternative hypothesis (Ha):

States that there is a difference (Tx A Tx B)

Results of “hypothesis testing” will indicate

whether there is enough “evidence” to reject H0

H0 is “rejected”= statistically significant (SS) difference

H0 is “not rejected” = no SS difference

We are not concluding that the treatments are equal.

Hypothesis Testing

9/4/2014

2

Decision Errors

Probability of making type I error = significance level (α)

Usually α=0.05, 5.0% of the time, we will conclude there is a

SS difference when actually one does not exist.

Calculated chance that a type I error has occurred is called

the “p-value.”

Lower p-value does not suggest more importance, only SS

and less likely attributable to chance

Type II error (b)

Usually b=0.10-0.20

Concluding that no difference exists when one truly

does (not rejecting H0 when it should be rejected)

Statistical Significance Interpreting p-values

Size of p-value is not related to the importance of the result

Statistically significant does not necessarily mean clinically significant

Lack of statistical significance does not mean results are unimportant

Post hoc power calculations

For negative results…

For positive results (reject Ho)….

Confidence Intervals

Can be reported for many different types of analysis

A CI is a range of values that are likely to cover the

true population parameter Difference between two means:

“range” of values along with a point estimate of the difference between the two groups.

A 95 % CI that includes a value consistent with no difference (i.e. 0) can be interpreted as a p 0.05

OR, RR, Hazard Ratio

Statistical significance assessed based on whether bounds of CI include 1

Choosing A Test

Type of

Variable

2 Samples

(independent)

2 Samples

(related)

> 2 Samples

(independent)

> 2 Samples

(related)

Nominal χ2 or Fisher exact test McNemar test χ2 Cochran Q

Ordinal Wilcoxon rank sum

Mann-Whitney U-test

a.k.a Mann-Whitney-

Wilcoxon test

Wilcoxon signed rank

Sign test

Kruskal-Wallis

(MCP)

Friedman ANOVA

Continuous

No factors

Equal variance t-test

Unequal variance t-test

Paired t-test

1-way ANOVA

(MCP)

Repeated-measures

ANOVA

1 factor ANCOVA

2-way repeated-

measures ANOVA

2-way ANOVA

(MCP)

2-way repeated-

measures ANOVA

Vignette 1 (Choosing Appropriate Measures of Central Tendency)

A pharmacy practice resident is planning a project that compares patient satisfaction with the services of a recently established medication therapy management program. She plans to evaluate several outcomes of the program’s impact (e.g., adherence, goal attainment) including patients’ satisfaction with the services provided. Satisfaction will be assessed using a Likert scale-based instrument. The potential responses to the question, are you satisfied with the services provided?

Strongly agree (1), Agree (2), No opinion (3), Disagree (4), Strongly disagree (5)

A pharmacy practice resident is planning a project that compares patient satisfaction with the services of a recently established medication therapy management program. She plans to evaluate several outcomes of the program’s impact (e.g., adherence, goal attainment) including patients’ satisfaction with the services provided. Satisfaction will be assessed using a Likert scale-based instrument. The potential responses to the question, are you satisfied with the services provided?

Strongly agree (1), Agree (2), No opinion (3), Disagree (4), Strongly disagree (5)

Question 1 Which measures of central tendency and data dispersion should be used to compare patient satisfaction between the intervention group and a control group?

9/4/2014

3

Question 1 (Choosing Appropriate Measures of Central Tendency)

Which measures of central tendency and

data dispersion should be used to

compare patient satisfaction between the

intervention group and a control group?

A. Mean and SD

B. Median and SD

C. Median and IQR

D. Mode and SEM

A pharmacy practice resident is planning a project that compares patient satisfaction with the services of a recently established medication therapy management program. She plans to evaluate several outcomes of the program’s impact (e.g., adherence, goal attainment) including patients’ satisfaction with the services provided. Satisfaction will be assessed using a Likert scale-based instrument. The potential responses to the question, are you satisfied with the services provided?

Strongly agree (1), Agree (2), No opinion (3), Disagree (4), Strongly disagree (5)

Question 1a Which statistical test should be used to compare patient satisfaction between the intervention group and a control group?

Question 1a (Choosing Appropriate Statistical Tests)

Which statistical test should be used to

compare patient satisfaction between the

intervention group and a control of group?

A. Fisher’s Exact Test

B. Mann-Whitney U

C. ANOVA

D. Wilcoxon signed-rank

Question 1a-Follow-Up (Choosing Appropriate Statistical Tests)

If the results were presented and analyzed

in the following way, is there an alternative

analysis that could be used?

SA A NO D SD

Control 40% 24% 25% 6% 5%

Intervention 60% 15% 13% 7% 5%

Vignette 2 (Decision Errors)

A recent study comparing two medications on their ability to lower blood pressure concludes no difference between them (p > 0.05). In reviewing the methods section you read that the investigators calculated their sample size by setting alpha at 0.05 and beta at 0.40.

A recent study comparing two medications on their ability to lower blood pressure concludes no difference between them (p < 0.05). In reviewing the methods section you read that the investigators calculated their sample size by setting alpha at 0.05 and beta at 0.40.

Question 2 Which of the following is the best interpretation of the results of this study?

9/4/2014

4

Question 2 (Decision Errors)

Which of the following is the best interpretation

of the results of this study?

A. Type I error may have occurred, alpha should

be set at < 0.10

B. Type II error may have occurred, beta

should be set at < 0.20

C. Type I error may have occurred, alpha should

be set at < 0.01

D. Type II error may have occurred, beta should

be set at > 0.40

Vignette 3 (Confidence Intervals)

A randomized controlled trial reports that the difference in LDL-concentrations between a new HMG CoA reductase inhibitor and an existing one is 0.65%. The 95% confidence interval for this difference is (0.15% to 1.1%).

A randomized controlled trial reports that the difference in LDL-concentrations between a new HMG CoA reductase inhibitor and an existing one is 0.65%. The 95% confidence interval for this difference is (0.15% to 1.1%).

Question 3 Which of the following would be an appropriate interpretation of these results?

Question 3 (Confidence Intervals)

Which of the following would be an appropriate

interpretation of these results?

A. There is no statistically significant difference

between this drug and placebo (p > 0.05)

B. There is a statistically significant difference

between this drug and placebo (p < 0.05)

C. There is statistically significant difference between

this drug and placebo (p < 0.01)

D. The authors need to report a p-value to indicate if

the observed difference is statistically significant

Correlation and regression

Correlation examines the strength/degree of association between 2 variables

Correlation coefficient (r) ranges from -1 to +1

Regression analysis examines the ability of one or more variables to predict another variable

e.g., Simple/multiple linear regression, simple/multiple logistic regression, non-linear, polynomial,etc……

Y = mx+ b, CrCl from SeCr, etc….

Coefficient of determination (r2) can range 0 to 1.

An r2 of 0.80: 80% of the variability in Y is “explained”

by the variability in X

Survival Analysis

Studies the time between entry in a study

and some event (e.g., death, MI)

Kaplan-Meier method: Uses survival times to estimate

the proportion of people who would survive a length of

time

Log-Rank Test: Compare the survival distributions > 2

groups

Cox proportional hazards model

Impact of covariates on survival in > 2 groups

Allows calculation of a hazard ratio (and CI)

9/4/2014

5

Observational Study Design

Summary of Characteristics Study Design Measure of

Association

Major Advantages Major Disadvantages

Case Report/

Case Series

Generate new information about

natural history of Dz

ID new disease/condition

Usually can’t measure rates

of association

Case-control Odds Ratio Study relatively rare outcomes

Low cost and short duration

Not practical for studying rare

exposures

Inability to study multiple

outcomes in one study

Cohort Relative Risk;

Risk Ratio

Study relatively rare outcomes

Study temporal associations

Estimate direct risk estimates

Not practical for studying rare

exposures

Increased cost and longer

duration (prospective)

Cross-sectional Prevalence Low cost and short duration

Temporal associations can’t

be established

Adapted from Pharmacotherapy 2010;30:973-984

Relative vs. Absolute Differences

and NNT

NNT: The reciprocal of the ARR

NNT = 1/(ARR)…Rounded to the next highest

whole number

Medication Placebo Absolute

Difference

Relative

Difference

40% (4/10) 20% (2/10) 20% 50%

4% (4/100) 2% (2/100) 2% 50%

0.4% (4/1000) 0.2% (2/1000) 0.2% 50%

Relative vs. Absolute Differences

and NNT: Calculations

Medication Placebo Absolute Diff Relative Diff ARR NNT

40% (4/10) 20% (2/10) 20% 50% 0.2 5

4% (4/100) 2% (2/100) 2% 50% 0.02 50

0.4%

(4/1000)

0.2%

(2/1000) 0.2% 50% 0.002 500

Superiority: Detect a difference between Txs

Typical design in a clinical trial.

Equivalence: Confirm the absence of

meaningful difference(s) between Txs

What difference is important?

Non-inferiority: Investigate whether a Tx is

not clinically worse (no less effective)

May be the most effective, or have a similar effect.

If placebo is not possible due to ethical reasons

Superiority vs. Equivalence

vs. Non-inferiority

Vignette 4 (Correlation/Regression/Survival Analysis)

In working with some data concerning the occurrence of adverse effects when taking a new drug, you are deciding whether to carry out a correlation analysis or a regression analysis.

In working with some data concerning the occurrence of adverse effects when taking a new drug, you are deciding whether to carry out a correlation analysis or a regression analysis.

Question 4

Which of the following would be an appropriate consideration in making this decision?

9/4/2014

6

Question 4 (Correlation/Regression/Survival Analysis)

Which of the following would be an appropriate

consideration in making this decision?

A. Correlation analyses require more data than

regression analysis

B. Regression analyses are useful for detecting

only linear relationships between variables

C. Correlation analyses work best for data that

has been collected in retrospective trials

D. Regression analyses are useful in

predicting the value of one variable based

upon the value of another variable

Vignette 5 (Relative Risk/Odds Ratios)

A case-control trial (n = 102 cases, n = 650 controls) reports that the odds ratio (OR) for myocardial infarction (MI) is 0.7 (95% CI: 0.5, 0.9) in persons who have a history of receiving a certain vaccine in childhood.

A case-control trial (n = 102 cases, n = 650 controls) reports that the odds ratio (OR) for myocardial infarction (MI) is 0.7 (95% CI: 0.5, 0.9) in persons who have a history of receiving a certain vaccine in childhood.

Question 5

Which of the following represents the best interpretation of this result?

Question 5 (Relative Risk/Odds Ratios)

Which of the following represents the best

interpretation of this result?

A. This is a statistically significant difference; the vaccine is associated with a lower risk of MI

B. This is not a statistically significant difference; the vaccine is not associated with a lower risk of MI

C. This is a statistically significant difference; the vaccine decreases the risk of MI

D. This is not a statistically significant difference; the vaccine does not increase the risk of MI

Vignette 6 (Clinical Trials: Randomization Issues)

A randomized, double blind trial is conducted to determine if a new anti-ischemic drug improves symptoms in patients with stable angina and coronary artery disease. Patients are randomized to either the new drug or placebo, in additional to continuing existing standard medications. The intention-to-treat analysis shows no statistical difference in treatments. An actual treatment analysis showed that the new drug was more effective than placebo. Patients who did not take at least 70% of the study drug were reclassified as placebo for the actual treatment analysis.

A randomized, double blind trial is conducted to determine if a new anti-ischemic drug improves symptoms in patients with stable angina and coronary artery disease. Patients are randomized to either the new drug or placebo, in additional to continuing existing standard medications. The intention-to-treat analysis shows no statistical difference in treatments. An actual treatment analysis showed that the new drug was more effective than placebo. Patients who did not take at least 70% of the study drug were reclassified as placebo for the actual treatment analysis.

Question 6 Which one of the following is the best course of action based on these results?

9/4/2014

7

Question 6 (Clinical Trials: Analysis Issues)

Which one of the following is the best course of

action based on these results?

A. Recommend the new drug for all patients with HF

B. Recommend the new drug for patients who adhere

to their current therapies

C. Recommend the new drug only for early-stage HF

D. Do not recommend the new drug, wait for

further studies

Vignette 7 (Meta-analysis)

A meta-analysis of 6 randomized, placebo-controlled trials investigating the association between influenza vaccine and cardiovascular outcomes was performed, the results are shown in the figure below.

10 1 0.1

RR

A meta-analysis of 6 randomized, placebo-controlled trials investigating the association between influenza vaccine and cardiovascular outcomes was performed, the results are shown in the figure below.

Question 7 Which one of the following is the following is the best interpretation of these results, as shown in the figure?

10 1 0.1

RR

Question 7

(Meta-analysis)

Which one of the following is the following is the

best interpretation of these results, as shown in the

figure?

A. Influenza vaccine is not associated with a change in

the risk of major adverse CV events

B. Influenza vaccine is associated with a significantly

lower risk of major adverse CV events

C. Influenza vaccine is associated with a significantly

higher risk of major adverse CV events

D. Influenza vaccine’s effect on major adverse CV events

cannot be determined from the figure

Vignette 8 (Relative and Absolute Differences, NNT)

A trial comparing a new antibiotic to ciprofloxacin in the treatment of uncomplicated urinary tract infections shows that 88% of patients receiving the new drug were symptom-free at 3 days, while 82% of the ciprofloxacin patients were symptom-free at 3 days (p < 0.05).

A recent trial comparing a new antibiotic to ciprofloxacin in the treatment of uncomplicated urinary tract infections shows that 88% of patients receiving the new drug were symptom-free at 3 days, while 82% of the ciprofloxacin patients were symptom-free at 3 days (p < 0.05).

Question 8 How many patients would need to be treated with the new drug to have one additional patient symptom-free at 3 days?

9/4/2014

8

Question 8 (Relative and Absolute Differences, NNT)

How many patients would need to be treated with

the new drug to have one additional patient

symptom-free at 3 days?

A. 2

B. 6

C. 17

D. 167

Vignette R1 (Choosing Appropriate Statistical Tests)

A study compares the plasma concentrations of a new antiretroviral medication both with and without the concomitant administration of ganciclovir. Patients will take a fixed-dose of the new medication and have steady-state plasma concentrations evaluated before receiving ganciclovir and again 2 weeks later after having received ganciclovir for 48 hours.

A study compares the plasma concentrations of a new antiretroviral medication both with and without the concomitant administration of ganciclovir. Patients will take a fixed-dose of the new medication and have steady-state plasma concentrations evaluated before receiving ganciclovir and again 2 weeks later after having received ganciclovir for 48 hours.

Question R1 To determine whether there are differences in steady-state plasma concentrations, which test should be used to compare the results of this investigation?

Question R1 (Choosing Appropriate Statistical Tests)

To determine whether there are differences

in steady-state plasma concentrations,

which test should be used to compare the

results of this investigation?

A. Independent samples t-test

B. Chi-squared with Bonferroni correction

C. Paired t-test

D. Kruskal-Wallis ANOVA

Vignette R2 (Decision Errors)

You are planning a trial to compare the incidence of major depression recurrence with 12 months of antidepressant treatment compared with 24 months of treatment.

You are planning a trial to compare the incidence of major depression recurrence with 12 months of antidepressant treatment compared with 24 months of treatment.

Question R2 Which of the following strategies could be used to increase the power to detect any differences?

9/4/2014

9

Question R2 (Decision Errors)

Which of the following strategies could be used to increase the power to detect any differences?

A. Decrease the sample size

B. Decrease alpha

C. Decrease beta

D. Decrease the size of the difference you wish to detect

Vignette R3 (Confidence Intervals)

In a large (n=4218) randomized controlled trial a new drug is compared to placebo to assess its effect on raising HDL cholesterol. Compared to placebo the new drug is demonstrated to increase HDL cholesterol by 2 mg/dL, 95% CI (0.6, 3.4).

In a large (n=4218) randomized controlled trial a new drug is compared to placebo to assess its effect on raising HDL cholesterol. Compared to placebo the new drug is demonstrated to increase HDL cholesterol by 2 mg/dL, 95% CI (0.6, 3.4).

Question R3 Which of the following represents the best interpretation of such results?

Question R3 (Confidence Intervals)

Which of the following represents the best

interpretation of such results?

A. The results are neither statistically, nor clinically

significant (i.e., don’t use the drug)

B. The results are not statistically significant, but are

clinically significant (i.e., use the drug)

C. The results are statistically significant, but may not

be clinically significant

D. The results are statistically significant, and clinically

significant

Vignette R4 (Relative Risk/Odds Ratios)

A cohort study reports a relative risk (RR) of breast cancer of 1.4 (1.1,1.7) in women who reported having used an over-the-counter, herbal “antidepressant” in the past 5 years.

A cohort study reports a relative risk (RR) of breast cancer of 1.4 (1.1,1.7) in women who reported having used an over-the-counter, herbal “antidepressant” in the past 5 years.

Question R4 Which of the following is the best way to interpret these results?

9/4/2014

10

Question R4 (Relative Risk/Odds Ratios)

Which of the following is the best way to interpret these

results?

A. the herbal product should not be used, it increases

the risk of breast cancer substantially

B. the herbal product may be associated with

increased risk of breast cancer

C. the herbal product is not associated with any

increase in breast cancer risk

D. the herbal product appears safe and effective

Vignette R5 (Regression Analysis)

A study was conducted investigating the relationship between a the dose of a beta-agonist and FEV1. The results shown in the figure were: r=-0.46, p<0.05.

A study was conducted investigating the relationship between the dose of a beta-agonist and FEV1. The results shown in the figure were: r=-0.46, p<0.05.

Question R5 Which of the following represents the percent in the variability in FEV1 that is explained by the variability in the dose of the beta-agonist?

Question R5 (Regression Analysis)

Which of the following represents the percent in the

variability in FEV1 that is explained by the variability in

the dose of the beta-agonist?

A. 70 percent

B. 21 percent

C. 46 percent

D. 92 percent

Question R5-Follow-Up (Regression Analysis)

When reporting the results in the table as described,

was the reporting correct?

If not, which statistical parameter is more appropriate

to report in this situation?

Vignette R6 (Meta-analysis)

In reporting the results of a meta-analysis designed to assess the impact of pharmacist interventions to improve medication adherence, the authors report the results of their analyses testing their data for heterogeneity.

9/4/2014

11

In reporting the results of a met-analysis designed to assess the impact of pharmacist interventions to improve medication adherence, the authors report the results of their analyses testing their data for heterogeneity.

Question R6 Which of the following would be appropriately associated with such an analysis?

Question R6 (Meta-analysis)

Which of the following would be appropriately

associated with such an analysis?

A. The use of ANOVA

B. The reporting of a Χ2 or Cochrane’s Q test

C. A summary odds ratio

D. A regression analysis using multivariate techniques

Vignette R7 (Risk Reduction and NNT)

The results of a prospective, randomized, double-blind, placebo-controlled trial show that over a 6-month period 27/1232 patients receiving a medication required hospitalization for symptoms of asthma, while 42/1230 patients receiving the gold standard therapy required hospitalization for symptoms of asthma (p < 0.05).

The results of a prospective, randomized, double-blind, placebo-controlled trial show that over a 6-month period 27/1232 patients receiving a medication required hospitalization for symptoms of asthma, while 42/1230 patients receiving the gold standard therapy required hospitalization for symptoms of asthma (p < 0.05).

Question R7 Which of the following statements best represents these results?

Question R7 (Risk Reduction and NNT)

Which of the following statements best represents

these results?

A. Relative Reduction in events = 35%, NNT = 83

B. Relative reduction in events = 55%, NNT = 8

C. Relative reduction in events = 15%, NNT = 83

D. Relative reduction in events = 83%, NNT = 35

Vignette E1 (Approaches to Analyses)

In deciding an approach to statistical analysis, investigators wish to focus on demonstrating the maximum effectiveness of a new medication when it is used correctly.

9/4/2014

12

In deciding an approach to statistical analysis, investigators wish to focus on demonstrating the maximum effectiveness of a new medication when it is used correctly.

Question E1 In pursuing such a goal, which of the following types of analysis would be most appropriate?

Question E1 (Approaches to Analyses)

In pursuing such a goal, which of the

following types of analysis would be most

appropriate?

A. Intention-to-treat

B. Intention-to-randomize

C. Per-protocol

D. As-treated

Vignette E2 (Composite End-Points)

During your residency program you are presenting a journal club article regarding a new drug for hypertension. The primary outcome of the study is the occurrence of the combination of stroke, myocardial infarction (MI), and cardiovascular (CV) death.

During your residency program you are presenting a journal club article regarding a new drug for hypertension. The primary outcome of the study is the occurrence of the combination of stroke, myocardial infarction (MI), and cardiovascular (CV) death.

Question E2 In discussing this trial during your departmental journal club, which of the following would be appropriate comments regarding this type of end-point?

Question E2 (Composite End-Points)

In discussing this trial during your departmental journal club, which of the following would be appropriate comments regarding this type of end-point?

A. Composite end-points provide increased power, without any risks to trial validity or interpretability

B. Composite end-points create additional issues around multiple (statistical) testing

C. Composite end-points that combine non-fatal outcomes with death are the most robust

D. Composite end-points that include “soft” (i.e., subjective) end-points can be unduly influenced by one or more component end-points

Vignette E3 (Subgroup Analysis)

In a study of a new drug used to treat heart failure the authors describe several subgroups that were analyzed separately after the trial concluded based upon age, sex, and smoking history, history of stroke, and history of diabetes. They report that while the drug did not show effectiveness in the overall sample, that patients who were greater than 70 years of age did benefit from decreased hospitalizations due to HF (p < 0.05).

9/4/2014

13

In a study of a new drug used to treat heart failure the authors describe several subgroups that were analyzed separately after the trial concluded based upon age, sex, and smoking history, history of stroke, and history of diabetes. They report that while the drug did not show effectiveness in the overall sample, that patients who were greater than 70 years of age did benefit from decreased hospitalizations due to HF (p < 0.05).

Question E3 Which of the following statements represents the best interpretation of these results?

Question E3 (Subgroup Analysis)

Which of the following statements represents the

best interpretation of these results?

A. Patients over 70 years old with HF should

receive the medication

B. Any patient with HF should receive the

medication

C. Post-hoc subgroup analyses like this one,

should be considered hypothesis

generating

D. Any patient with HF and additional risk factors

should receive the medication

Vignette E4 (Descriptive Statistics)

A drug interaction study is conducted to assess the effects on the time to maximum concentrations (Tmax) of one drug when a p-glycoprotein inhibitor is taken concomitantly.

A drug interaction study is conducted to assess the effects on the time to maximum concentrations (Tmax) of one drug when a p-glycoprotein inhibitor is taken concomitantly.

Question E4 Which of the following would be the best way to report the results of this study?

Question E4 (Descriptive Statistics)

Which of the following would be the best way to

report the Tmax results of this study?

A. Mode and SEM

B. Mean and median

C. Median and range

D. Mean and standard deviation

Vignette E5 (Observational Study Designs)

You wish to investigate whether or not patients being treated pharmacologically for hypertension have an increased occurrence of memory loss. This hypothesis was suggested by a recent first-ever case-series published in the medical/pharmacy literature. You also hope to ascertain if certain medications or medication classes have more or less of this potential effect on memory.

9/4/2014

14

You wish to investigate whether or not patients being treated pharmacologically for hypertension have an increased occurrence of memory loss. This hypothesis was suggested by a recent first-ever case-series published in the medical/pharmacy literature. You also hope to ascertain if certain medications or medication classes have more or less of this potential effect on memory.

Question E5 Which of the following would be an appropriate strategy for investigating this hypothesis initially?

Question E5 (Observational Designs)

Which of the following would be an appropriate

strategy for investigating this hypothesis initially?

A. cohort trial

B. case-control trial

C. randomized, controlled trial

D. meta-analysis