How to Evaluate a Randomized Clinical Trial ( RCT ):

What Every Pharmacist Should Know

• Pharmacists often need to review primary literature to make patient specific recommendation or apply information to clinical practice.

• There are many forms of primary literature but RCT is the “gold standard” & most commonly used to assess drug efficacy.

• Bias may be introduced to any part of RCT either intentionally or unintentionally.

• Pharmacists should be able to evaluate all components & interpret RCT quality.

• Goals:1. Familiarize the pharmacist with components of RCT.2. Review sources of RCT bias.3. Help the pharmacist apply RCT findings to clinical

practice.

• No RCT is free of bias.• Components of an RCT:- Title- Abstract- Introduction- Methods- Results- Discussion- Conclusion/Summary- References** Each component provides important information

on overall quality of the RCT.

1. Title:Should :a. Be Succinct.b. Fairly & adequately represent content of the study.c. Not be too long.d. Stimulate the reader’s interest.e. Provide the design in the title is preferred ( i.e. Randomized or Blinding technique used). f. Not be confusing or misleading.

2. Abstract:• Summarizes only the important parts of :

- Goals- Methods- Results- Author conclusion

• Purpose: to provide enough information for the reader to decide if to read the article or not due to limited time.• Problem: if the clinician bases a clinical decision on reviewing just the abstract because clinical abstract can be misleading, incomplete, or incorrect .

• “Structured Abstract”:

-Best abstract format.

-Contains headers with specific information. -Many reputable journals use these structured

abstracts now.

• Structured Abstract contain:

1. Study Objectives2. Study Design3. Clinical Setting 4. Participants5. Methods6. Outcome Measures7. Results8. Conclusions

3. Introduction:• Provides background to disease in question.• Helps define research question or therapeutic dilemma.• Should clearly explain why the study was undertaken.• Should discuss prior research that led to current study.• Often contains references to other research.• Both opposing & supporting views should be

represented equally.• Objectives should be found in the introduction.

• Objectives: Impossible to evaluate an RCT without objectives.

• Objectives should include:-Clinical outcomes or endpoints.-How the author plans to measure the outcomes.(i.e. death , MI , post MI)- “Study time period”.

* The more specific the objectives & end points, the more applicable the study will be on the clinical

practice.

4. Methods: Should contain:*Actual steps taken to conduct the study.*Detailed description of: 1.Subject Selection

2.Randomizaton 3.Treatment Allocation

4.Blinding 5.Tests Used 6.Standards Used 7.Method & Route of Drug Administration

*All of which should be represented in a clear & organized way.

• Subject Selection:• Sample should represent population in all

characteristics that may have impact on efficacy & safety of drug.

• Eligibility Criteria: a. Inclusion Criteria (Conditions): Specific characteristics for enrollment & other drugs allowed.b. Exclusion Criteria:

Patients who may be harmed (i.e. pregnant or other diseases that may be affected by

outcomes.

• Randomization:* Each patient selected from the population has an

equal chance of being included or excluded.

*Common way : Random number generator ( computer program or random # table)

*Bias may be included in : “ Alternate Selection” “ Only from One Clinic” “Clinic Visits on Certain

Days”

• Allocation:• The random distribution of sample into treatment

groups.

• One treatment group should NOT be predisposed to failing or succeeding based on baseline characteristics.

• Demographics Table: May be used to:1. Assess appropriate allocation.2. Determine if sample selected

represents the population as a whole.

• Demographic table should include any objective patient characteristic that may affect the study outcome, i.e.: - gender - age - baseline lab values - other diseases - weight - ethnicity - other confounding variables

*May contain raw #s,means,% values,standard deviations

• Demographic table should represent differences & similarities among groups investigated.

• All patients enrolled & allocated should be similar in all characteristics.

• Researchers should analyze numbers & demonstrate that groups were statistically similar.

• Any statistical significant difference should be indicated.

• Blinding:( Masking)• 3 Types: 1. Unblinded2. Single-Blinded3. Double-Blinded

**May have a large impact on drug efficacy.

1. Unblinded: (Open)• Both investigator & participant know treatment assigned to.• Sometimes makes sense (e.g. in surgical procedures or

life-style modifications)• Advantages:

-Easier to execute-Investigator is more comfortable making

decisions regarding treatment continuation.• Disadvantages:

-Bias by both investigator & patient.-Investigator is more or less likely to consider treatment

effective if knows what the patient is receiving.-Investigator may think the patient is suffering side effects but it is a sign or symptom of disease.

2. Single-Blinded Technique:• Only investigator know but patients do NOT know

which treatment group the patient belongs to.• Disadvantages:

1. Investigators bias can NOT be ruled out.2. Investigator may attempt to treat patients

aggressively with other treatments “Compensation Therapy” which can change the outcomes & introduce bias.

3. Double-Blinded:• Most desirable.• Neither the investigator nor the patient

knows which treatment group the participant belongs to.

• Controls bias from both the investigator & patients especially some measurements of drug effectiveness are subjective.

• Control Arm Ethics:* It is unethical to assign subjects to placebo, inferior, or

ineffective active treatment.

* If so, should compare with a standard drug or active treatment ( e.g. new cephalosporin with a standard cephalosporin ).

* Sometimes, study is stopped when benefit or risk becomes significant in either arm.

• Placebo Matching:• Sometimes blinding technique is compromised, e.g. Zinc

Lozenges used in common colds. Patients may know its taste & the subjective symptoms may be biase in favor of the zinc lozenges.

• Should try to maintain blinding.

• Investigators should describe how the placebo matching was prepared & to what degree the placebo matched the treatment drug in appearance, odor, taste, & other physical characteristics.

• Outcome Measures:• Definition of outcome measures is important esp. if RCT is done at multiple centers.

Example: Comparing the effectiveness of SQ & LMWH in preventing DVT.

The diagnosis of DVT is considered the primary outcome measure. Venograms are

considered the standard for diagnosing DVT. Ultrasound is less specific. Same method

should be used in all centers participating in the study.

• Outcome measures should be:-Reliable-Specific-Objective

• Examples:

-Death-Length of hospital stay-Drug levels-ECG-Peak flow meter

• Baseline measures are important:e.g. If a study evaluates the effect of ginseng on reducing fasting blood sugarin diabetics without defining the baseline FBS, then the reduction in these levels would be difficult to interpret clinically (validity ?).

• Tracking Patients:• In a study, the author should account for every

patient assigned to treatment or placebo.

• The # of patients completing the study should match the # represented in the results section of the article.

• The # of drop outs ( withdrawals) should be described in detail.

• Possible Reasons for Withdrawals Include:-intolerable side effects-worsening disease-life-threatening complications-other medical reasons-lost interest with study monitoring-difficulty with transportation

• These patients may not return for clinic appointment & monitoring, and are sometimes “lost to follow-up”.

• Ignoring all patients who discontinued taking the drug is considered a BIAS.

• Many RCTs now use the “intent to treat analysis” which includes the results of all patients allocated to a treatment arm, even if they did not complete the study or stopped taking the study drug.

* In the method section, it should be mentioned

whether or not the results were analyzed on an “intent to treat analysis” basis.

4. Results:- Represents the study’s findings.- Should contain an objective narrative of the measured observations.- No author opinion is included in this section.- All information presented should be clearly explained.- Can use tables, figures, graphs which should be

clear, unbiased, parallel to narrative information.

• Clinical Significance:• Clinical significance is just as important as statistical significance.• Example: Drug X lowered mortality by 1% compared with Drug Y which

lowered mortality by 2%.Drug Y was associated with severe & sometimes debilitating side effects compared with Drug X which was not.While all the results were statistically significant, the 1% greater reduction in mortality associated with Drug Y is not

clinically significant.

• Side Effects:• Some side effects are seen with any drug.

• The authors should report & describe all side effects ( even with placebo ).

• Side effects may be assessed prospectively or retrospectively.

• Prospectively: more effective because the investigator can keep track of severity & frequency of the symptoms.

• Retrospectively: more prone to bias due to forgetting the details.

• Side effects may cause compliance bias which is difficult to overcome.

• Using the “intent to treat” analysis attempts to mimic real world conditions by accounting for drug discontinuation due to SEs & early study withdrawal.

6. Discussion: Should contain:

-Brief review of findings as they pertain to clinical practice.

-The clinical significance of the study.

-Explain why the study’s findings refute or support the work of others.

-Discuss problems that may have affected the results (i.e. changing lab for measuring blood

samples).-Explain any inconsistency, i.e.:

*Better or quicker response of some patients.*Response of some patients at lower dose than expected.

7. Conclusion:-Concise summary of author’s findings.-May be with discussion.-Should be:*Fully supported by data in study.*Factually correct.*Plausible*No sweeping generalizations.

• Before recommending any new treatment, consider other variables:-Lack of safety information.-Cost.-Additional monitoring.-Dosing frequency.-Side effects.-Drug interactions.-Patient acceptance.

8. References:• Should contain the most current citations with

the classic landmark articles on the topic.

* The investigators should cite themselves to a limited extent because citing themselves extensively can introduce bias by excluding other investigators’ findings.

• Not all RCTs are perfect: Each study will have positives & negatives.

• Identifying a study flaw is important.

• Identifying fatal flaws is crucial.

• Fatal flaw occurs when any RCT limits the applicability of the findings to any patient for whom the drug is designed.

• Examples of fatal flaws:*using inappropriate dose*high drop-out rate*inappropriate subject selection*inappropriate analysis of results