Parts of the Research Study

Title, Abstract, Methodology, Results, Discussion

Even before the Title...

Where is study published? Respected journal? Is journal in same field as the research

study? Is journal peer reviewed? Was paper revised? Is is published in a journal of like

content?

Title

First potential source of biasIt should not state any conclusionsIt should reflect the actual content as

clearly and concisely as possibleIt should be consistent with the

abstract and summary

AuthorsEach author should have participated sufficiently in the work represented by the article to take public responsibility for the content. Conception or design, analysis or

interpretation of data Drafting the article or revising it for

critically important content, or in final approval

Participation only in data collection doesn’t qualify for authorship

Authors

Persons who contributed intellectually, but whose contributions do not justify authorship may be named separately.

Authors should list credentials for carrying out research.

Conflicts of interest should be noted.55% of articles today have multiple

authors

About Authors...

Reputable?Independent from drug company?Affiliated with research institutions?No conflicts of interest or bias?If funded by Drug Company, it should

be declared as suchReader bias…-- too much weight

given for credentials, big names, etc.

Abstract

Purpose: provide a brief summary of the research to help the reader determine if the article is worth reading in its entirety.

Some are structured abstracts and some are have restrictions about number of words used.

You cannot form a critical opinion of the study’s validity without reading the whole article.

Introduction

Contains the specific problem which exists

Rationale for the study with background material (review of literature)

Identifies the purpose of study = study objective- stated as a study hypothesis.

Should not contain bias or any results.

Introduction Should Include:

Statement of the importance of anticipated results from the study

Reasons for doing drug efficacy studies: there is no other effective treatment for cond. This drug is potentially superior to other

drugs Due to low SE, this would be a better choice Cost savings Pharmaceutical properties- tolerance, safety

Purpose of Study/Study Objective

Should be explicitly stated--you shouldn’t have to infer purpose

Is/are objective(s) reasonable?Are there too many objectives to be

answered in a single study?Will results measure the study

objective, ie. Are there valid endpoint measurements?

Study Objective

Describes anticipated relationships between factors to be studied

Specific and reasonable enough to study

Define clearly and exactly what the investigators are going to do

Relevant to what the investigators would like to determine

Stated as null or alternative hypothesis

Null Hypothesis

This assumes that there is no relationship between the factors to be studied and the outcome.

Is assumed to be true until proven otherwise.

Stated as: “There is NO difference between products”, or, “Both products are equal”

Alternative Hypothesis

Assumes that there is a relationship between the factor to be studied and the outcomes.

Two types of alternative hypothesis: one tailed: indicated the direction of the

relationship between the factor to be studied and the outcome

two tailed: indicates there is a relationship between the factor and outcome but doesn’t state the direction

Examples of Hypothesis

Null: Pravastatin is equivalent to Simvastatin in terms of lowering of cholesterol.

One tailed: Pravastatin is more effective than Simvastatin in lowering cholesterol.

Two tailed: Pravastatin and Simvastatin differ in their efficacy to lower cholesterol.

Methodology

Written so study could be repeated from the investigator’s description.

Includes design, patient selection criteria, sample size, inclusion/exclusion criteria, randomization, controls, blinding, etc.

Determines internal validity of study

Study Design

Study design guides evaluation methods:RCT: methods of treatment assignments,

blinding and controlsLongitudinal: duration of the follow upCrossover study: use and details of

washout periodRetrospective: methods to avoid recall

bias should be included

Validity

Related to precision and accuracyInternal validity: adequacy of the

study as a whole. Relies on study design, bias, and

random variationExternal validity: can results be

extrapolated to other settings Relies on inclusion/exclusion criteria

Internal Validity

A study has internal validity if the following have been done properly: Study design

controls, blinding

Methods of patient selection sample size, random sampling, inclusion/exclusion

criteria, external validity

Randomization Outcomes and endpoint measurements Statistical analysis

External Validity

External validity is determined by: inclusion criteria --

Are the study participants like your patient population, ie. Elderly, diabetic, CHF, etc.

exclusion criteria --Who is not included in study, ie. Diabetics,

elderly, CHF, etc.

both criteria are used to determine if results can be extrapolated to other settings.

Homogeneous Groups

Study groups are closely related in terms of important clinical characteristics or disease attributes

The more homogenous, the easier to identify and quantify the effects which a drug exerts

Increases the internal validity of the study.

Heterogenous groups

Patients differ in one or more identifiable clinical characteristics of the disease or condition being treated.

Acceptable when there aren’t enough patients who meet some narrowly defined inclusion criteria.

Acceptable when the results of the study won’t be affected by the differences

Inclusion Criteria

Characteristics patients must have to be eligible for participation in study

Homogeneous groups preferred--easiest to identify and quantify effects and increases internal validity of study.

Heterogeneous groups okay when results won’t be affected by the differences.

Exclusion Criteria

Characteristics which prohibit the patient from participating in the study

Examples: presence of other disease states, severity of disease, other medications/therapies affecting study results, patient safety, ethics, compliance.

Exclusion criteria helps ensure the study sample is homogenous.

Patient Selection Criteria

How many patients did they have in the study?

Is this number appropriate for the study design?

Does the study population represent the population from which it is drawn?

Was random sampling truly done?

Sample Size

Determined during initial planning stages of study

Need enough subjects to allow for significant differences between treatment groups to be detected statistically.

Need to balance statistical concerns with subject availability, cost, time constraints

Sample size considerations

RCT’s with small number of subjects may not be adequate to determine long term toxicity.

Other study designs may be needed based on the study sample size.

Sample size factors

Alpha or level of significance: the probability of obtaining a false positive result -- indicated as the “p-value”.

Beta: probability of false negative result--indicated as the “power”.

Delta: amount of difference that one wants to detect between groups

variance or standard deviation needed

Sample Size Factors

Investigator sets 4 factor levels, goes to table (or program) and selects appropriate sample size.

Very rough minimum, 30 patients needed for parallel study, 15 needed for crossover

Increasing sample size beyond certain point can lead to wasteful time and money--law of diminishing returns

Random Sampling

Selection of population into the study Each member of the population has the

same opportunity to be selected into the study.

Each is selected independently of anyone else.

Non-Random Sampling techniques to beware of

Consecutive non-random sampling: accept every patient who meets study criteria until a certain number is reached.

Convenience non-random sampling: select patients from a population which is easily or readily accessible.

Systematic non-random sampling: Every nth person is selected for study inclusion

Controls

What are investigators comparing the study drug/test to? Active control Placebo control No control Historical control

Active Control

Study drug is compared to another drug

Tells only “relative” efficacy Is study drug more, less or of equal

efficacy to comparison drug

Placebo Control

An inactive medicine without pharmacological effect.

Same dosage form and routeIt will contain small amount of sugar,

lactose or other inert substance which has no therapeutic action.

Can tell “actual” efficacyMinimizes bias, controls confounders

Ligation of Mammary Artery Trials

1940’s, double blinded, placebo controlled trial (sham operation vs. actual operation)

saved lives of many high risk patients from going through risky surgery which was not effective.

No Treatment Control

Refers to a group of patients in a study who do not receive any study drug or placebo

Tells “actual” efficacy Ethical concerns arise for placebo

and no treatment control groupsSalk polio vaccine trials in 1950’s

Historical Control

Utilizes a group of patients from who data have previously been collected.

Uses: effectiveness of surgical procedures, rare diseases, oncology studies

Disadvantages: inability to determine if control group was truly comparable, esp. when disease/condition can change over time.

BlindingOpen label

Both investigator and patient know treatment

Single blind Investigator knows who is receiving which

treatment, but patients don’t know what they are receiving.

Double blind Neither investigator and patient know

treatment

Keeping the study blinded

Make placebo look like active drug“Double Dummy”-- patients take 2

drugs each-- one placebo and one study drug.

RPh often involved with studies-- we keep investigators and patients blinded.

Unblinding can occur

Randomization

Refers to: assignment of patients to a treatment

group in a parallel or time series design Assignment of the order of treatments

in a crossover designPurpose of randomization in

assignment to groups -reduces bias, keeps groups balanced

Simple Randomization

Random numbers tablePulling names out of a hat

Systematic Randomization

Selecting a treatment group in which every “nth” person is selected for a treatment group

Acceptable if the starting point for selection (random sampling) is determined properly (randomly).

Block Randomization

Useful when using small numbers of patients

Ensures equal number of patients are randomized to each treatment group.

Cluster Randomization

The population is divided into natural groupings (geographical locations) and a random sample is selected from each group. A multicenter study across the U.S. All

lpatients from SE are divided into treatment and placebo groups, all from NE are divided, etc.

Stratified Randomization

Patients are assigned to subgroups, called strata, based on important characteristics called confounding factors. Then a separate randomization schedule for each stratum is chosen.

Useful when confounding factors will have large effect, and when small sample size

Non-Random Assignment

Potential Bias increasedMay use hospital admission numbers,

phone numbers, SS#, days of the week patients joined the study, etc.

Tendency to show larger treatment effects and increase the risk of false positive results

Results are difficult to evaluate--need multivariate modeling in statistical analysis

Outcome Measurements

Do measured endpoints match objective endpoints?

Are they measured correctly?Is statistical analysis done?

By independent investigator?

Results

Clearly presented and accurately reflect the study hypothesis.

Summary of study groups all patients should be accounted for reasons for missing data explained why drop-outs occurred Is length of study appropriate for study

objective?

Patient/Subject Drop-out Drop-outs change balance of “groups”Reasons for drop-outs can impact

results: Non compliance with study protocol development of side effects lack of efficacy subject was found not to meet inclusion

criteria developed another condition which

interfered Unavailable for follow up

How to Handle Data from Drop-outs

Intent-to-treat method: all data from all patients are included in analysis, regardless of whether or not their treatment was modified in any way

Exclusion of subjects method: patients are excluded from analysis if their treatment was modified in any way.

Intent-To-Treat

Advantage: reflects normal or actual clinical practice for a drug, in which patients are often started on a drug and later have their therapy altered.

Disadvantage: If large numbers of patients drop out or have therapy altered, the true efficacy of the drug itself will be obscured

Intent to Treat measurements:

Intent to treat method takes drop out patients and measures their scores by: A. Their last score or measurement at

the time they dropped out B. The average for the entire group C. The worst score or measurement for

the group.

Exclusion of Subjects

Advantage: The true efficacy of a drug in the regimen outlined in the study can be better determined.

Disadvantage: Patients can drop out of a study for reasons that can affect the usefulness of a drug in practice-- this method will not always reflect the actual clinical usefulness of a drug

Missing DataPatient completes study but one or more of their data measurements are missing

The greater the number of variables to measure in a study, the greater chance that certain data points will be missing

Missing data points are not significant if: only a small percentage of data points are

missing missing points occur by chance rather

than by a single factor

Options for Handling Missing Data

Dropping patients with incomplete data from study

Submitting the mean of the other scores or mathematically estimating the value for the missing point

Excluding the missing points for analysis

Types of Data

Raw data: actual measurements obtained

Derived data: measurements which have had some manipulations

Summary data: results which represent the combine data for all patients

Types of DataDerived data should be accompanied

by the raw data it was prepared from to allow for interpretation.

Summary data should be accompanied by the individual data to allow you to: fully evaluate how well it represents all

the patients determine if appropriate statistical

analysis performed repeat calculation

Outcomes Reported

Data presented must be: complete clear missing data must be explained and

accounted forResults section: determine whether a

study has fulfilled its objectives and proven or disproven its hypothesis

Tables and Graphs

ClearAccurateNot misleadingSimple

Things to watch for when analyzing the data:

Graphs with skewed vertical axis or no zero point

misleading line graphsTruncated bar graphsPercentagesColumns and rows not equaling 100%Sample size inflated

Percentages

Listed as: percent cure rate percent response rate percent of patients achieving desired

outcomePercent change can be misleading

without knowing baseline value.Exact amount of change most valuable

Sample Size

Artificially inflating sample size when repeated observations of a particular parameter are made and the author considers the total number of observations, and not the number of patients to be the sample size.

Discussion/Summary

Form your own conclusions before reading the Discussion/Summary

Watch out for persuasive languageWatch out for downplay of conflicting

evidenceStudy objectives have to be

consistent with results

What to watch out for in the Discussion/Summary

Author bias, reader bias Investigator interpretation of percentage

change or degree of change relative to control.

Biased citation or related publications Cause and effect relationship Errors in explaining a “non-significant p-

value” Statistical significance vs. clinical significance Quality of life vs. death as endpoints Inappropriate conclusions